Escali Escali, LLC Network Router 4.4 User Manual
Section: 2.3 Communication protocols on DAT-devices
Scali MPI Connect Release 4.4 Users Guide
16
2.2.6 SCI
This is a built-in device that uses the Scali SCI driver and library (ScaSCI). This driver is for the
Dolphin SCI network cards. Please see the ScaSCI Release Notes for specific requirements. This
device is straight forward and requires no configuration itself, but for multi-dimensional toruses
(2D and 3D) the Scali SCI Management system (ScaConf) needs to be running somewhere in
your system. Refer to Appendix C for installation and configuration of the Scali SCI
Management software.
Dolphin SCI network cards. Please see the ScaSCI Release Notes for specific requirements. This
device is straight forward and requires no configuration itself, but for multi-dimensional toruses
(2D and 3D) the Scali SCI Management system (ScaConf) needs to be running somewhere in
your system. Refer to Appendix C for installation and configuration of the Scali SCI
Management software.
2.3 Communication protocols on DAT-devices
In SMC, the communication protocol used to transfer data between a sender and a receiver
depends on the size of the message to transmit, as illustrated in Figure 2-3:.
depends on the size of the message to transmit, as illustrated in Figure 2-3:.
Figure 2-3: Thresholds for different communication protocol
The default thresholds that control whether a message belongs to the inlining, eagerbuffering
or transporter protocols can be controlled from the application launch program (mpimon)
or transporter protocols can be controlled from the application launch program (mpimon)
described in chapter 3.
Figure 2-4: illustrates the node resources associated with communication and mechanisms
implemented in Scali MPI Connect for handling messages of different sizes. The three
communication protocols from Figure 2-3: rely on buffers located in the main memory of the
nodes. This memory is allocated as shared, i.e., it is not private to a particular process in the
node. Each process has one set of receiving buffers for of the processes it communicates with.
As the figure shows all communication relies on the sending process depositing messages
directly into the communication buffers of the receiver. For Inline and Eagerbuffering the
management of the buffer resources does not require participation from the receiving process,
because of their designs as ring buffers.
implemented in Scali MPI Connect for handling messages of different sizes. The three
communication protocols from Figure 2-3: rely on buffers located in the main memory of the
nodes. This memory is allocated as shared, i.e., it is not private to a particular process in the
node. Each process has one set of receiving buffers for of the processes it communicates with.
As the figure shows all communication relies on the sending process depositing messages
directly into the communication buffers of the receiver. For Inline and Eagerbuffering the
management of the buffer resources does not require participation from the receiving process,
because of their designs as ring buffers.
2.3.1 Channel buffer
The Channel ringbuffer is divided into equally sized entries. The size varies differs for different
architectures and networks; see “Scali MPI Connect Release Notes” for details. An entry in the
ringbuffer, which is used to hold the information forming the message envelope, is reserved
each time a message is being sent, and is used by the inline protocol, the eagerbuffering
protocol, and the transporter protocol. In addition, one ore more entries are used by the inline
protocol for application data being transmitted.
architectures and networks; see “Scali MPI Connect Release Notes” for details. An entry in the
ringbuffer, which is used to hold the information forming the message envelope, is reserved
each time a message is being sent, and is used by the inline protocol, the eagerbuffering
protocol, and the transporter protocol. In addition, one ore more entries are used by the inline
protocol for application data being transmitted.
Transporter protocol:
message size > eager_size
message size > eager_size
Eagerbuffering protocol:
channel_inline_threshold < message size <= eager_size
channel_inline_threshold < message size <= eager_size
Inlining protocol
0 <= message size <= channel_inline_threshold
0 <= message size <= channel_inline_threshold
Increasing
message size