Escali 4.4 사용자 설명서
Section: 2.2 SMC network devices
Scali MPI Connect Release 4.4 Users Guide
13
library, which in turn may (e.g. Myrinet or SCI) or may not require a kernel driver (e.g. TCP/IP).
These provider libraries provide a network device to SMC.
These provider libraries provide a network device to SMC.
2.2.1 Network devices
There are two basic types of network devices in SMC, native and DAT. The native devices are
built-in and are neither replaceble nor upgradable without replacing the Scali MPI Connect
package. There are currently five built in devices, SMP, TCP, IB, GM and SCI; the Release Notes
included with the Scali MPI Connect package should have more details on this issue.
built-in and are neither replaceble nor upgradable without replacing the Scali MPI Connect
package. There are currently five built in devices, SMP, TCP, IB, GM and SCI; the Release Notes
included with the Scali MPI Connect package should have more details on this issue.
To find out what network device is used between two processes, set the environment variable
SCAMPI_NETWORKS_VERBOSE=2. With value 2 the MPI library will print out during startup a
table over every process and what device it's using to every other process.
SCAMPI_NETWORKS_VERBOSE=2. With value 2 the MPI library will print out during startup a
table over every process and what device it's using to every other process.
2.2.1.1 Direct Access Transport (DAT)
The other type of devices use the DAT uDAPL API in order to have an open API for generic third
party vendors. uDAPL is an abbrevation for User DAT Provider library. This is a shared library
that SMC loads at runtime through the static DAT registry. These libraries are normally listed
in /etc/dat.conf. For clusters using ‘exotic’ interconnects whose vendor provides a uDAPL
shared object, these can be added to this file (if this isn’t done automatically by the vendor).
The device name is given by the uDAPL, and the interconnect vendor must provide it.
Please note that Scali has a certification program, and may not provide support for unknown
third party vendors.
party vendors. uDAPL is an abbrevation for User DAT Provider library. This is a shared library
that SMC loads at runtime through the static DAT registry. These libraries are normally listed
in /etc/dat.conf. For clusters using ‘exotic’ interconnects whose vendor provides a uDAPL
shared object, these can be added to this file (if this isn’t done automatically by the vendor).
The device name is given by the uDAPL, and the interconnect vendor must provide it.
Please note that Scali has a certification program, and may not provide support for unknown
third party vendors.
The DAT header files and registry library conforming to the uDAPL v1.1 specification, is
provided by the dat-registry package.
provided by the dat-registry package.
For more information on DAT, please refer to http://www.datcollaborative.org.
2.2.2 Shared Memory Device
The SMP device is a shared memory device that is used exclusively for intra-node
communication and use SYS V IPC shared memory. Mulit CPU nodes are frequent in clusters,
and SMP provide optimal communication between the CPUs. In cases where only one processor
per node is used, SMP is not used.
communication and use SYS V IPC shared memory. Mulit CPU nodes are frequent in clusters,
and SMP provide optimal communication between the CPUs. In cases where only one processor
per node is used, SMP is not used.
2.2.3 Ethernet Devices
An Ethernet for networking is a basic requirement fpr a cluster. For some uses this also has
enough performance for carrying application communication. To serve this Scali MPI Connect
has a TCP device. In addition there are Direct Ethernet Transport (DET) devices which
implement a protocol devised by Scali for aggregating multiple TCP-type interconnects.
enough performance for carrying application communication. To serve this Scali MPI Connect
has a TCP device. In addition there are Direct Ethernet Transport (DET) devices which
implement a protocol devised by Scali for aggregating multiple TCP-type interconnects.
2.2.3.1 TCP
The TCP device is really a generic device that works over any TCP/IP network, even WANs. This
network device requires only that the node names given to mpimon map correctly to the nodes
IP address. TCP/IP connectivity is required for SMC operation, and for this reason the TCP
device is always perational.
network device requires only that the node names given to mpimon map correctly to the nodes
IP address. TCP/IP connectivity is required for SMC operation, and for this reason the TCP
device is always perational.
Note: Users should always append the TCP device at the end of a devicelist as the device of
last resort. This way communication will fall back to the management ethernet that anyway has
to be present for the cluster to work.
last resort. This way communication will fall back to the management ethernet that anyway has
to be present for the cluster to work.