Escali Escali, LLC Network Router 4.4 User Manual

Page of 81
Section: 3.7 Debugging and profiling
Scali MPI Connect Release 4.4 Users Guide 
30
3.7.2 Built-in-tools for debugging
Built-in tools for debugging in Scali MPI Connect covers discovery of the MPI calls used through 
tracing and timing, and an attachment point to processes that fault with segmentation 
violation. The tracing and timing is covered in Chapter 4.
3.7.2.1 Using built-in segment protect violation handler
When running applications that terminate with a SIGSEGV-signal it is often useful to be able 
to freeze the situation instead of exiting, the default behavior. The built-in SIGSEGV-handler 
can be made to do this by defining the environment-variable 
SCAMPI_INSTALL_SIGSEGV_HANDLER
Legal options are:
6. The handler dumps all registers and starts looping. Attaching with a debugger will then 
make it possible to examine the situation which resulted in the segment protect violation. 
7. The handler dumps all registers but all processes will exit afterwards.
All other values will disable the installation of the handler.
To attach to process <pid> on a machine with the GNU debugger (gdb) do; 
 
user% gdb /proc/<pid>/exe <pid>
 
In general, this will allow gdb to inspect the stack trace and identify the functions active when 
the sigsegv occurred, and disssasemble the functions. If the application is compiled with 
debug info (-g) and the source code is available, then source level debugging can be carried 
out. 
3.7.3 Assistance for external profiling
Profiling parallel applications is complicated by having multiple processes at the same time. But 
Scali MPI Connect comes to assistance; through the SCAMPI_PROFILE_APPLICATION 
environment variable together with the -separate_output option (SCAMPI_SEPARATE_OUTPUT) 
the output from the application runs is directed at one file per process for easier use.
The environment variables SCAMPI_PROFILE_APPLICATION_START and 
SCAMPI_PROFILE_APPLICATION_END are also available for steering the range of memory 
addresses applicable to profiling.
3.7.4 Debugging with Etnus Totalview
SMC applications can be debugged using the Etnus Totalview, see http://www.etnus.com 
for more information about this product.
To start the Etnus Totalview debugger with a Scali MPI application, use the tvmpimon wrapper 
script. This wrapper script accepts the regular options that mpimon accepts, and sets up the 
environment for Totalview. The totalview binary must be in the search path when launching 
tvmpimon. If the mpirun script is the preferred way of starting jobs, it accepts the “standard” 
-tv option, however the same rules applies with regards to the search path.