MPICH2 User’s Guide
∗
Version 1.0.7
Mathematics and Computer Science Division
Argonne National Laboratory
William Gropp
Ewing Lusk
David Ashton
Pavan Balaji
Darius Buntinas
Ralph Butler
Anthony Chan
David Goodell
Jayesh Krishna
Guillaume Mercier
Rob Ross
Rajeev Thakur
Brian Toonen
April 2, 2008
∗
This work was supported by the Mathematical, Information, and Computational Sci-
ences Division subprogram of the Office of Advanced Scientific Computing Research, Sci-
DAC Program, Office of Science, U.S. Department of Energy, under Contract DE-AC02-
06CH11357.
1
Contents
1 Introduction 1
2 Migrating to MPICH2 from MPICH1 1
2.1 Default Runtime Environment . . . . . . . . . . . . . . . . . 1
2.2 Starting Parallel Jobs . . . . . . . . . . . . . . . . . . . . . . 2
2.3 Command-Line Arguments in Fortran . . . . . . . . . . . . . 2
3 Quick Start 2
4 Compiling and Linking 3
4.1 Specifying Compilers . . . . . . . . . . . . . . . . . . . . . . . 4
4.2 Shared Libraries . . . . . . . . . . . . . . . . . . . . . . . . . 4
4.3 Special Issues for C++ . . . . . . . . . . . . . . . . . . . . . . 4
4.4 Special Issues for Fortran . . . . . . . . . . . . . . . . . . . . 5
5 Running Programs with mpiexec 5
5.1 Standard mpiexec . . . . . . . . . . . . . . . . . . . . . . . . 5
5.2 Extensions for All Process Management Environments . . . . 6
5.3 Extensions for the MPD Process Management Environment . 6
5.3.1 Basic mpiexec arguments for MPD . . . . . . . . . . . 6
5.3.2 Other Command-Line Arguments to mpiexec for MPD 7
5.3.3 Environment Variables Affecting mpiexec for MPD . . 11
5.4 Extensions for SMPD Process Management Environment . . 12
5.4.1 mpiexec arguments for SMPD . . . . . . . . . . . . . 12
5.5 Extensions for the gforker Process Management Environment 15
5.5.1 mpiexec arguments for gforker . . . . . . . . . . . . . 15
i
5.6 Restrictions of the remshell Process Management Environment 17
5.7 Using MPICH2 with SLURM and PBS . . . . . . . . . . . . . 17
5.7.1 MPD in the PBS environment . . . . . . . . . . . . . 18
5.7.2 OSC mpiexec . . . . . . . . . . . . . . . . . . . . . . . 18
6 Managing the Process Management Environment 18
6.1 MPD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
7 Debugging 19
7.1 gdb via mpiexec . . . . . . . . . . . . . . . . . . . . . . . . . 19
7.2 TotalView . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
8 MPE 24
8.1 MPI Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
8.2 User-defined logging . . . . . . . . . . . . . . . . . . . . . . . 25
8.3 MPI Checking . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
8.4 MPE options . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
9 Other Tools Provided with MPICH2 28
10 MPICH2 under Windows 28
10.1 Directories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
10.2 Compiling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
10.3 Running . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
A Frequently Asked Questions 30
A.1 General Information . . . . . . . . . . . . . . . . . . . . . . . 30
A.1.1 Q: What is MPICH2? . . . . . . . . . . . . . . . . . . 30
A.1.2 Q: What does MPICH stand for? . . . . . . . . . . . . 30
ii
A.1.3 Q: Can MPI be used to program multicore systems? . 30
A.2 Building MPICH2 . . . . . . . . . . . . . . . . . . . . . . . . 31
A.2.1 Q: What is the difference between the MPD and SMPD
process managers? . . . . . . . . . . . . . . . . . . . . 31
A.2.2 Q: Do I have to configure/make/install MPICH2 each
time for each compiler I use? . . . . . . . . . . . . . . 31
A.2.3 Q: How do I configure to use the Absoft Fortran com-
pilers? . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
A.2.4 Q: When I configure MPICH2, I get a message about
FDZERO and the configure aborts . . . . . . . . . . . 33
A.2.5 Q: When I use the g95 Fortran compiler on a 64-bit
platform, some of the tests fail . . . . . . . . . . . . . 33
A.2.6 Q: When I run make, it fails immediately with many
errors beginning with “sock.c:8:24: mpidu sock.h: No
such file or directory In file included from sock.c:9:
../../../../include/mpiimpl.h:91:21: mpidpre.h: No such
file or directory In file included from sock.c:9: ../../../../in-
clude/mpiimpl.h:1150: error: syntax error before ”MPID VCRT”
../../../../include/mpiimpl.h:1150: warning: no semi-
colon at end of struct or union” . . . . . . . . . . . . . 34
A.2.7 Q: When building the ssm or sshm channel, I get the
error “mpidu process locks.h:234:2: error: #error ***
No atomic memory operation specified to implement
busy locks ***” . . . . . . . . . . . . . . . . . . . . . . 34
A.2.8 Q: When using the Intel Fortran 90 compiler (version
9), the make fails with errors in compiling statement
that reference MPI ADDRESS KIND. . . . . . . . . . . . . 34
A.2.9 Q: The build fails when I use parallel make . . . . . . 35
A.3 Windows version of MPICH2 . . . . . . . . . . . . . . . . . . 35
A.3.1 I am having trouble installing and using the Windows
version of MPICH2 . . . . . . . . . . . . . . . . . . . . 35
A.4 Compiling MPI Programs . . . . . . . . . . . . . . . . . . . . 35
iii
A.4.1 C++ and SEEK SET . . . . . . . . . . . . . . . . . . . 35
A.4.2 C++ and Errors in Nullcomm::Clone . . . . . . . . . 36
A.5 Running MPI Programs . . . . . . . . . . . . . . . . . . . . . 37
A.5.1 Q: How do I pass environment variables to the pro-
cesses of my parallel program . . . . . . . . . . . . . . 37
A.5.2 Q: How do I pass environment variables to the pro-
cesses of my parallel program when using the mpd
process manager? . . . . . . . . . . . . . . . . . . . . . 37
A.5.3 Q: What determines the hosts on which my MPI pro-
cesses run? . . . . . . . . . . . . . . . . . . . . . . . . 37
A.5.4 Q: On Windows, I get an error when I attempt to call
MPI Comm spawn. . . . . . . . . . . . . . . . . . . . . . 39
A.5.5 Q: My output does not appear until the program exits 39
A.5.6 Q: Fortran programs using stdio fail when using g95 . 40
A.5.7 Q: How do I run MPI programs in the background
when using the default MPD process manager? . . . . 40
iv
