MPICH2 User’s Guide
∗
Version 1.0.1b
Mathematics and Computer Science Division
Argonne National Laboratory
William Gropp
Ewing Lusk
David Ashton
Darius Buntinas
Ralph Butler
Anthony Chan
Rob Ross
Rajeev Thakur
Brian Toonen
June 10, 2005
∗
This work was supported by the Mathematical, Information, and Computational Sci-
ences Division subprogram of the Office of Advanced Scientific Computing Research, Sci-
DAC Program, Office of Science, U.S. Department of Energy, under Contract W-31-109-
ENG-38.
1
Contents
1 Introduction 1
2 Migrating to MPICH2 from MPICH1 1
2.1 Default Runtime Environment . . . . . . . . . . . . . . . . . 1
2.2 Starting Parallel Jobs . . . . . . . . . . . . . . . . . . . . . . 2
2.3 Command-Line Arguments in Fortran . . . . . . . . . . . . . 2
2.4 Configure Options . . . . . . . . . . . . . . . . . . . . . . . . 2
3 Setting Paths 3
4 Quick Start 3
5 Compiling and Linking 4
5.1 Specifying Compilers . . . . . . . . . . . . . . . . . . . . . . . 4
5.2 Shared Libraries . . . . . . . . . . . . . . . . . . . . . . . . . 4
5.3 Special Issues for C++ . . . . . . . . . . . . . . . . . . . . . . 5
5.4 Special Issues for Fortran . . . . . . . . . . . . . . . . . . . . 5
6 Running Programs with mpiexec 5
6.1 Standard mpiexec . . . . . . . . . . . . . . . . . . . . . . . . 6
6.2 Extensions for All Process Management Environments . . . . 6
6.3 Extensions for the MPD Process Management Environment . 7
6.3.1 Basic mpiexec arguments for MPD . . . . . . . . . . . 7
6.3.2 Other Command-Line Arguments to mpiexec . . . . . 8
6.3.3 Environment Variables Affecting mpiexec . . . . . . . 12
6.4 Extensions for SMPD . . . . . . . . . . . . . . . . . . . . . . 12
6.4.1 mpiexec arguments for SMPD . . . . . . . . . . . . . 12
i
6.5 Extensions for gforker . . . . . . . . . . . . . . . . . . . . . . 15
6.5.1 mpiexec arguments for gforker . . . . . . . . . . . . . 15
7 Managing the Process Management Environment 15
7.1 MPD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
8 Debugging 16
8.1 gdb via mpiexec . . . . . . . . . . . . . . . . . . . . . . . . . 16
8.2 TotalView . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
9 MPICH2 under Windows 20
9.1 Directories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
9.2 Compiling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
9.3 Running . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
ii
1 INTRODUCTION 1
1 Introduction
This manual assumes that MPICH2 has already been installed. For instruc-
tions on how to install MPICH2, see the MPICH2 Installer’s Guide, or the
README in the top-level MPICH2 directory. This manual explains how to
compile, link, and run MPI applications, and use certain tools that come
with MPICH2. This is a preliminary version and some sections are not
complete yet. However, there should be enough here to get you started with
MPICH2.
2 Migrating to MPICH2 from MPICH1
If you have been using MPICH 1.2.x (1.2.6 is the latest version), you will
find a number of things about MPICH2 that are different (and hopefully
better in every case.) Your MPI application programs need not change, of
course, but a number of things about how you run them will be different.
MPICH2 is an all-new implementation of the MPI Standard, designed to
implement all of the MPI-2 additions to MPI (dynamic process management,
one-sided operations, parallel I/O, and other extensions) and to apply the
lessons learned in implementing MPICH1 to make MPICH2 more robust,
efficient, and convenient to use.
2.1 Default Runtime Environment
In MPICH1, the default configuration used the now-old p4 portable pro-
gramming environment. Processes were started via remote shell commands
(rsh or ssh) and the information necessary for pro cesses to find and con-
nect with one another over sockets was collected and then distributed at
startup time in a non-scalable fashion. Furthermore, the entanglement of
process managment functionality with the communication mechanism led to
confusing behavior of the system when things went wrong.
MPICH2 provides a separation of process management and communica-
tion. The default runtime environment consists of a set of daemons, called
mpd’s, that establish communication among the machines to be used be-
fore application process startup, thus providing a clearer picture of what
is wrong when communication cannot be established and providing a fast
2 MIGRATING TO MPICH2 FROM MPICH1 2
and scalable startup mechanism when parallel jobs are started. Section 7.1
describes the MPD process management system in more detail.
2.2 Starting Parallel Jobs
MPICH1 provided the mpirun command to start MPICH1 jobs. The MPI-2
Forum recommended a standard, portable command, called mpiexec, for
this purpose. MPICH2 implements mpiexec and all of its standard argu-
ments, together with some extensions. See Section 6.1 for standard ar-
guments to mpiexec and various subsections of Section 6 for extensions
particular to various process management systems.
MPICH2 also provides an mpirun command for simple backward com-
patibility, but MPICH2’s mpirun does not provide all the options of mpiexec
or all of the options of MPICH1’s mpirun.
2.3 Command-Line Arguments in Fortran
MPICH1 (more precisely) MPICH1’s mpirun) required access to command
line arguments in all application programs, including Fortran ones, and
MPICH1’s configure devoted some effort to finding the libraries that con-
tained the right versions of iargc and getarg and including those libraries
with which the mpif77 script linked MPI programs. Since MPICH2 does not
require access to command line arguments to applications, these functions
are optional, and configure does nothing special with them. If you need
them in your applications, you will have to ensure that they are available in
the Fortran environment you are using.
2.4 Configure Options
The arguments to configure are different in MPICH1 and MPICH2; the
Installer’s Guide discusses configure. In particular, the newer configure
in MPICH2 does not supp ort the -cc=<compiler-name> (or -fc, -c++, or
-f90) options. Instead, many of the items that could be specified in the
command line to configure in MPICH1 must now be set by defining an
environment variable. E.g., while MPICH1 allowed
./configure -cc=pgcc
