The Sourcebook of Parallel Computing.pdf

SOURCEBOOK

OF PARALLEL

COMPUTING

JACK DONGARRA

University of Tennessee

IAN FOSTER

Argonne National Laboratory

GEOFFREY FOX

Indiana University

ANDY WHITE

Los Alamos National Laboratory

Senior Editor Denise Penrose

Publishing Services Manager Edward Wade

Production Editor Howard Severson

Editorial Coordinator Emilia Thiuri

Cover Design Frances Baca

Text Design Detta Penna

Illustration Dartmouth Publishing, Inc.

Composition Windfall Software, using ZzT

X

Designations used by companies to distinguish their products are often claimed as trademarks

or registered trademarks. In all instances in which Morgan Kaufmann Publishers is aware of a

claim, the product names appear in initial capital or all capital letters. Readers, however, should

contact the appropriate companies for more complete information regarding trademarks and

registration.

Morgan Kaufmann Publishers

An imprint of Elsevier Science

340 Pine Street, Sixth Floor

San Francisco, CA 94104-3205

www.mkp.com

prior written permission of the publisher.

Library of Congress Control Number: 2002107244

ISBN: 1-55860-871-0

This book is printed on acid-free paper.

During its 11-year lifetime, the Center for Research on Parallel Computation (CRPC),

a National Science Foundation (NSF) Science and Technology Center, was focused on

research and technology development that would be needed to make parallel com-

puting “truly usable.” Over that period, a remarkable number of applications were

converted to work on scalable parallel computers, and an equally remarkable num-

together a collection of works covering both applications and the technologies used

to convert them to run on scalable parallel systems.

The principal goal of this book is to make it easy for newcomers to the field

of parallel computing to understand the technologies available and how to apply

them. The book is intended for students and practitioners of technical computing

who need to understand both the promise and practice of high-performance parallel

computation. It assumes that the reader has a good background in either applications

or computational science, but not in parallel computation. Thus, the book includes

a significant amount of tutorial material.

We also intend that the book serve as a useful reference for the practicing parallel

gained over the past fifteen years, while avoiding some of the pitfalls that we

ourselves encountered in gaining that knowledge.

The book is organized into five major parts. Part I, entitled Parallelism, is a tutorial

introduction to the field of parallel computers and computing, with separate chapters

providing a broad overview of the field, an in-depth treatment of the architecture of

modern parallel computer systems, and a survey of the issues that should be taken

into consideration when programming them.

performance computation can be applied in a variety of specific application areas. It

both scientific areas and numerical approaches. The applications are cross-referenced

to the material in later chapters that cover the needed software technologies and

algorithms in depth.

logical strategies for application development, including message-passing libraries;

run-time libraries for parallel computing, such as class libraries for HPC++, languages

like HPF, Co-Array Fortran, and HPC++; performance analysis and tuning tools such

as Pablo; and high-level programming systems. The goal of this part is to provide a

survey of progress with hints to the user that will help in selecting the right tech-

nology for use in a given application.

of important future problems for the high-performance science and engineering

community, including distributed computing in a grid environment.

This book can be used in several different ways. The newcomer to parallel compu-

tation seeking a tutorial introduction should read all of Part I, along with Chapters

4, 9, 16, 17, and 25. Chapters 4, 9, and 17 provide overviews of Parts II, III, and IV,