[PDC Center for High Performance Computing]

PDC Reference Reading List

Note: special thanks to NERSC for providing a starting point to this list.

PDC consultants often get asked for recommendations for reading material for independent study. Most such questions fall into one of the areas below. While we don't officially recommend any book or other source, the lists below contain books we know of, some of which we are familiar with; where appropriate, brief descriptions are included, along with comments on their usefulness.

Users are invited to submit their own reviews, by email to pdc-staff@pdc.kth.se. We will review each one carefully, and consider it for inclusion below.


Books We Warn Against Computing Science The Unix Operating System
Numerical Methods Programming and Languages Parallel Computing
Useful Journals Other Useful Papers  

Books We Warn Against

  • Numerical Recipes

    More than one mathematician has been heard to recommend against using any of the books in this series, due to lack of rigor in their development, and lack of stability, robustness, or modernity in their presented algorithms and methods. Since there isn't a single book that covers everything NR covers and does it better, one mathematician suggests that searchers go to the netlib.org web site for specific software, then refer to a numerical analysis textbook for details. He also suggests the newsgroup sci.math.num-analysis, and the ACM journal Transactions on Mathematical Software (TOMS); most if not all of the software reported in TOMS is on the netlib.org site. A second mathematician claims to be unsatisfied with the code presented in Numerical Recipes, but says the book can be used as a source of basic ideas, as long as they're thoroughly researched elsewhere.

Computing Science

  • The Art of Computer Programming; by Donald E. Knuth; Addison-Wesley, 1997.

    This "book" is actually a three volume set (with a fourth in preparation) that covers a great deal of the elegant, useful, technical content of computing science. The volumes are titled Fundamental Algorithms, Seminumerical Algorithms, and Sorting and Searching. This corpus is generally enough useful to benefit even computational scientists, and should certainly be consulted for ideas on ways to improve data handling incidental to technical computing.

  • Structure and Interpretation of Computer Programs; by Harold Abelson, Gerald Jay Sussman, and Julie Sussman; MIT Press, 1984.

    This book may seem, on first sight, to be all about Lisp (or more correctly, Scheme) programming, but in fact it's about all of what we call computer science. It's used as a text for a first (or early) course in CS in many places, but in fact has enough depth to drive some graduate work. It's a very good text for learning about how to think about programs, programming, and program execution, but as such is more intended for those for whom such matters are the ends rather than the means. Still, if you're interested in what CS is really about, this is the book you'll find it in.

The Unix Operating System

  • UNIX in a Nutshell: System V Edition, 3rd Edition; by Arnold Robbins; O'Reilly, 1999.
  • The Unix C Shell Field Guide; by Gail Anderson and Paul Anderson; Prentice Hall, 1986.

    This is a useful and fast-reading text on the most popular shell now in use in Unix, the basic C Shell. It can be used as both text and reference, although it is aimed more at the former. As a text, it can easily be used, in a few hours of effort and accompanied by a basic text on Unix, as a bootstrap into using the most common features of the Unix operating system. A very useful book.

  • Using csh & tcsh; by Paul DuBois; O'Reilly, 1995.

Numerical Methods

  • Numerical Recipes - The Art of Scientific Computing; by William H. Press, Brian P. Plannery, Saul A. Teukolsky, and William T. Vetterling; Cambridge University Press, 1986.

    This book, which comes in both Fortran and C versions, is widely recommended for use as a source of basic ideas in the field of computational numerics. It contains a lot of code, illustrating and implementing the mathematics. However, as noted elsewhere on this web page, much if not all of the code is suspect, and some of the specific methods are less than trustworthy. It's probably best used as a preface to investigation in more authoritative and specialized sources. It's generally agreed that any serious numerical work should be preceded by a perusal, if not deep study of What every computer scientist should know about floating-point arithmetic, listed below.

Programming and Languages

  • Fortran 90 Explained; by Michael Metcalf and John Reid; Oxford Science Publications, 1990.

    This book is neither the worst nor best of a bad lot. Fortran 90 tends to be difficult to learn, because little advice is given to learners on how best to apply its advanced features. This book is no exception. It's more of a reference than a text, and has a decent index, but finding out exactly how to use some parts of the expanded language is still difficult. Better than nothing.

  • Upgrading to Fortran 90; by Cooper Redwine; Springer, 1995.
  • The C Programming Language - ANSI C, Second Edition; by Brian W. Kernighan and Dennis M. Ritchie; Prentice Hall, 1988.

    These authors invented the C language, and this book is the upgrade of their classic text and reference dealing with the ANSI standard version of C. As such, it's the bible, and belongs on any C user's bookshelf. However, if you're new to programming, you should also find a good text, as this book is quite terse - it wastes no words on in-depth explanations or good examples.

  • C - A Reference Manual, Third Edition; by Samuel P. Harbison and Guy L. Steele; Prentice Hall, 1991.

    This is another language reference manual, with expanded discussion of some areas, from the standpoint of language design. The authors are well-known experts in programming language design and implementation, and they speak with authority on such matters. The book is also good to have because it includes discussion of standard library functions and the C Preprocessor, topics often missing or given short-shrift in other books. Not a text, but another good reference.

  • Practical C Programming, 3rd Edition; by Steve Oualline; O'Reilly, 1997.
  • Using MPI - Portable Parallel Programming with The Message-Passing Interface; by William Gropp, Ewing Lusk, and Anthony Skjellum; MIT Press, 1996.

    This is the original text on the standard Message Passing library now available on all distributed memory systems. The authors are among the initiators of this standard, and the text is a gentle introduction to the concepts of communication among cooperating parallel processors. It provides, in seven chapters, a smooth ramp-up into the use of MPI. This volume is a companion to the MPI reference, and between them constitute the required spanning set of information on this standard. NOTE: There's a new edition of this book out, now, that includes all the new stuff in MPI 2.0, even though MPI 2.0 isn't widely available, yet.

  • The C++ Programming Language, Third Edition; by Bjarne Stroustroup; Addison-Wesley, 1990.

    This is the most authoritative reference to the C++ language, written by the inventor and single person most responsible for the standard as it now exists. The book can be used as a text, but is probably not for the faint of heart or those new to programming in C. Its 800 pages describe the C++ language and library in detail. This edition supersedes earlier ones, due to changes in the language standard. It's considered the encyclopedia of C++. It can be argued that C++ is the wrong language in which to write high performance parallel programs, due to the complexity of managing C++ objects. However, it is undeniably growing in popularity as a software engineering tool, for developing libraries and user interfaces, and it has probably done more for the cause of reusable code than any other single development in computing.

  • Software Portability with imake, 2nd Edition; by Paul DuBois; O'Reilly, 1996.
  • Managing Projects with make, 2nd Edition; by Andrew Oram and Steve Talbott; O'Reilly, 1991.
  • The Mythical Man-Month, Anniversary Edition; by Frederick P. Brooks; Addison-Wesley, 1995.

    This book is about large software projects and the ways they can go awry. The author was the manager of the development of IBM's OS/360 operating system for its System/360 computer line, in the early 1960's. The book is so famous and widely recognized that it's achieved a mythic status of its own. Managers of large software projects should read and learn from its wisdom.

Parallel Computing

  • High Performance Computing, 2nd edition; by Kevin Dowd & Charles Severance; O'Reilly, 1998.

    This book is as close to a "Nutshell" book on this topic I've found. It contains four chapters of basics on high performance computing, followed by four more on software matters - compilers, profiling, and optimizations; three chapters are devoted to shared memory systems, and three more to "scalable" or larger-scale parallel systems (distributed memory or message-based ones); and finally, two chapters discuss benchmarking. It ends with five appendices on various related matters. The text is succinct and plainly written, aimed at the serious (but not necessarily very experienced) professional practitioner, and constructed so that most of the chapters are stand-alone in presentation. It is both a text and a reference. It's a very good introductory book.

  • MPI - The Complete Reference, Second Edition (two volumes); by Marc Snir, Steve W. Otto, Steven Huss-Lederman, David W. Walker, and Jack Dongarra; MIT Press, 1997.

    This is the second edition of the original, most authoritative reference for the MPI standard. It consists of two volumes. Volume 1, The MPI Core, contains everything that was in the single-volume first edition, with corrections and C++ bindings. Volume 2, The MPI-2 Extensions, contains extensions added to MPI in version 2 of the standard. Together, they are an encyclopedic description and definition of the MPI standard and its use in parallel programs. They are not a text, however, and those wanting a tutorial introduction to MPI would be advised to use the companion text, Using MPI, reviewed above. NOTE: Since MPI 2.0 isn't widely available, yet, if you have the first edition of this book, it will probably suffice for a while; if you don't, the second edition is a better choice; the first may no longer be available.

  • Parallel Computer Architecture - a Hardware/Software Approach; by David E. Culler and Jaswinder Pal Singh, with Anoop Gupta; Morgan Kaufmann, 1999.

    This book's twelve chapters and over 1000 pages covers the entire range of architectural practices that have been used in multiprocessor systems, since they first appeared. While it claims coverage of software as well as hardware, it offers mainly deep coverage of the hardware aspects of parallel computing as taught to graduate engineering students. As such, it does as complete a job as any text I've seen. It represents most all that a serious practitioner of parallel computer engineering should have mastered, and somewhat more, I believe, than most computational scientists will be interested in knowing. However, the first chapter contains a very good introduction to parallel computing in general.

  • Designing and Building Parallel Programs; by Ian Foster; Addison Wesley, 1994.

    This books covers both the techniques used to design parallel programs and the tools used to implement these programs. It contains a thorough discussion of parallel algorithm design, performance analysis, program construction. Languages covered include MPI, HPF, and C++. Tools for collecting and analyzing performance data are covered. Last but not least, an on-line version of this book is available via the Web. It includes tools and programs that you can download.

  • Concurrent Scientific Computing; by Eric F. Van de Velde; Springer-Verlag, 1994.

    This book, one of the Texts in Applied Mathematics (TAM) series, is a more advanced book focuses on the algorithmic structure of numerical methods. You'll find the collection of algorithms in this book to be sufficiently representative of almost every fundamental technique of concurrent scientific computing.

  • Introduction to Grid Computing with Globus; by Luis Ferreira et al; IBM RedBook, Dec 2002.

    This book, which can be downloaded in its entirety, contains nearly 300 pages on Grid Computing and Globus with demos and applications.

Useful Journals

  • The ACM Transactions on Mathematical Software (TOMS)

    TOMS publishes articles about algorithms for solving numerical computing problems, usually including either a pseudo-code description of the algorithm or real software (usually in Fortran). Most (if not all) of the software that appears in TOMS is available for download from the Netlib repository (www.netlib.org, netlib.bell-labs.com, ftp.netlib.org).

  • Lecture Notes in Computational Science and Engineering (LNCSE)

    This series includes theoretical aspects of scientific computing such as mathematical modeling, discretization techniques, multiscale approaches, fast solution algorithms, parallelization and visualization methods as well as the application of these approaches throughout the disciplines of biology, chemistry, physics, engineering, earth sciences, and economics. Volume 13 is the proceedings of the seventh annual PDC conference, Simulation and Visualization on the Grid.

  • Computing and Visualization in Science

    This relatively new journal has set its goal on publishing pioneering methods and applications which precipitate the solution of complex problems in the computational sciences and visualization. Material is available both in hardcopy and online in postscript or PDF.

Other Useful Papers

  • Twelve Ways to Fool the Masses When Giving Performance Results on Parallel Computers; by David H. Bailey; Supercomputing Review, Aug. 1991, pg. 54--55.

    Many of us in the field of highly parallel scientific computing recognize that it is often quite difficult to match the run time performance of the best conventional supercomputers. This humorous article outlines twelve ways commonly used in scientific papers and presentations to artificially boost performance rates and to present these results in the "best possible light" compared to other systems.

  • What every computer scientist should know about floating-point arithmetic; by David Goldberg; ACM Computing Surveys, Vol. 23, No. 1, March, 1991, pg 5-48.

    Floating-point arithmetic is considered an esoteric subject by many people. This is rather surprising, because floating-point is ubiquitous in computer systems: Almost every language has a floating-point datatype; computers from PCs to supercomputers have floating-point accelerators; most compilers will be called upon to compile floating-point algorithms from time to time; and virtually every operating system must respond to floating-point exceptions such as overflow. This paper presents a tutorial on the aspects of floating-point that have a direct impact on designers of computer systems. It begins with background on floating-point representation and rounding error, continues with a discussion of the IEEE floating point standard, and concludes with examples of how computer system builders can better support floating point.

Changed by: $Author: smeds $, $Date: 2004/12/07 15:15:27 $