One approach to building supercomputers is to interconnect any number of common PCs using an interface like Ethernet. This method, called clustering or "Beowulf" clustering, produces very inexpensive powerful computers whose capabilities would previously have cost millions of dollars. Now for a few tens of thousands of dollars, very powerful computers can be built for any number of computing solutions. From scientific applications to transaction processing, clustering technology provides an affordable, scalable computing solution.Building Linux Clusters introduces the reader to the basics of cluster installation and configuration, and comes complete with a CD full of cluster installation programs and tools for parallel programming. Focusing on the "how to" of building a Linux cluster, this book is a hands-on guide for people new to clustering. It is the definitive guide to scaling Linux for scientific and enterprise applications.
Given that computers are a creation/projection of the human spirit, it is no surprise that they work against each other more often than not. Clients compete for the attention of servers, networks drain each other's bandwidth, and firewalls repel packet storms. But, just as system design mirrors our own greed, so too can it capture our Utopian dream that the whole is greater than the sum of the parts. Enter clusters, computer networks whose interconnectivity and communication protocols are interwoven so closely that the network can be used to solve single problems.
Author David Spector and the editors at O'Reilly achieve rare hacker-text synergy in recounting the adventure and in teaching the methods of networked hardware/software clusters in Building Linux Clusters, an extended how-to on coupling Linux boxes of all flavors (Alphas, Suns, 486 Intels, Pentiums) to work synchronously to compete with a multimillion-dollar supercomputer. Currently, the 62nd-fastest computer in the world is CPlant, a Linux cluster at Sandia National Labs (www.top500.org). The CPlant cluster is the equivalent of 1890 Intel-based Linux boxes that are running an expanded version of Don Becker's freely redistributable Beowulf platform for cluster operation.
The review of cluster building begins on hands and knees with an overview of networking basics: IP addressing and routing. Bandwidth and CPU-CPU timing requirements can be limiting factors; and, because interdependency is essential, proper design requires a weak-link analysis that establishes the compatibility of CPUs, buses, hard drives, Ethernet cards, hubs, switches, and routers. Strategies for cluster sizes from a few to several hundred are discussed.
In the book's second half, Spector turns his attention to cluster programming and applications, and describes tools, languages (where FORTRAN is still well regarded), libraries, and environments for parallel programming. Also, he gives examples of parallel virtual machines that serve MP3, persistence-of-vision graphics, and Web data to other devices or applications. Four brief appendices provide the essential technical details: an annotated Webography, a message-passing application programming interface, installation scripts for starting up the cluster of nodes at boot time, and a database to administer the activity of the nodes.
The fast pace and light pedantic touch in this book illuminate complexities and engender an excitement in the idea that new capabilities are yet to be found, if we all could just get along. --Peter Leopold
