Extreme-Scale Computing

Scientific computing is essential for tackling complex problems across many domains—but how can scientists develop high-performance and high-quality software that scales efficiently? This book serves as an accessible introduction to extreme-scale computing, specifically designed for domain scientists who may not have formal computer science training but need to harness the power of C++ and parallel computing for large-scale applications.

The book begins by covering the fundamentals of scientific computing software management, including essential tools like Linux, Git, and CMake, before diving into a detailed exploration of C++ for extreme-scale computing. Readers familiar with languages like Python will gain the necessary skills to transition to C++ and build scalable, efficient software.  Beyond basic programming, this book delves into hardware-aware computing, teaching readers how to optimize software performance by understanding the underlying architecture of modern computational systems. It then introduces parallel computing techniques, covering MPI for distributed memory parallelism, shared memory parallelism, CUDA for GPU programming, and Kokkos for performance portability.

Further chapters focus on efficient I/O, debugging, and profiling, which all address aspects of the critical challenge of performance optimization in extreme-scale computing. The book concludes with an overview of popular libraries for extreme-scale computing, equipping readers with the tools they need to solve real-world computational problems.  With a balance of theory, practical applications, and illustrative case studies, this book provides domain scientists with a comprehensive roadmap to mastering extreme-scale computing and developing highly parallel and performant software.

Raymond J. Spiteri is Professor of Computer Science at the University of Saskatchewan. His research interests include numerical analysis, scientific computing, and extreme-scale computing. He has served as President of the Canadian Applied and Industrial Mathematics Society as well as Natural Sciences and Engineering Research Council of Canada Group Chair of the Evaluation Group for Mathematics and Statistics. He has received the CAIMS–Fields Industrial Mathematics Prize, the Mitacs Award for Exceptional Leadership — Professor, and the Arthur Beaumont Distinguished Service Award.

Kyle Klenk is a Research Systems Analyst at the University of Saskatchewan. His research interests include scientific computing, concurrency, information systems, and extreme-scale computing. He is an early career scientist who has worked under Raymond J. Spiteri as a Research Assistant for 3 years. He currently works for the Computer Science department at the University of Saskatchewan where he provides research and IT support.