AI Systems Performance Engineering: Optimizing Model Training and Inference Workloads with GPUs, CUDA, and PyTorch

Elevate your AI system performance capabilities with this definitive guide to maximizing efficiency across every layer of your AI infrastructure. In today's era of ever-growing generative models, AI Systems Performance Engineering provides engineers, researchers, and developers with a hands-on set of actionable optimization strategies. Learn to co-optimize hardware, software, and algorithms to build resilient, scalable, and cost-effective AI systems that excel in both training and inference. Authored by Chris Fregly, a performance-focused engineering and product leader, this resource transforms complex AI systems into streamlined, high-impact AI solutions.

Inside, you'll discover step-by-step methodologies for fine-tuning GPU CUDA kernels, PyTorch-based algorithms, and multinode training and inference systems. You'll also master the art of scaling GPU clusters for high performance, distributed model training jobs, and inference servers. The book ends with a 175+-item checklist of proven, ready-to-use optimizations.

• Codesign and optimize hardware, software, and algorithms to achieve maximum throughput and cost savings
• Implement cutting-edge inference strategies that reduce latency and boost throughput in real-world settings
• Utilize industry-leading scalability tools and frameworks
• Profile, diagnose, and eliminate performance bottlenecks across complex AI pipelines
• Integrate full stack optimization techniques for robust, reliable AI system performance

Chris Fregly is a performance engineer and AI product leader who has driven innovations at Netflix, Databricks, Amazon Web Services (AWS), and multiple startups. He has led performance-focused engineering teams that built AI/ML products, scaled go-to-market initiatives, and reduced cost for large-scale generative-AI and analytics workloads. Chris is coauthor of the O’Reilly books Data Science on AWS and Generative AI on AWS, and creator of the O’Reilly course "High-Performance AI in Production with NVIDIA GPUs." His work spans kernel-level tuning, compiler-driven acceleration, distributed training, and high-throughput inference. Chris is the organizer of the global AI Performance Engineering meetup with over 100,000 members worldwide.