Stop Wasting GPU Compute. Build the High-Throughput, Low-Latency AI Infrastructure of 2026.

The "VRAM Wall" is the biggest bottleneck in modern AI. Standard Python wrappers and out-of-the-box runtimes are fine for prototyping, but at scale, memory fragmentation and Global Interpreter Lock (GIL) overhead will destroy your throughput. LLM Inference in C++ is the definitive engineering manual for bypassing Python entirely and building custom, bare-metal inference engines that maximize hardware utilization.

Focusing on the cutting-edge 2026 landscape, this book bridges the gap between high-level AI concepts and low-level GPU execution. You will learn how to implement enterprise-grade features like PagedAttention, FlashAttention-3, and Continuous Batching directly in C++ and CUDA, unlocking massive performance gains for large-scale language models.
Inside, you will discover:

Hardware-Aware Memory Management: Eliminate memory waste by implementing PagedAttention logic and custom allocators to bypass std::malloc overhead.
Accelerated Tensor Algebra: Master C++23's std::mdspan and write fused SIMD kernels with AVX-512 to minimize GPU context switching.
Custom CUDA Kernels: Write high-speed FlashAttention-3, LayerNorm, and RMSNorm kernels while managing CUDA streams for maximum GPU occupancy.
The Cost Killer (Quantization): Slash VRAM requirements with bit-level manipulation for 4-bit (AWQ) and 8-bit (FP8) inference using NVIDIA Tensor Cores.
Distributed & Speculative Execution: Scale across clusters using zero-copy NCCL/RDMA interconnects and implement Draft Models to accelerate massive architectures.
The Production Serving Layer: Build lock-free C++ request queues for continuous batching and track P99 "Time to First Token" (TTFT) at the systems level.

THE IMPLEMENTATION VAULT (Appendix)

Built for the infrastructure engineer in the trenches, the Appendix provides immediate, battle-tested utility:

The 15-Point Production-Ready Checklist: Your mandatory safety and performance audit before deploying any custom engine.
Latency vs. Throughput Reference Table: The ultimate cheat sheet for balancing batch sizes against user wait times.
Troubleshooting Guide: Direct solutions for the top 10 most common and devastating CUDA and C++ memory errors.

Don't let inefficient software architecture throttle your hardware. Master C++ LLM inference and build the fastest, most cost-effective AI engines in the industry.

"Synopsis" may belong to another edition of this title.

PBShop.store UK

Fairford, GLOS, United Kingdom

5-star seller

AbeBooks seller since June 11, 1999

Shipping rates from United Kingdom to U.S.A.

Item	10 to 20 business days	10 to 20 business days
First item	US$ 5.63	US$ 5.63

Delivery times are set by sellers and vary by carrier and location. Orders passing through Customs may face delays and buyers are responsible for any associated duties or fees. Sellers may contact you regarding additional charges to cover any increased costs to ship your items.

Payment methods

Store description

**Please note our transit times are from the date of dispatch.** We first started out as “The Paperback Exchange,” a chain of physical bookstores where we would part exchange your beloved books for new stories to transport you to faraway places. However, as shopping started to evolve to online shops and marketplaces, we bid our stores goodbye to become “PBShop.” This transition has only allowed us to blossom as we now ship thousands of titles to book lovers across the globe. We pride ourselves in being a community of local book lovers which allows our passion and devotion to shine in everything we do. In 2020 we not only celebrated our 20th birthday but our 1st birthday as being completely employee owned after becoming an E.O.T in September 2019. We are proud to be different and embrace standing apart on a book mountain by working from a virtual inventory which allows us to provide thousands of books that may be difficult to get for your bookshelf or your studies. Working with a number of different suppliers allows us to explore other avenues such as puzzles, sheet music and even stationery so we really do have something for everyone. Life is about being versatile in all realms of existence. If this is your first purchase with us, or you are a returning customer, we would like to welcome you to the PBShop family, for there is no friend as loyal as a book. We are a company who put our customers at the centre of everything we do as we understand the importance of reading because once you learn to read, you will forever be free. There are a whole lot of things in this world of ours that we are yet to explore, which is why we will forever inspire curious minds.

Specialty

Hardbacks, Paperbacks

Seller's business information

Pbshop.co.uk

Unit 22 Horcott Industrial Estate, Horcott Road
Fairford, United Kingdom GL7 4BX

Contact sellerView this seller's itemsView seller's storefront

LLM Inference in C++

Item description from seller

Bibliographic details

Synopsis