NVIDIA GPU Programming in CUDA and Python in 14 Days (Paperback)

Paperback. Fourteen days of drills on GPU programming, written in Python and measured on a real NVIDIA card. This is a drill manual, not a narrative course. Every one of the 463 listings was executed on a real GPU, and every printed output block and every printed timing is a paste from that run -- including the ones that came out level, and the ones that contradicted the theory. Each chapter states a rule in as few words as it can, shows the rule working in a short numbered example with its captured output, and hands it straight back as problems. The programming problems build one program across the whole book: gridsim, a grid relaxation that starts as a field of floats on the device in Chapter 1 and ends as a tuned simulation with a shared-memory stencil, a reduced convergence measure and a measured launch configuration. Most kernels here are written in Python with Numba's atcuda.jit. Where the C level is the lesson, the kernel is written in CUDA C and compiled at run time through CuPy, so a reader finishes able to read and write both and to move arrays between them without a copy. The fourteen chapters carry the whole working core. The first week covers the execution model and the first kernel, thread indexing in one and two dimensions, device memory and transfers, the CuPy array layer, honest timing and device properties, warps and divergence, and global memory access patterns. The second week covers shared memory and bank conflicts, synchronization and atomics, reduction and prefix sum, writing CUDA C kernels from Python, tiled matrix multiplication, streams and overlap, and occupancy and launch tuning. The measurements are reported as they came out. A tiled matrix multiply is 2.6 times the naive one and the library is still eight times faster than both. A pipeline in eight streams beats one stream by three per cent, not the forty the arithmetic promises. Raising occupancy by capping registers makes one kernel slower. A book that only printed the wins would be easier to write and worth less to work through. Your 14-Day Path 1. THE GPU AND THE FIRST KERNEL2. THREAD INDEXING3. DEVICE MEMORY AND TRANSFERS4. ARRAY PROGRAMMING WITH CUPY5. TIMING, ERRORS, AND DEVICE PROPERTIES6. WARPS AND DIVERGENCE7. GLOBAL MEMORY ACCESS PATTERNS8. SHARED MEMORY9. SYNCHRONIZATION AND ATOMICS10. REDUCTION AND SCAN11. WRITING CUDA C KERNELS FROM PYTHON12. MATRIX MULTIPLICATION AND TILING13. STREAMS, EVENTS, AND OVERLAP14. OCCUPANCY AND LAUNCH TUNING Who It's For Students on a parallel-computing or high-performance-computing course. Readers preparing for an examination or a technical interview. Working scientists and engineers whose NumPy code has outgrown the CPU. Python programmers who want the GPU without first learning C++. No prior CUDA is assumed; comfort with Python and NumPy is. 336 worked examples. 448 figures. 672 numbered problems. 380 pages. Measured on an RTX 4070 with CuPy 14.1 and Numba 0.67. You need an NVIDIA GPU and about an hour a day. Start Chapter 1. This item is printed on demand. Shipping may be from multiple locations in the US or from the UK, depending on stock availability.…