Designing NVIDIA AI Infrastructure: GPU compute, networking, orchestration, and security in NVIDIA's stack, explained

Navigate NVIDIA's enterprise AI infrastructure with confidence, from GPUs and data movement to orchestration, security, monitoring, edge systems, and model serving.

Key Features

• Build career-relevant knowledge of the NVIDIA AI infrastructure stack
• Make informed architecture decisions for performance, scalability, security, and cost
• Learn through practical configurations, deployment patterns, and enterprise case studies

Book Description

Designing NVIDIA AI Infrastructure is a concise reference guide for professionals who want to develop career-relevant knowledge of GPU-powered platforms without working through a lengthy manual.

The book explains how CPUs, GPUs, DPUs, storage, networking, software, and orchestration combine to support AI workloads. You will explore MIG and vGPU resource models, Kubernetes and Slurm scheduling, data pipelines, performance profiling, monitoring, TensorRT optimization, multi-tenant security, and governance. You will also learn how NVIDIA Jetson and Orin support edge AI and how NGC and Triton Inference Server contribute to model deployment and scalable serving.

Selected commands, configuration examples, architecture diagrams, and enterprise scenarios connect these technologies to operational contexts. By the end, you will be able to discuss the NVIDIA AI infrastructure stack with greater confidence, evaluate common design choices and bottlenecks, and use the book as a quick reference when planning cloud, on-premises, hybrid, and edge AI environments.

What you will learn

• Understand what MIG and vGPU isolate and what they don't
• Distinguish RBAC, network policy, and encryption's separate roles
• See how storage, NVLink, and InfiniBand affect GPU utilization
• Recognize where Kubernetes tools' responsibilities stop
• Understand how GDPR, HIPAA, and FedRAMP shape AI infrastructure controls and evidence
• Use GPU profiling and telemetry data to investigate bottlenecks
• Learn how NGC, Triton, and ensembles fit a serving pipeline
• Compare on-prem, cloud, and hybrid AI cluster trade-offs

Who this book is for

This book is for infrastructure engineers, ML and MLOps engineers, solutions architects, and technical leads who need a reliable mental model of NVIDIA’s AI infrastructure stack before designing, evaluating, or securing a GPU platform. It also suits professionals moving into AI infrastructure roles. Familiarity with Linux, containers, networking, cloud computing, or Kubernetes is helpful; advanced model-development knowledge and access to enterprise GPU hardware are not required.

Table of Contents

1. Foundations of AI Infrastructure
2. GPU Resource Management and Virtualization
3. Storage, Networking, and Data Pipelines for AI
4. AI Cluster Orchestration and Scalability
5. Performance Optimization and Monitoring
6. Security, Compliance, and Data Governance
7. Edge AI Infrastructure and Integration
8. NGC, Triton Inference Server, and Deployment
9. Real-World AI Infrastructure and Enterprise Workflows

Vivian Aranha is an AI educator, technology leader, and founder of School of AI, with over 20 years of industry experience. He earned a Bachelor's degree in Information Technology in 2004 and a Master's degree in Computer Science in 2006. His career spans web technologies, mobile app development for iOS and Android, blockchain solutions, and AI systems and applications. Vivian has worked with Fortune 500 organizations, including The Washington Post, Delta Air Lines, and IBM. An instructor since 2009, he has trained professionals worldwide and now teaches AI globally. His courses have attracted over 2.5 million enrollments, with more than 500,000 students learning through School of AI, Udemy, Skool, and Maven.