Building Trusted Data Platforms with Azure Databricks and GenAI: A Hands-On Guide to Creating Governed Data Products in a Lakehouse - Softcover

Manoj Kukreja

Softcover

ISBN 10: 1806679779 ISBN 13: 9781806679775

Publisher: Packt Publishing, 2026

Synopsis

A practical guide to building a modern, GenAI-powered data platform with a Lakehouse foundation, covering MDM, data mesh, AI enablement, streaming pipelines, observability, and cloud-driven architectures for trusted analytics.

Key Features

• Discover characteristics of future-ready platforms - data mesh, automation, & observability
• Design trustworthy data products with contracts, federated governance, and decentralized ownership
• Understand how GenAI accelerates Lakehouse development and enables self‑service analytics

Book Description

Discover the defining hallmarks of future‑ready data platforms, including data mesh architectures, intelligent automation, and end‑to‑end data observability. Learn how to design and deliver trusted data products through data contracts, federated governance, decentralized domain ownership, and endorsed datasets. The book explores modern Lakehouse patterns with a strong focus on the medallion architecture, explaining how bronze, silver, and gold layers transform raw data into analytics‑ready assets governed through Unity Catalog. You’ll gain practical guidance on MDM linkages, survivorship rules, and entity resolution to ensure consistent master data across domains. It also covers real‑time and streaming pipelines that integrate seamlessly with the Lakehouse. We focus on self‑service analytics, showing how governed data products let business users explore, analyze, and derive insights independently with confidence. Finally, understand how GenAI accelerates platform development through automated code generation using tools like Claude Code and Databricks Genie Code, enabling faster pipeline creation, governance, and analytics delivery.

What you will learn

• Future‑ready platforms: data mesh, automation, observability
• Design trusted data products with contracts and governance
• Build Lakehouses with medallion architecture: bronze, silver, gold
• Apply Unity Catalog for governance and endorsed datasets
• Implement MDM using linkages, survivorship, and entity resolution
• Develop real‑time and streaming pipelines at scale
• Enable governed self‑service analytics for business users
• Use GenAI to generate code with Claude and Databricks Genie

Who this book is for

This book is crafted for aspiring data and AI/ML architects, engineers and analysts starting their data engineering journey and seeking a practical, hands‑on guide to building scalable, cloud‑driven data platforms. It’s ideal for professionals familiar with PySpark who want to design modern Lakehouse architectures using Delta Lake, while learning MDM, data mesh, AI enablement, streaming pipelines, automation, and data observability. A working knowledge of Python, Spark, and SQL is expected.

Table of Contents

1. The Story of Data Engineering and Analytics
2. Discovering Data Platforms
3. Data Engineering on Microsoft Azure
4. Designing Future Data Platforms
5. Understanding Databricks
6. Understanding Medallion Architecture and Delta Lake
7. Understanding Modern Data Pipelines
8. Data Ingestion Stage: Building the Landing Zone
9. Building the Bronze Layer of the Medallion Architecture using Gen AI
10. Building the Silver Layer of the Medallion Architecture using Gen AI
11. Building the Next-Gen Data Analytics Gold Layer
12. Governing Data Products

About this title

About the Author

Manoj is a Cloud Data Architect and GRC Consultant with deep expertise in data engineering, data governance, regulatory compliance, and modern cloud platforms. Across a career spanning enterprise strategy, large‑scale architectures, governance frameworks, and ML/AI adoption, he has led transformative initiatives that help organizations build trusted, secure, compliant, and scalable data ecosystems. His work integrates regulatory rigor, covering privacy, protection, and AI governance, with practical engineering clarity to ensure organizations meet global standards while accelerating innovation. As a mentor and active contributor to the technology community, he brings a practitioner's precision and a strategist's vision to the rapidly evolving world of data and AI.

He is the author of a best‑selling book on modern data architecture, and this publication marks his second book, further deepening his thought leadership in the field. Manoj has worked with major public & private sector organizations, including US & Canadian government agencies. A CISSP‑certified professional, he brings a security‑first, governance‑led approach to data platforms, specializing in MDM, Data Mesh, Lakehouse architectures, streaming, automation, observability, and AI enablement. He also serves as a data & AI governance advisor to several data-driven organizations.

Edition details

• Publisher: Packt Publishing
• Publication date: 2026
• Language: English
• ISBN 10: 1806679779
• ISBN 13: 9781806679775
• Binding: Paperback
• Edition number: 2
• Number of pages: 774