The Definitive Guide to Azure Data Engineering: Modern ELT, DevOps, and Analytics on the Azure Cloud Platform - Softcover

L'Esteve, Ron C.

Softcover

ISBN 10: 1484271815 ISBN 13: 9781484271810

Publisher: Apress, 2021

Synopsis

Build efficient and scalable batch and real-time data ingestion pipelines, DevOps continuous integration and deployment pipelines, and advanced analytics solutions on the Azure Data Platform. This book teaches you to design and implement robust data engineering solutions using Data Factory, Databricks, Synapse Analytics, Snowflake, Azure SQL database, Stream Analytics, Cosmos database, and Data Lake Storage Gen2. You will learn how to engineer your use of these Azure Data Platform components for optimal performance and scalability. You will also learn to design self-service capabilities to maintain and drive the pipelines and your workloads. 

The approach in this book is to guide you through a hands-on, scenario-based learning process that will empower you to promote digital innovation best practices while you work through your organization’s projects, challenges, and needs. The clear examples enable you to use this book as a reference and guide for building data engineering solutions in Azure. After reading this book, you will have a far stronger skill set and confidence level in getting hands on with the Azure Data Platform.

What You Will Learn

• Build dynamic, parameterized ELT data ingestion orchestration pipelines in Azure Data Factory
• Create data ingestion pipelines that integrate control tables for self-service ELT
• Implement a reusable logging framework that can be applied to multiple pipelines
• Integrate Azure Data Factory pipelines with a variety of Azure data sources and tools
• Transform data with Mapping Data Flows in Azure Data Factory
• Apply Azure DevOps continuous integration and deployment practices to your Azure Data Factory pipelines and development SQL databases
• Design and implement real-time streaming and advanced analytics solutions using Databricks, Stream Analytics, and Synapse Analytics
• Get started with a variety of Azure data services through hands-on examples

Who This Book Is For

Data engineers and data architects who are interested in learning architectural and engineering best practices around ELT and ETL on the Azure Data Platform, those who are creating complex Azure data engineering projects and are searching for patterns of success, and aspiring cloud and data professionals involved in data engineering, data governance, continuous integration and deployment of DevOps practices, and advanced analytics who want a full understanding of the many different tools and technologies that Azure Data Platform provides

About this title

About the Author

​Ron L’Esteve is a professional author residing in Chicago, IL, USA. His passion for Azure Data Engineering stems from his deep experience with implementing, leading, and delivering Azure Data projects for numerous clients. He is a trusted architectural leader and digital innovation strategist, responsible for scaling key data architectures, defining the road map and strategy for the future of data and business intelligence (BI) needs, and challenging customers to grow by thoroughly understanding the fluid business opportunities and enabling change by translating them into high quality and sustainable technical solutions that solve the most complex business challenges and promote digital innovation and transformation. Ron has been an advocate for data excellence across industries and consulting practices, while empowering self-service data, BI, and AI through his contributions to the Microsoft technical community.

From the Back Cover

Build efficient and scalable batch and real-time data ingestion pipelines, DevOps continuous integration and deployment pipelines, and advanced analytics solutions on the Azure Data Platform. This book teaches you to design and implement robust data engineering solutions using Data Factory, Databricks, Synapse Analytics, Snowflake, Azure SQL database, Stream Analytics, Cosmos database, and Data Lake Storage Gen2. You will learn how to engineer your use of these Azure Data Platform components for optimal performance and scalability. You will also learn to design self-service capabilities to maintain and drive the pipelines and your workloads. 

The approach in this book is to guide you through a hands-on, scenario-based learning process that will empower you to promote digital innovation best practices while you work through your organization’s projects, challenges, and needs. The clear examples enable you to use this book as a reference and guide for building data engineering solutions in Azure. After reading this book, you will have a far stronger skill set and confidence level in getting hands on with the Azure Data Platform.

You will learn to:

• Build dynamic, parameterized ELT data ingestion orchestration pipelines in Azure Data Factory
• Create data ingestion pipelines that integrate control tables for self-service ELT
• Implement a reusable logging framework that can be applied to multiple pipelines
• Integrate Azure Data Factory pipelines with a variety of Azure data sources and tools
• Transform data with Mapping Data Flows in Azure Data Factory
• Apply Azure DevOps continuous integration and deployment practices to your Azure Data Factory pipelines and development SQL databases
• Design and implement real-time streaming and advanced analytics solutions using Databricks, Stream Analytics, and Synapse Analytics
• Get started with a varietyof Azure data services through hands-on examples

From the Inside Flap

With the numerous cloud computing technologies being at the forefront of the modern day data architectural and engineering platforms, Microsoft Azure's cloud platform has contributed over 200 products and services that have been specifically designed to solve complex data challenges, empower self-service data engineering, and pave the way for the future of data and AI. Navigating through these many offerings in the Azure Data Platform can become daunting for aspiring Azure Data Engineers, architects, consultants, and organizations that are seeking to build scalable, performant, and production-ready data solutions. This book is intended to uncover many of the complexities within the Azure data ecosystem with ease through structured end-to-end scenario-based demonstrations, exercises, and reusable architectural patterns for working with data in Azure and building highly performant data ingestion and ELT pipelines. As Azure continues to introduce numerous data services to their ever-growing and evolving platform, this book will demystify many of the complexities of Azure Data Engineering with ease and introduce you to tried, tested, and production-ready patterns and pipelines for data of all different volumes, varieties, and velocities. Additionally, you will be introduced to the many capabilities of bringing value and insights to your data through real-time and advanced analytics, continuously integrating and deploying your data ingestion pipelines, and getting started with many Azure data services to help you progress through your journey within the Azure Data Engineering ecosystem.

Edition details

• Publisher: Apress
• Publication date: 2021
• Language: English
• ISBN 10: 1484271815
• ISBN 13: 9781484271810
• Binding: Paperback
• Edition number: 1
• Number of pages: 636