Data Engineering by Diagram: A Visual Field Guide to the Modern Data Stack

Every data engineering concept you need is a picture waiting to be drawn. Batch against streaming. ETL against ELT. Star against snowflake. Bronze, silver, gold. Read about them and they blur together. See them and they stay.

This is a visual field guide to the modern data stack. Thirty-four diagrams carry the explanation, and the prose is there to walk you through each one — not the other way round. Every figure is drawn in a single visual language and designed to survive black-and-white print, so the book works the same on a screen, on paper, and photocopied onto a whiteboard.

It is vendor-neutral by design. You will learn the shapes that stay true whether your stack runs on Databricks, Snowflake, Fabric or a laptop.

Twenty-two short chapters take you from the source system to the dashboard:

• What a data engineer actually does, and where the job starts


• Batch and streaming, full loads, incremental loads and change data capture


• Databases, ACID and the write-ahead log


• Warehouse, lake, lakehouse and lakebase — what each one solved


• Columnar file formats and open table formats


• ETL and ELT, and why the order matters more than the letters


• Distributed computing from HDFS and MapReduce to Spark


• Idempotency, watermarks, upserts and backfill


• Data quality and testing


• OLTP, OLAP, normal forms and dimensional modeling


• Star schemas, snowflake schemas and slowly changing dimensions


• Medallion architecture


• Orchestration and the DAG


• Lambda and Kappa streaming architectures


• Data mesh, data fabric and who owns what


• A production stack, end to end

A diagram index at the back lets you find the one picture you need in seconds, which makes the book as useful the week before an interview as it is the week you start the job.

Written for the analyst moving into engineering, the software engineer inheriting a pipeline, and the engineer who can build the thing but wants the whole map in an afternoon.