Kafka Connect: Build and Run Data Pipelines

Used by more than 80% of Fortune 100 companies, Apache Kafka has become the de facto event streaming platform. Kafka Connect is a key component of Kafka that lets you flow data between your existing systems and Kafka to process data in real time.

With this practical guide, authors Mickael Maison and Kate Stanley show data engineers, site reliability engineers, and application developers how to build data pipelines between Kafka clusters and a variety of data sources and sinks. Kafka Connect allows you to quickly adopt Kafka by tapping into existing data and enabling many advanced use cases. No matter where you are in your event streaming journey, Kafka Connect is the ideal tool for building a modern data pipeline.

• Learn Kafka Connect's capabilities, main concepts, and terminology
• Design data and event streaming pipelines that use Kafka Connect
• Configure and operate Kafka Connect environments at scale
• Deploy secured and highly available Kafka Connect clusters
• Build sink and source connectors and single message transforms and converters

Mickael Maison is a committer and member of the Project Management Committer (PMC) for Apache Kafka. He has been contributing to Apache Kafka and its wider ecosystem since 2015. Mickael is a software engineer with over 10 years of software development experience. While working at IBM, he was part of the Kafka team that runs hundreds of Kafka clusters for customers. He is now working in the Kafka team at Red Hat and has accumulated a lot of expertise about Kafka Connect.

In addition, Mickael has developed and contributed to several connectors for Connect. He also has deep expertise in Connect’s internals as he has made a number of code contributions to Connect itself and regularly reviews pull requests from the community on this component.

Finally, Mickael really enjoys sharing expertise and teaching. He has been writing monthly Kafka digests since 2018 and enjoys presenting at conferences.

Kate Stanley is a software engineer, technical speaker and Java Champion. She has experience running Apache Kafka on Kubernetes, developing enterprise Kafka applications and writing connectors for Kafka Connect.

Kate currently works as a Principal Software Engineer across the Red Hat Kafka offerings. She also contributes to multiple projects in the Kafka ecosystem, including the open-source Kafka operator, Strimzi. Kate started her journey with Kafka as part of the Event Streams team at IBM in 2018, quickly becoming well known in the community.

Alongside development, Kate has a passion for presenting and sharing knowledge. She is a regular speaker at technical conferences around the world, including events such as Kafka Summit, Jfokus, Devoxx UK and JavaOne. She has authored two LinkedIn Learning courses on MicroProfile and Kafka and written an eBook on writing microservices with Java.