Kafka Connect: Build and Run Data Pipelines
Senior Principal Software Engineer at Red Hat & Co-Author of "Kafka Connect"
Principal Software Engineer at Red Hat & Co-Author of "Kafka Connect"
The book by Mickael Maison and Kate Stanley offers a hands-on guide to Kafka Connect, a crucial tool within Apache Kafka that enables seamless data streaming between various systems and Kafka. With Kafka Connect, organizations can integrate data from multiple sources and destinations to enable real-time data processing, a capability that over 80% of Fortune 100 companies rely on Kafka for. This guide is designed for data engineers, site reliability engineers, and application developers, providing them with practical insights on:
Kafka Connect fundamentals: Learn its features, core concepts, and terminology. Pipeline design: Understand how to structure data and event streaming pipelines using Kafka Connect. Environment management: Configure and scale Kafka Connect setups to handle large volumes of data. Secure and reliable deployments: Implement Kafka Connect clusters with high availability and strong security. Connector development: Build source and sink connectors, as well as custom message transformations and data converters. This resource aims to equip professionals at any stage of their Kafka journey with the tools to build modern, efficient data pipelines.
BOOK EPISODE
Kafka Connect: Build and Run Data Pipelines
Danica Fine together with the authors of “Kafka Connect” Kate Stanley and Mickael Maison, unpack Kafka Connect's game-changing power for building data pipelines—no tedious custom scripts needed! Kate and Mickael Maison discuss how they structured the book to help everyone, from data engineers to developers, tap into Kafka Connect’s strengths, including Change Data Capture (CDC), real-time data flow, and fail-safe reliability. They dig into Kafka Connect’s customization options, share tips on keeping pipelines robust in production, and reveal some powerful tweaks for those looking to get advanced. Throughout, they emphasize how Kafka Connect’s open-source community makes learning and using it easier than ever, inviting readers to jump in and start building smarter data pipelines.
Watch the video
