Optimizing Amazon EventBridge Pipes with Apache Kafka at ProSiebenSat.1
You need to be signed in to add a collection
ProSiebenSat.1 Media SE is one of the largest German television companies operating in the entertainment sector, offering various products. Joyn, their streaming application used in the DACH regions across multiple platforms. Serving millions of customers during major events, such as popular shows or live sports, is crucial, where customers expect a fast, seamless experience. Slow responses or unavailability leads to frustration, dissatisfaction, and loss of trust. In this presentation, we will guide you through the journey of optimizing Kafka within our system, starting from its initial status upon takeover. We will tell why we went with Confluent Kafka and not with Amazon MSK. We will look into integrating Kafka with Amazon EventBridge Pipes and share encountered limitations and issues. Next, we will introduce the specific architectural patterns such as the Claim-check and Point-to-point fan-out to enhance resilience and efficiency. The talk also outlines a wishlist for AWS, highlighting desired improvements in payload size, monitoring capabilities, and processing speed. We will end with lessons learned and strategies for leveraging Kafka more effectively in complex systems.
Transcript
ProSiebenSat.1 Media SE is one of the largest German television companies operating in the entertainment sector, offering various products. Joyn, their streaming application used in the DACH regions across multiple platforms. Serving millions of customers during major events, such as popular shows or live sports, is crucial, where customers expect a fast, seamless experience. Slow responses or unavailability leads to frustration, dissatisfaction, and loss of trust.
In this presentation, we will guide you through the journey of optimizing Kafka within our system, starting from its initial status upon takeover. We will tell why we went with Confluent Kafka and not with Amazon MSK. We will look into integrating Kafka with Amazon EventBridge Pipes and share encountered limitations and issues. Next, we will introduce the specific architectural patterns such as the Claim-check and Point-to-point fan-out to enhance resilience and efficiency.
The talk also outlines a wishlist for AWS, highlighting desired improvements in payload size, monitoring capabilities, and processing speed. We will end with lessons learned and strategies for leveraging Kafka more effectively in complex systems.
About the speakers
Team Lead of Content Discovery and AWS Serverless Community Builder
Specialist Solution Architect for Serverless