Streams
Showing 7 out of 7 results
Kafka in Action
Kafka has been on developers’ radars for quite a while now. Viktor Gamov’s co-authored book “Kafka in Action” ensures that you have a list of recipes to dive into. Joined by Tim Berglund, VP DevRel at StarTree, they explore the fundamentals of Apache Kafka. Learn what Kafka can help you achieve, what Viktor’s favorite MCU film is and what “Highway to Mars” by Beast In Black has to do with all of this.
Deep Learning with Apache Spark
Apache Spark is an amazing framework for distributing computations in a cluster in an easy and declarative way. With it becoming a standard across industries so it would be great to add the amazing advances of Deep Learning to it. There are parts of Deep Learning that are computationally heavy, very heavy! Distributing these processes may be the solution to this problem, and Apache Spark is the easiest way Favio could think to distribute them. Here Favio will talk about Deep Learning Pipelines an open source library created by Databricks that provides high-level APIs for scalable deep learning in Python with Apache Spark and how to distribute your DL workflows with Spark.
Batching vs. Streaming - John Deere's Journey to Scale & Process Millions of Measurements a Second
The John Deere data platform receives and processes millions of sensor measurements per second from machines around the world.<br /> In this talk, Adam will discuss the various technologies and architectures John Deere has implemented and how they had to adapt to ever increasing volumes of data. Adam will explore their usage of Lambda/Kinesis, Spark and Flink...where they worked, where they didn't, lessons learned and how their current architecture is a blend of all three.
Observability for Data Pipelines: Monitoring, Alerting, and Tracing Lineage
Data-intensive applications, with many layers of transformations and movement from different data sources, can often be challenging to maintain even after they are initially built and validated. To truly expand and develop a code base, developers must be able to test confidently during the development process and monitor the production system. Monitoring and testing data pipelines or real-time streaming processes can be very different from monitoring web services. Jiaqi draws on her experience building and maintaining both batch and real-time stream data pipelines to discuss how to leverage monitoring tools like Prometheus and Grafana to define and visualize metrics, how and when to alert on common health indicators, and how to gain visibility in monitoring not just the system health but the health of the data. General concepts she touches on include observability of pipeline health, interpretability of data results and building features into data pipelines that makes monitoring and testing just a little bit easier, such as the ability to trace data lineage and designing for immutable data.
Beyond Microservices: Streams, State and Scalability
Microservices have been a popular architecture choice for at least 5 years by now. Over these years we've adopted microservices architectures to ever growing set of use-cases and different development and deployment strategies. Lessons were learned and our ability to design, develop, deploy and operate microservices has improved. This presentation will give an opinionated view of how microservices evolved in the last few years, based on experience gained while working with companies using Apache Kafka to update their application architecture. We'll discuss the rise of API gateways, service mesh, state management and serverless architectures - what works well, and in which cases. We'll show real-world examples of how applications become more resilient and scalable when new patterns are introduced, and make sure to include caveats - because patterns are all about using them in the right context.
Batching vs. Streaming - John Deere's Journey to Scale & Process Millions of Measurements a Second
The John Deere data platform receives and processes millions of sensor measurements per second from machines around the world.<br /> In this talk, Adam will discuss the various technologies and architectures John Deere has implemented and how they had to adapt to ever-increasing volumes of data. Adam will explore their usage of Lambda/Kinesis, Spark and Flink...where they worked, where they didn't, lessons learned and how their current architecture is a blend of all three. **In this talk you'll learn:** * The inner workings of the John Deere data platform * How to adapt to ever-increasing volumes of data * Pros and Cons of using Lambda/Kinesis, Spark and Flink
Lunch & Roundtable Discussions
Don't miss your chance to share, learn, and network with speakers and attendees! Grab your lunch and join us at one of the roundtables. **Teams & Inclusion** Join Bryan Cantrill and Sara Caldwell in a vibrant conversation about building inclusive, high-performing teams in today's diverse tech landscape. Share your experiences and learn from others on fostering collaboration and embracing diversity in the workplace. **Programming with AI** Unlock the potential of AI in software development with Alex Castrounis and Linda Rising. Delve into the intersection of programming and artificial intelligence, discussing techniques, tools, and breakthroughs. Engage in thought-provoking conversation and exchange ideas. **Data Streams** Embark on a deep dive into data streams with Mary Grygleski and Kasun Indrasiri. Share and explore the latest in streaming technologies, architectures, and use cases. Share your experiences and learn from others in this dynamic roundtable on the cutting edge of data processing.