Batching vs. Streaming - John Deere's Journey to Scale & Process Millions of Measurements a Second
You need to be signed in to add a collection
The John Deere data platform receives and processes millions of sensor measurements per second from machines around the world.<br /> In this talk, Adam will discuss the various technologies and architectures John Deere has implemented and how they had to adapt to ever increasing volumes of data. Adam will explore their usage of Lambda/Kinesis, Spark and Flink...where they worked, where they didn't, lessons learned and how their current architecture is a blend of all three.
Transcript
The John Deere data platform receives and processes millions of sensor measurements per second from machines around the world.
In this talk, Adam will discuss the various technologies and architectures John Deere has implemented and how they had to adapt to ever increasing volumes of data. Adam will explore their usage of Lambda/Kinesis, Spark and Flink...where they worked, where they didn't, lessons learned and how their current architecture is a blend of all three.
About the speakers
Principal engineer for Intelligent Solution Group's data platform