Data - The Land DevOps Forgot
You need to be signed in to add a collection
We've transformed our operational systems to microservices with stream-aligned teams that own what they build. We know that working in short cycles with fast feedback produces better value sooner, safer, and happier. But there's a dark underbelly. Those microservice databases get pulled out into analytics processing that runs in monolithic batches that run for hours on hundreds of cores, then break with an error. Feedback takes days, and the centralized data team can't keep up with the pace of the business or data growth. There's another way, gaining traction. Data mesh promises to re-decentralize data processing along with ownership. It applies product thinking to internal data products. We will talk about moving to data mesh: the promise and the pitfalls, what you can buy and what you'll need to build, and how to get internal buy-in for the transformation.
Transcript
We've transformed our operational systems to microservices with stream-aligned teams that own what they build. We know that working in short cycles with fast feedback produces better value sooner, safer, and happier. But there's a dark underbelly. Those microservice databases get pulled out into analytics processing that runs in monolithic batches that run for hours on hundreds of cores, then break with an error. Feedback takes days, and the centralized data team can't keep up with the pace of the business or data growth. There's another way, gaining traction. Data mesh promises to re-decentralize data processing along with ownership. It applies product thinking to internal data products.
We will talk about moving to data mesh: the promise and the pitfalls, what you can buy and what you'll need to build, and how to get internal buy-in for the transformation.
About the speakers
General Manager of Data at Nubank