Showing 4 out of 4 results
Expert Talk: Unlocking the Power of Real-Time Analytics
Adi Polak and Tim Berglund explore the concept of analytics and what it truly means in the software development world. They delve into the benefits of real-time analytics for product development, highlighting the fine line between compute and storage and the technical requirements for achieving effective real-time analytics. They also discuss the applications of real-time analytics through the lens of Apache Pinot and StarTree Cloud, exploring use cases such as the popular "Who's Watched My Profile on LinkedIn" feature powered by Apache Pinot.
Event Streaming in The Cloud Native World with Apache Pulsar
The world is moving at an unprecedented pace and much of it has been powered by the innovations in software and systems. While event handling, messaging, and processing are not necessarily brand new concepts, the recent emergence in hardware such as virtualizations, multi-core processors, and so on, are in fact pushing the envelope in software design and development, elevating it to higher levels of capabilities never seen before. In the case of streaming which very often leverages on the underlying messaging mechanism(s) to bring distributed messaging to higher forms of purposes, such as IoT/IIoT applications, AI/ML data pipelines, or even eCommerce recommendations, event streaming platform has indeed become the “glue” in enabling data to flow through disparate systems in the pipeline and in a very dynamic fashion. This talk on event streaming is meant for anyone interested in learning about it, and understanding how it fits into the modern software development design and architecture, as well as seeing some of the challenges it faces especially in the Cloud Native environment. We’ll then take a look at an open source platform - Apache Pulsar, which is poised to become the de facto new generation of distributed messaging and streaming platform that will bring joy to developers, and enable systems and applications to be highly responsive with its true real-time capabilities.
Building a Real-Time Analytics Database: A 'Choose Your Own Adventure' Journey
Have you ever stopped to think about how to build a database? The thing is, there isn't just one way, as we can see by the massive number of data infrastructure options we have to choose from. It's a nonstop series of tradeoffs, each motivated by the constraints the database wants to satisfy. An in-memory transactional database would be one thing. A general-purpose, single-server relational database would be another. A low-latency, horizontally scalable analytics database would be...the journey we're going to take. In this talk, we'll start by picking a data model, make decisions about serialization and storage, choose indexing strategies, pick a query language, and figure out how to scale, eventually ending up with something that looks remarkably like Apache Pinot, a real-time analytics database. Pinot was built on a journey like this, always optimized for ultra low-latency, user-facing analytics at scale. In the real world, Pinot is used by applications like LinkedIn and UberEats to expose the state of the system not just to internal decision-makers, but to the users of the system itself, including all of us people who consumers of analytical queries. By focusing on the internals of Pinot and the tradeoffs made along the way to build a database of its kind, we'll see how it enables a new class of applications that every user of a system into a decision maker.
Introduction to Real-Time Analytics with Apache Pinot
When things get a little bit cheaper, we buy a little bit more of them. When things get cheaper by several orders of magnitude, you don't just see changes in the margins, but fundamental transformations in entire ecosystems. Apache Pinot is a driver of this kind of transformation in the world of real-time analytics. Pinot is a real-time, distributed, user-facing analytics database. The rich set of indexing strategies makes it a perfect fit for running highly concurrent queries on multi-dimensional data, often with millisecond latency. It has out-of-the box integration with Apache Kafka, S3, Presto, HDFS, and more. And it's so much faster on typical analytics workloads that it is not just a marginally better data warehouse, but the cornerstone of the next revolution in analytics: systems that expose data not just to internal decision makers, but to customers using the system itself. Pinot helps expand the definition of a "decision-maker" not just down the org chart, but out of the organization to everyone who uses the system. In this talk, you'll learn how Pinot is put together and why it performs the way it does. You'll leave knowing its architecture, how to query it, and why it's a critical infrastructure component in the modern data stack. This is a technology you're likely to need soon, so come to this talk for a jumpstart.