Apache Pinot

Apache Pinot is a distributed OLAP database built for real-time analytics where latency and concurrency actually matter. Originally developed at LinkedIn, it's now used by Stripe, Uber, Walmart, and others to serve analytical queries at hundreds of thousands of requests per second, often with P99 latencies under 100ms.

The core use case: you have streaming data and you need to query it interactively, at scale, without pre-aggregating everything in advance. Pinot handles that through columnar storage, a rich set of pluggable indexes, and a distributed architecture that scales horizontally.

Key capabilities:

• Real-time ingestion from Kafka, Kinesis, and Pulsar, plus batch sources like S3, Hadoop, and Spark. Both can feed the same table.
• Upserts built in, so repeated records resolve to their latest value at query time without extra application logic.
• High-concurrency query serving, designed for scenarios where end users or AI agents are hitting the database directly, not just internal dashboards.
• Versatile joins across petabyte datasets, not just simple lookups.
• Multitenancy with logical namespace isolation, useful for SaaS products serving per-customer analytics.
• SQL interface with a built-in query editor and REST API.

Pinot fits two broad patterns. First, user-facing analytics: embedded dashboards, customer-facing data exploration, leaderboards, and usage metrics where your product's users are running queries in real time. Second, AI agent backends: LLM-powered systems that need fresh context from streaming data rather than stale snapshots. Think fraud scoring, real-time RAG retrieval, or agentic observability over live logs.

Compared to tools like ClickHouse, Pinot's design leans heavily toward serving many concurrent external users rather than fewer internal analysts running complex ad-hoc queries. The trade-off is intentional. Pinot's indexing options (inverted, range, text, geospatial, and more) let you tune for specific query patterns without restructuring your data model.

At Stripe, it handles 200K queries per second across 3 petabytes with P99 latency of 70ms. At Uber, a single service runs 500 million Pinot queries daily. These aren't edge cases; the architecture is built around this kind of load from the start.