Key takeaway
- Apache Druid explained: when the real-time analytics database fits, its key features, what Druid 38 changed for ZooKeeper and Java, and first steps.
Apache Druid is a real-time analytics database designed for fast slice-and-dice analytics on large datasets. It stores data in columns, processes each query in parallel across the cluster, and suits event data with a time component where inserts are frequent and updates are rare.
This guide is for data engineers, platform architects and analytics leads deciding whether Druid fits a dashboard or telemetry workload, or planning an upgrade. It covers where Druid fits and where it does not, its key features, what Druid 38 changed, and what to do first. For the surrounding platform, see our guides to big data architecture and Apache Spark architecture.
What is Apache Druid used for?
The Druid design documentation describes Druid as a real-time analytics database for fast slice-and-dice analytics. In practice that means clickstream and product analytics, network and application telemetry, and operational dashboards where many users filter and aggregate the same large event table.
| Good fit | Poor fit |
|---|---|
| Insert rates are very high, but updates are less common | You need low-latency updates of existing records using a primary key |
| You are targeting query latencies of 100 ms to a few seconds | You are building an offline reporting system where query latency is not very important |
The documentation notes that Druid supports streaming inserts but not streaming updates. If your workload is mostly about correcting existing rows, a warehouse or a database with native updates is usually simpler.
What are the key features of Apache Druid?
| Feature | What the documentation says | Why it matters |
|---|---|---|
| Column storage | Druid uses column-oriented storage, so queries load only the columns they need | Less data read per query |
| Parallel processing | Each query can run in parallel across the entire cluster | Latency stays predictable as data grows |
| Continuous operation | Designed to run without planned downtime, including for configuration changes and software updates | Fewer maintenance windows |
| Summarisation at ingestion | Optionally summarises data at ingestion time to cut cost and speed up queries | Smaller storage footprint |
Pipelines that feed Druid are covered in our guide to data ingestion tools and architecture, and securing a Kafka source is covered in Kafka security with Kerberos on Kubernetes.
What changed in Apache Druid 38?
The Druid release notes list Apache Druid 38.0.0 as containing over 291 new features, with bug fixes, performance work and documentation. Several changes affect how you deploy and upgrade.
| Change | Detail from the release notes | What to do |
|---|---|---|
| ZooKeeper-based task runner removed | The RemoteTaskRunner (druid.indexer.runner.type=remote) is gone; httpRemote is the only supported distributed task runner | Remove the property or set it to httpRemote, because startup fails otherwise |
| ZooKeeper-based segment discovery removed | HTTP is the only supported option for druid.serverview.type | Remove the property or set it to http |
| Java support | Java 25 is supported, Java 21 remains supported and Java 17 support was dropped | Check runtime images and JAVA_HOME before upgrading |
| Query context | realtimeSegmentsOnly is deprecated in favour of realtimeSegmentsMode | Update applications that set it |
| Legacy ZSTD frames | Druid 38 is the last major release that supports reading legacy ZSTD frames (versions 0.4 to 0.7) | Re-encode affected input data before a later upgrade |
The notes add that ZooKeeper is still used for Coordinator and Overlord leader election and for service announcement and discovery, so Druid 38 does not remove ZooKeeper entirely. They also list several ZooKeeper-related properties that are no longer recognised and should be deleted from common.runtime.properties. If you run ZooKeeper for Druid, our guide to Apache ZooKeeper security and architecture covers hardening.
How should you run and govern Druid?
- Treat upgrades as configuration work: audit runtime properties before moving to Druid 38 so removed options do not stop services from starting.
- Upgrade services together: the release notes warn that a new shard spec type has no fallback, so upgrade all services before enabling the feature that uses it.
- Control cost: summarising data at ingestion is the documented lever for smaller storage and faster queries.
- Plan the surrounding stack: deep storage, a metadata database and monitoring all need owners, as with any data platform. Our Apache Druid managed services cover deployment, upgrades and operations.
How do AI agents change real-time analytics?
When an agent reads a live metric to decide what to do next, query latency and data freshness become part of the decision, not only of a dashboard. That is where a fast analytics store helps, and where agents need clear limits on which metrics they may act on and a record of why. ElixirData, XenonStack's Context OS, is aimed at governed data and decision context for agents that act on analytics.
What should you do first?
- Test your workload against the good-fit and poor-fit lists above. If primary-key updates dominate, stop here.
- Run a proof of concept with your real queries and concurrency, ideally beside one alternative store.
- If you already run Druid, audit your properties against the Druid 38 changes before upgrading.
- Name owners for ingestion, storage, metadata and monitoring.
- Decide which metrics agents may act on automatically and which need a person.
To map one real-time analytics decision your agents should make, and the data and policy it depends on, book a Context Workshop. See what happens in the workshop.
Planning the warehouse layer around Druid? Download our data warehouse guide.
Frequently asked questions
Is Apache Druid a database?
Yes. The Druid documentation describes it as a real-time analytics database designed for fast slice-and-dice analytics on large datasets.
Does Apache Druid still need ZooKeeper?
Yes, for some jobs. The Druid 38 release notes say ZooKeeper is still used for Coordinator and Overlord leader election and for service announcement and discovery, while ZooKeeper-based segment discovery and the ZooKeeper-based task runner were removed.
Which Java versions does Druid 38 support?
Druid 38 supports Java 25 and Java 21, and support for Java 17 has been dropped.
Can Druid update existing records?
Not well. The documentation lists low-latency updates of existing records using a primary key as a poor fit, and says Druid supports streaming inserts but not streaming updates.
When is Druid a good fit?
It fits when insert rates are very high, updates are less common and you are targeting query latencies of 100 ms to a few seconds.
Follow XenonStack on LinkedIn: XenonStack on LinkedIn for new guides on agentic AI, data and platform engineering.
Ready to put AI agents to work? Explore ElixirClaw, the agentic OS platform to build, orchestrate and govern AI agents.
Explore the XenonStack platform family: ElixirData · ElixirClaw · ElixirHub · NexaStack · MetaSecure · Neural AI · AgentGRC · AgentSRE · AgentAnalyst · AgentInstruct · AgentRAI · AgentScrum · AgentSearch · AgentForce · AI Aviator · SAIF Aviator · CohortIQ · XenonStack AI · XenonStack UAE
