Skip to content

SYSTEM Cited by 1 source

Netflix Druid Interval-Aware Cache

Netflix's Druid interval-aware cache is an experimental external caching service that sits between dashboards and Apache Druid to handle the rolling-window dashboard query pattern. It decomposes time-series queries into granularity-aligned time buckets, caches each bucket with an age-based TTL, and assembles responses from a cached prefix plus a single narrowed Druid fetch for the missing tail (Source: sources/2026-04-06-netflix-stop-answering-the-same-question-twice-interval-aware-caching-for-druid).

Why it exists

Druid's own full-result cache misses every time a rolling-window query's right boundary shifts, and it refuses to cache responses involving realtime segments. Druid's per-segment cache avoids redundant historical scans but the Broker still merges per-segment results with realtime data on every query. At Netflix's scale — one popular dashboard generates ~192 queries/sec (64 queries per load × 30 viewers / 10-second refresh) mostly for near-identical data — the near-duplicate flood became a dominant cost. Adding more Druid hardware is prohibitively expensive (Source: sources/2026-04-06-netflix-stop-answering-the-same-question-twice-interval-aware-caching-for-druid).

How it works

1. Request interception at the Druid Router

The cache runs as an external service. Requests are intercepted at the Druid Router and redirected to the cache — this is an instance of patterns/intercepting-proxy-for-transparent-cache. Dashboards need no code changes. If the cache fully satisfies a query, it returns the result. Otherwise it shrinks the interval to the uncached portion and calls back into the Router, bypassing the redirect, to query Druid directly. Non-cacheable queries (metadata, queries without time group-bys) pass straight through untouched (Source: sources/2026-04-06-netflix-stop-answering-the-same-question-twice-interval-aware-caching-for-druid).

2. Parsing + hashing

For cacheable timeseries / groupBy queries, the cache:

  • Parses the query to extract time interval, granularity, and structure
  • Computes SHA-256 over the query with the time interval and volatile context properties removed
  • Uses that hash as the outer key — it encodes what is being asked (datasource, filters, aggregations, granularity) but not when

Druid context properties that alter response structure / contents are included in the key so they don't collide across different shapes (Source: sources/2026-04-06-netflix-stop-answering-the-same-question-twice-interval-aware-caching-for-druid).

3. Map-of-maps cache structure

Instead of a flat (query, interval) → result cache (Druid's full-result shape, which misses on every shift), Netflix uses a two-level map:

  • Outer key: query hash (as above)
  • Inner keys: timestamps bucketed to the query granularity (or 1 minute, whichever is larger), encoded as big-endian bytes so lexicographic order = chronological order, enabling efficient range scans

A 3-hour query at 1-minute granularity becomes 180 independent cached buckets, each with its own TTL. When the window shifts 30 seconds later, most buckets are reused; only the new tail triggers a Druid fetch. This is the defining property that makes the cache useful for rolling-window dashboards — the bucket granularity is what decouples shifted windows from cache-key changes (Source: sources/2026-04-06-netflix-stop-answering-the-same-question-twice-interval-aware-caching-for-druid).

4. Exponential TTL ladder (age-based)

Not all data points are equally trustworthy because of late-arriving events. Netflix sets TTLs that grow with data age:

Bucket age TTL
< 2 min 5 s (min)
2 min 10 s
3 min 20 s
4 min 40 s
5 min 80 s
... doubles per additional minute of age
(cap) 1 hour (max)

Fresh buckets cycle rapidly (late-arriving corrections picked up quickly); old buckets linger (confidence grows with time). For a 3-hour rolling window, the ladder ensures the vast majority of the query is served from cache, leaving Druid to scan only the most recent unsettled data. This is the canonical patterns/age-based-exponential-ttl pattern (Source: sources/2026-04-06-netflix-stop-answering-the-same-question-twice-interval-aware-caching-for-druid).

5. Contiguous-prefix lookup + single-tail fetch

On a cache lookup, the cache fetches cached buckets in the requested range only if they're contiguous from the start. Because bucket TTLs expire unevenly, gaps can appear; on the first gap, the cache stops and fetches all newer data from Druid in one narrowed query. This guarantees a complete, unbroken result set with at most one Druid query per request — rather than filling gaps with multiple small fragmented queries that would increase Druid load. This is the patterns/partial-cache-hit-with-tail-fetch pattern (Source: sources/2026-04-06-netflix-stop-answering-the-same-question-twice-interval-aware-caching-for-druid).

6. Negative caching (with trailing-bucket exception)

For naturally sparse metrics, time buckets may legitimately have no data. Without special handling, the gap detector would re-query Druid for these every time. Netflix caches empty sentinel values for no-data buckets and the gap detector treats them as valid cached data — this is concepts/negative-caching.

Crucial exception: trailing empty buckets (after the last real data point) are not negatively cached. An empty tail might be data that hasn't arrived yet; caching "no data" would make late-arrival chart delays worse (Source: sources/2026-04-06-netflix-stop-answering-the-same-question-twice-interval-aware-caching-for-druid).

7. Combining + asynchronous write-back

The cached prefix and the freshly-fetched Druid tail are concatenated, sorted by timestamp, and returned to the client. From the client's perspective, the JSON shape is byte-identical to what a direct Druid query would have returned — full transparency.

The fresh data from Druid is parsed into per-granularity buckets and written back to the cache asynchronously, so the cache write doesn't add latency to the response path (Source: sources/2026-04-06-netflix-stop-answering-the-same-question-twice-interval-aware-caching-for-druid).

Storage layer — KVDAL on Cassandra

Backing store is Netflix's KVDAL (backed by Cassandra). KVDAL's two-level-map abstraction is a direct fit:

  • Outer key = query hash
  • Inner keys = bucketed timestamps

Two KVDAL features are load-bearing for this use case:

  1. Independent TTLs on each inner key-value pair — exactly what the age-based TTL ladder needs, with zero manual eviction logic.
  2. Efficient range queries over inner keys — "give me all cached buckets between A and B for query hash X" — directly maps onto the contiguous-prefix lookup.

This is the first wiki-documented KVDAL consumer use case beyond the 2024-09-19 KVDAL launch post itself, and it exercises KVDAL features that the launch post described only in the abstract (Source: sources/2026-04-06-netflix-stop-answering-the-same-question-twice-interval-aware-caching-for-druid).

The deliberate staleness trade-off

Caching introduces up to 5 seconds of additional staleness on the newest data. This is acceptable at Netflix because:

  • Operational dashboards refresh every 10–30 seconds
  • Many queries already set end = now-1m or now-5s to avoid the "flappy tail" of currently-arriving data
  • End-to-end pipeline latency is <5 s at P90 — the 5 s cache TTL adds negligibly on top of staleness that's already there

Netflix declares this a deliberate staleness-vs-load trade-off (Source: sources/2026-04-06-netflix-stop-answering-the-same-question-twice-interval-aware-caching-for-druid).

Production results

Typical day

  • 82% of real user queries get at least a partial cache hit
  • 84% of result data served from cache
  • Cache P90 response time ~5.5 ms
  • Druid scans much narrower time ranges → fewer segments touched → less data processed
  • Scaling bottleneck moves from Druid query capacity (expensive) to cache capacity (much cheaper)

A/B experiment

  • ~33% drop in queries to Druid
  • ~66% improvement in overall P90 query times
  • Up to 14× reduction in result bytes on some shapes

Caveat from the post: gains depend heavily on how similar + repetitive the query workload is — the rolling-window-dashboard pattern is the sweet spot, not ad-hoc analytics.

Hyperscale event handling

During high-volume events (e.g. live shows), many users view the same dashboards → the cache serves most identical queries as full hits → the query rate reaching Druid is "essentially the same with 1 viewer or 100" (Source: sources/2026-04-06-netflix-stop-answering-the-same-question-twice-interval-aware-caching-for-druid).

Current status + forward direction

  • Experimental — the cache is in production use but flagged as experimental
  • Partial SQL-templating support has been added so dashboard tools can benefit without writing native Druid queries
  • Long-term direction: upstream into Druid natively — Netflix explicitly frames the external-proxy shape as a workaround. Target: opt-in, configurable, result-level cache in Druid Brokers, with metrics to tune TTLs and measure effectiveness — eliminating the infra overhead + network hops + SQL-templating workarounds + benefiting the broader Druid community (Source: sources/2026-04-06-netflix-stop-answering-the-same-question-twice-interval-aware-caching-for-druid)

Caveats

  • Experimental status; production but not GA-claimed.
  • Gains workload-dependent — rolling-window dashboards are the sweet spot.
  • No discussion of cache-stampede or request coalescing on hot bucket expiries.
  • No discussion of Cassandra / KVDAL capacity planning for this workload.
  • No discussion of what happens when Druid schemas change mid-TTL.
  • Non-time-group-by queries (metadata, general select) pass through unchanged — limits hit-rate ceiling.
  • SQL-templating support described as "partial."

Seen in

Last updated · 319 distilled / 1,201 read