Skip to content

SYSTEM Cited by 1 source

Dicer (Databricks auto-sharder)

Dicer is Databricks' open-sourced auto-sharder: a control plane that continuously and asynchronously updates a service's shard assignment in response to pod health, load, termination notices, and other environmental signals. It sits behind "every major Databricks product" and is explicitly positioned in the lineage of Google's systems/slicer, Microsoft's systems/centrifuge, and Meta's systems/shard-manager. Open-sourced January 2026 at https://github.com/databricks/dicer.

Model

  • Key — application-level id (user id, tenant id, chat-room id, …).
  • SliceKey — hash of the Key; the uniform space Dicer operates on.
  • Slice — contiguous range of SliceKeys.
  • Resource — a pod (or equivalent compute unit).
  • Assignment — partition of the full SliceKey space into Slices, each mapped to one or more Resources. The Assigner mutates Assignments via splits, merges, replications, dereplications, and moves — always minimal adjustments, not full reshuffles.
  • Target — a sharded application served by Dicer. The Assigner is multi-tenant and region-scoped; one Assigner serves many Targets within a region.

Components

  • Assigner — the controller service. Runs the sharding algorithm; consumes health + per-key-load signals; publishes Assignments.
  • Slicelet (server-side library) — embedded in application pods. Caches the current Assignment, watches the Assigner for updates, notifies the app via listener API. Also records per-key load locally and reports summaries asynchronously. Both paths are off the request critical path.
  • Clerk (client-side library) — embedded in clients. Caches the Assignment so Clerk.lookup(key) → pod is a local call with no RPC.

Consistency model

Assignments observed by Slicelets and Clerks are eventually consistent. The team frames this as a deliberate availability-and-recovery-speed choice over strong key-ownership — see concepts/eventual-consistency. systems/slicer and systems/centrifuge chose stronger lease-based ownership; Dicer's paper explicitly says "we do plan to support stronger guarantees in the future."

Primitive capabilities

  • Dynamic rebalancing — slices moved off overloaded or draining pods before failure.
  • Hot-key isolation + replication — a single hot key gets its own one-key slice, and that slice is assigned to multiple pods to split load. See patterns/shard-replication-for-hot-keys and concepts/hot-key.
  • State transfer — migrate per-slice application state between pods during resharding (used by Softstore to preserve ~85% cache hit rate across rolling restarts, vs. ~30% drop without).
  • Graceful restart / autoscale — assignment reacts to termination notices before the pod is gone, avoiding split-brain and traffic loss.

Canonical use cases

(From the open-source announcement.)

  • In-memory and GPU serving — KV stores with sub-ms local reads; LLM per-session KV cache affinity; LoRA-adapter placement on constrained GPUs.
  • Control and scheduling systems — cluster managers, query orchestration engines. Local state + multi-tenant routing.
  • Remote caches — systems/softstore.
  • Work partitioning / background work — non-overlapping keyspace ownership for GC / cleanup.
  • Batching & aggregation on write paths — route related records to the same pod so the pod can batch in memory.
  • Soft leader selection — see concepts/soft-leader-election.
  • Rendezvous / coordination — chat rooms, multi-client session coordination.

Production case studies

  • systems/unity-catalog — previously stateless, hit DB-read wall. Remote cache rejected (needed incremental, snapshot-consistent updates against gigabyte-scale customer catalogs). With Dicer: sharded in-memory cache, 90–95% hit rate, drastic DB-call reduction.
  • SQL Query Orchestration Engine — previously in-memory + static sharding. Manual resharding was toilsome and rolling restarts caused availability dips. Post-Dicer: zero-downtime restarts/scale, chronic CPU throttling resolved by dynamic load balancing.
  • systems/softstore — distributed KV cache built on Dicer, uses state transfer to ride rolling restarts (99.9% of planned restarts) with ~85% hit rate vs ~30% drop without.

Why it exists (motivating context)

Two dominant architectures at Databricks both had failure modes (Source: sources/2026-01-13-databricks-open-sourcing-dicer-auto-sharder):

Dicer's claim is that concepts/dynamic-sharding as a primitive gets the memory-locality benefit of (2) while fixing its three structural flaws.

Seen in

Last updated · 200 distilled / 1,178 read