Skip to content

SYSTEM Cited by 1 source

Slicer (Google auto-sharder)

Slicer is Google's internal auto-sharding system for datacenter applications, published at OSDI 2016 (Adya, Myers, Howell, Elson, Meek, Khemani, Fulger, Gu, Bhuvanagiri, Hunter, Peon, Kai, Shraer, Merchant, Lev-Ari). Operates on key ranges (not individual keys), uses a centralized assigner that pushes assignments to clients and servers, and historically used leases for stronger key-ownership guarantees than an eventual-consistency model would give.

Reference: https://www.usenix.org/system/files/conference/osdi16/osdi16-adya.pdf.

Slicer is the closest ancestor of systems/dicer; Databricks' open-source announcement explicitly cites it as prior art alongside systems/centrifuge and systems/shard-manager. Dicer diverges from Slicer primarily on the consistency model: Slicer used leases for strong key ownership, Dicer chose concepts/eventual-consistency of assignments, prioritizing availability and fast recovery over exclusive ownership.

Seen in

Last updated · 200 distilled / 1,178 read