Skip to content

PATTERN Cited by 1 source

Graduated redistribution for skewed data

Graduated redistribution is the pattern of spreading heavily-skewed data across a multi-stage pipeline by re-hashing and redistributing at each stage boundary, so that even extreme power-law concentrations never overwhelm a single node.

Pattern shape

Stage 1: aggregate locally → hash on key A → distribute
Stage 2: process → hash on key B → distribute
Stage 3: final aggregate → persist

Each redistribution step uses a different hash key or different partitioning, ensuring that data concentrated at one stage spreads across multiple instances at the next.

Why single-stage fails with skewed data

Consistent hashing distributes load evenly only when keys have uniform traffic. With power-law distributions (some keys are 100× hotter than others), a single hash-and-route step concentrates all that heat on one instance — the "owner" for the hot key.

Multi-stage redistribution breaks this: even if Stage 2 has a hot key, the output from that hot key gets redistributed across multiple Stage 3 instances via a different hash. The hot key's processing load spreads across the pipeline rather than accumulating at one point.

Netflix's application (2026-07-13)

Netflix initially used two stages for Service Topology's flow-log pipeline. Stage 2 combined intermediary resolution (CPU-heavy, hot-key-prone) with enrichment (I/O-heavy). Hot intermediaries overwhelmed their owner instances.

The fix: split into three stages. Stage 2 focuses on resolution and redistributes to Stage 3 for enrichment. Each flow is "distributed, resolved, distributed again, and then persisted" — spreading work across multiple instances and isolating compute from I/O. Even with 100× traffic skew, no single instance becomes a bottleneck.

(Source: sources/2026-07-13-netflix-building-service-topology-at-scale-architecture-challenges)

Key insight

Architectural decisions driven by one requirement (intermediary resolution needed a dedicated stage) often solve other problems (load distribution) as beneficial side effects.

Seen in

Last updated · 585 distilled / 1,765 read