Skip to content

PATTERN Cited by 1 source

Cluster sharding for blast radius

Definition

Split a large streaming cluster into multiple smaller shards (independent clusters), each handling a subset of topics/tenants. During an incident, traffic from an affected shard is quickly routed to a healthy one — limiting the blast radius to a fraction of total throughput rather than the entire platform.

Trade-offs

  • Pro: Reduces blast radius; enables partial failover within minutes (~15 min at Atlassian).
  • Pro: Limits the "hot broker" problem to one shard rather than cross-contaminating all traffic.
  • Con: More clusters to manage; cross-shard coordination requires routing intelligence.
  • Con: Receiving shard must have spare capacity or accept rate-limited traffic during failover.

Seen in

Last updated · 602 distilled / 1,824 read