Skip to content

CONCEPT Cited by 1 source

Wide-area consensus

Wide-area consensus is the problem of running consensus algorithms across geographically distributed data centers connected by the public Internet — a network with unpredictable, variable, and sometimes very high latencies.

Challenges

  1. Latency is proportional to distance — consensus requires communication between a majority of replicas; if replicas span continents, every write pays intercontinental round-trip cost
  2. Latency is unpredictable — the Internet has variable delays, making timeout-based failure detection unreliable (see concepts/leader-timeout-problem)
  3. Partitions are common — links and cables get cut, queues fill up, data centers go down
  4. No single "right" timeout — WAN jitter invalidates any fixed timeout configuration

Design approaches

  • Leader-based with region-affinity — place the leader near the majority of writes; accept degraded write latency from distant regions and unavailability during leader failure + election
  • Leaderless (concepts/leaderless-consensus) — any replica can drive consensus; no single point of failure; system never pauses for elections. Higher per-write cost for non-leader proposals (3+ round-trips) but consistently available

Cloudflare's choice

Cloudflare chose QuePaxa (via Meerkat) because their network spans 330+ data centers with latencies that "can and do vary wildly," making leader-based protocols unsuitable. QuePaxa was designed for adversarial asynchronous networks and maintains ~10× higher throughput than Raft under targeted attack conditions.

Fundamental limit

No protocol can avoid the latency floor: proposal decision latency is at minimum the round-trip between the proposer and the farthest replica in the required majority. If replicas are far apart, latency increases — there is no getting around that.

Seen in

Last updated · 573 distilled / 1,747 read