Skip to content

PATTERN Cited by 1 source

Leader-driven safe-deletion watermark

Pattern

A designated leader collects per-replica monotonic progress values and computes a min-aggregate that represents the globally-safe deletion threshold. Any record below this threshold can be physically removed on any replica without risk of divergence.

Mechanism

Replica A: MCCO = 150  ──┐
Replica B: MCCO = 80   ──┼──► Leader: MTRO = min(150, 80, 120) = 80
Replica C: MCCO = 120  ──┘

All replicas may safely delete tombstones below offset 80. Replica B's slower progress constrains the group — correctness is bounded by the slowest replica.

Properties

  • Monotonicity — each replica's progress value only advances. The min-aggregate therefore also only advances (unless a new slow replica is added).
  • Stale-safe — if a replica is unreachable, its last-known value is used. Since values only increase, a stale value is always a conservative underestimate.
  • Leader-change tolerant — a new leader can resume from the last-distributed threshold without recollecting from scratch (though fresh collection is preferred for liveness).

Instances on the wiki

System Per-replica value Aggregate What it guards
Redpanda coordinated compaction MCCO / MXFO MTRO / MXRO Tombstone and transaction marker deletion
Redpanda Cloud Topics L0 GC Per-partition epoch M(p) Cluster M = min(M(p)) L0 object deletion

Relation to lazy-aggregate pattern

This is the leader-driven instantiation of patterns/lazy-aggregate-from-monotonic-local-state — the compaction protocol uses an explicit leader-push model while Cloud Topics L0 GC piggybacks on existing metadata dissemination. Both exploit monotonicity for stale-tolerance.

Seen in

Last updated · 559 distilled / 1,651 read