Skip to content

CONCEPT Cited by 1 source

delete.retention.ms

Definition

delete.retention.ms is a Kafka topic-level configuration (default: 24 hours) that controls how long a tombstone or expired transaction control batch remains in a compacted log before becoming eligible for physical deletion by the log cleaner. It serves as a time-based heuristic to give slow consumers and lagging replicas a window to observe deletion/transaction markers before they disappear.

Limitation

The parameter assumes all replicas will catch up within the configured window. This assumption is unfounded for:

  • Hardware failures requiring replacement
  • Extended maintenance windows
  • Network partitions lasting longer than the retention period

"A broker that's offline or lagging past these timers (due to a hardware failure, a long maintenance window, or a slow recovery) will miss both the marker and its empty-batch remnant, with no way to recover." (Source: sources/2026-06-25-redpanda-kafkas-log-compaction-corrupts-data)

No finite value of delete.retention.ms prevents the compaction–replication race — it only changes how long a broker must be offline before divergence occurs.

Seen in

Last updated · 559 distilled / 1,651 read