PATTERN Cited by 1 source

RocksDB compaction tuning for TTL

Pattern

Override RocksDB's default compaction parameters to ensure background compaction runs frequently enough (and processes enough data per run) to exercise the TTL filter and prevent unbounded state growth in streaming workloads.

Problem

Default RocksDB settings are tuned for general-purpose workloads. In streaming systems with high write rates and bounded retention (e.g., 15-minute event windows), the defaults cause: - Compaction runs too infrequently → expired keys accumulate - Each compaction processes too little data → TTL filter has limited opportunity - State grows without bound despite all entries being logically expired

Configuration recipe (from Zalando production)

# Larger, more frequent compaction runs that exercise TTL filter
state.backend.rocksdb.writebuffer.count: "16"                           # default: 2
state.backend.rocksdb.writebuffer.size: "512MB"                         # default: 64MB
state.backend.rocksdb.compaction.level.target-file-size-base: "256MB"   # default: 64MB
state.backend.rocksdb.compaction.level.max-size-level-base: "1GB"       # default: 256MB

# Post-rescale cleanup (orphaned keys from previous parallelism)
state.backend.rocksdb.incremental-restore-async-compact-after-rescale: "true"
state.backend.rocksdb.rescaling.use-delete-files-in-range: "true"

# Reduce on-disk footprint
state.backend.rocksdb.compression.per.level: "LZ4_COMPRESSION"         # default: SNAPPY
state.backend.rocksdb.predefined-options: "FLASH_SSD_OPTIMIZED"         # default: DEFAULT

Seen in

• Zalando Ad Platform (2026-07): Without tuning, state grew continuously despite 15-minute logical TTL. After applying the configuration above, state size stabilised. The post-rescale settings are critical for autoscaling: after parallelism changes, RocksDB files still contain keys from the previous partition assignment (Source: sources/2026-07-23-zalando-from-homegrown-to-flink-migrating-a-stateful-ad-event-join).

When to use

• Flink jobs using KeyedCoProcessFunction (or similar) with RocksDB TTL for state expiry
• High-throughput streaming with bounded retention windows
• Any Flink job where state size grows unexpectedly despite logical TTLs

Trade-offs

• Larger write buffers → higher memory usage per TaskManager
• More aggressive compaction → higher write amplification and CPU usage
• LZ4 over Snappy → slightly more CPU for compression, smaller on-disk size