CONCEPT Cited by 1 source
Flink memory tuning¶
Definition¶
The practice of configuring Flink's multi-layered JVM memory model to prevent Linux kernel OOM kills while maximising available state storage. Flink's memory model divides pod memory into: framework heap/off-heap, task heap/off-heap, managed memory (RocksDB cache/sort buffers), JVM metaspace, and JVM overhead (native allocations including RocksDB's own memory).
Why kernel OOM kills happen¶
Flink pods run as a single JVM process. When RocksDB (C++ via JNI) allocates native memory outside the JVM's tracked regions, the process can exceed the pod's memory limit without triggering a Java OutOfMemoryError. The Linux kernel terminates the process via OOM-killer — no graceful handling, entire pod lost.
Key parameters¶
| Parameter | Typical default | Production tuning |
|---|---|---|
taskmanager.memory.jvm-overhead.fraction |
0.1 | 0.2 |
taskmanager.memory.jvm-overhead.max |
1 GB | 4 GB |
taskmanager.memory.managed.fraction |
0.4 | 0.6 (reduce if RocksDB native memory is high) |
taskmanager.memory.jvm-metaspace.size |
256 MB | 256 MB |
Seen in¶
- Zalando Ad Platform (2026-07): Pod memory increased from 16 GB to 20 GB (4 GB buffer above JVM process size). Flink's auto-tuning recommendations from logs were incorrect — the overhead max and managed memory fraction both needed manual adjustment. Memory ratio empirically determined at 5 GB per CPU core (Source: sources/2026-07-23-zalando-from-homegrown-to-flink-migrating-a-stateful-ad-event-join).
Lessons¶
- Flink's auto-tuning log recommendations are a starting point, not final values
- RocksDB native memory is the unpredictable component — always leave a buffer
- CPU-to-memory ratio must be determined experimentally for each workload type (checkpointing workloads use more CPU than pure processing)