Skip to content

CONCEPT Cited by 2 sources

Broker-side write caching

Definition

Broker-side write caching is a broker configuration that relaxes the produce-ack latency path by acknowledging writes once the record is in memory on a quorum of brokers, rather than waiting for fsync to disk on every broker. The broker then flushes batched in-memory records to disk in larger blocks in the background, trading a well-bounded durability weakening for aggregate throughput + lower per-commit latency.

Canonicalised on the wiki from Redpanda's 2024-11-26 batch-tuning part 2 (Source: sources/2024-11-26-redpanda-batch-tuning-in-redpanda-to-optimize-performance-part-2).

Default vs. write-caching behaviour

Default (write caching off) — the Redpanda-native strong- durability path:

Client produce → leader broker → write to disk
                              → replicate to followers
                              → followers write to disk + ack
                              → leader ack to client

Write caching on — ack-on-memory with background flush:

Client produce → leader broker → write to memory
                              → replicate to followers (memory write)
                              → followers ack on in-memory-write
                              → leader ack to client
                              → (background) flush larger blocks to disk

Post verbatim:

"Write caching shifts the broker's behavior when writing out messages to disk… the brokers acknowledge as soon as the message is in memory. Once a quorum is reached, the leader acknowledges back to the client, and the brokers can flush larger blocks of messages to disk, taking better advantage of the storage."

Durability relaxation, bounded

The post states the relaxation precisely:

"The write caching mechanism is similar in function to how Kafka commits batches to in-memory buffer cache and then periodically flushes that cache out to disk. When write caching is enabled in Redpanda, the data durability guarantees are relaxed but no worse than a legacy Kafka cluster."

The equivalence is load-bearing: legacy Kafka's default commit path acks on OS-buffer-cache write (not fsync), and the filesystem flushes to disk asynchronously. Redpanda's default is stronger than legacy Kafka (explicit fsync before ack); write caching relaxes Redpanda to Kafka-legacy-equivalent, which large Kafka-native production fleets already operate on safely.

The canonical failure mode: simultaneous leader + follower-quorum memory loss before flush → committed writes are lost. This is the same failure mode classic Kafka deployments have accepted for a decade.

When to enable

Write caching is the broker-side mitigation when client-side batching tuning is unavailable — the explainer names three canonical cases:

  • "your architecture makes it hard to do client-side tuning"
  • "change the producer behavior"
  • "adjust topic and partition design"

For fleets where the producer is owned by a different team, where clients span languages / runtimes with inconsistent defaults, or where partition count is locked by a downstream ordering contract, write caching is the single-switch path to reclaim the small-batch performance lost at the producer.

Storage-controller analogy

The post draws the analogy verbatim:

"This is similar to how a storage controller might take many small writes and assemble them into a single large write to preserve storage IOPS capacity."

The mechanism is the same re-batching pattern the Linux page cache, hardware RAID controllers, and enterprise storage arrays all implement — a write-back cache that amortises the per-write fixed cost of the backing medium by coalescing in-memory.

Seen in

  • sources/2025-04-23-redpanda-need-for-speed-9-tips-to-supercharge-redpanda — canonicalises the hardware-shortfall trigger for enabling write caching: when local NVMe isn't available ("SSDs, spinning disks, SAN, remote storage — anything other than locally attached NVME"), write caching becomes the canonical mitigation. Complements Kinley 2024-11-26's organisational framing (producer team not tunable) — the same primitive solves two distinct classes of problem. Reaffirms acks=all as the mandatory companion to preserve the quorum-memory durability guarantee; suggests multi-AZ to reduce blast-radius exposure.
  • sources/2024-11-26-redpanda-batch-tuning-in-redpanda-to-optimize-performance-part-2 — canonical wiki source. Default vs. write-caching state machine; durability-equivalence-to-legacy-Kafka framing; storage-controller analogy; enumerates the three client-side-untunable cases.
Last updated · 470 distilled / 1,213 read