Skip to content

Redpanda — Batch tuning in Redpanda for optimized performance (part 1)

Summary

Part 1 of a two-part first-principles explainer on producer-side batching for Redpanda (and, by Kafka-API compatibility, Apache Kafka) streaming brokers. Frames batching from the economics of request servicing — every request has a fixed cost (work done regardless of size) plus a variable cost (work proportional to payload) — so amortising the fixed cost across many records is the only way small requests scale. Walks the three Kafka-client knobs that compose producer batching (linger.ms, batch.size, buffer.memory) and canonicalises the pseudo-code trigger logic: whichever of the two thresholds (linger-time elapsed OR next record would exceed batch.size) fires first closes and sends the batch. The load- bearing contribution is the seven-factor framework for effective batch size — the observation that operator-set batch.size and linger.ms are only two of at least seven inputs that determine actual in-production batch sizes; the others (message rate, partitioning strategy, producer fan-out, client buffer memory, broker backpressure) are outside the producer configuration surface but dominate in real workloads. Closes with the counterintuitive latency finding: increasing linger.ms under a CPU-saturated broker can reduce end-to-end latency by shrinking the broker's internal work-queue backlog, even though the naive read of linger.ms is "add up-to-this-much wait-time to each produce." Tier-3 substrate- qualifying: broker internals, CPU-saturation / queueing dynamics, Kafka-API semantics, distributed-systems trade-offs — the "counter intuitive latency" framing is the canonical wiki anchor for reducing request rate reduces tail latency under saturation, a substrate distinct from (though composing with) classic tail-latency-at-scale patterns.

Key takeaways

  1. Every request has a fixed and a variable cost. "Each request incurs a fixed cost (for work that happens regardless of the request size) and a variable cost (for work determined by what was requested)." Small requests cannot scale to zero because the fixed cost dominates. Batching is the primitive that amortises the fixed cost across many records. (Source: sources/2024-11-19-redpanda-batch-tuning-in-redpanda-for-optimized-performance-part-1)
  2. A Kafka/Redpanda batch = one request to one partition. "A batch in Redpanda is a group of one or more messages written to the same partition, which are bundled together and sent to a broker in a single request. Rather than each message being sent and acknowledged separately, requiring multiple calls to Redpanda, the client buffers messages for a short time, optionally compresses the whole batch, and then sends them later as a single request." The batch boundary is the partition boundary — which is why partition count directly affects achievable batch size.
  3. Batching compounds with compression. "The compression ratio improves as you compress more messages at once since it can take advantage of the similarities between messages." Compression is not independent of batching — the effective CPU-savings-per-byte from compression is a function of batch size.
  4. Three Kafka producer knobs compose batching behaviour. Redpanda's explainer names the canonical trio verbatim:
    • linger.ms — max wait before dispatch. Java-client default 0 ms (no batching). "It's worth double-checking the defaults for your preferred library."
    • batch.size — max bytes per batch. Default 16 KB.
    • buffer.memory — total producer memory for pending batches. Default 32 MB. Composing trigger logic verbatim as pseudo-code: 
      if (client not at max-in-flight cap):
    if (current linger > linger.ms || next message would exceed batch.size):
        close_and_send_current_batch()
else:
    if next message would exceed batch.size:
        close_and_enqueue_current_batch()
      This is canonicalised on the wiki as concepts/batching-latency-tradeoff.
  5. Seven factors determine effective batch size — the producer config is only two of them. "In reality, the actual batch sizes result from a range of factors, of which the producer configuration is a small, but crucial part." The full list: (1) message rate, (2) batch.size, (3) linger.ms, (4) partitioning (more partitions → fewer messages per partition → smaller batches; keyed partitioning intensifies this unless the sticky partitioner is active), (5) number of producers (a load-balanced HTTP-server fleet produces smaller batches per producer than a single writer at the same aggregate rate), (6) client buffer memory (if buffer.memory is insufficient for all open batches, the client must early-dispatch an existing batch to make room), (7) backpressure ("if Redpanda is heavily loaded, a client with all of the max-in-flight slots in use will experience a form of backpressure, such that the client will continue to add records into a queued batch even beyond the maximum batch size" — heavy broker load inflates batches). Canonicalised as concepts/effective-batch-size.
  6. The sticky partitioner makes partition count invariant to batch size. "Increased partition count doesn't affect batch size when the sticky partitioner is in use since that partitioner writes all messages to a single partition (read: batch) until a size threshold is reached based on batch.size." This is a substantive design choice: without sticky, 10× partition count ⇒ ~1/10× batch size; with sticky, records concentrate on one partition per client for a full batch.size before rotating. Canonicalised as concepts/sticky-partitioner.
  7. Adding brokers to a loaded cluster can decrease batch size. "Adding additional brokers to a loaded cluster can sometimes cause batch sizes to decrease since there is less backpressure." Counter- intuitive second-order effect — the backpressure that was inflating batches beyond batch.size eases, so batches shrink back toward the configured max. Canonicalised as concepts/producer-backpressure-batch-growth.
  8. Counterintuitive latency: increasing linger.ms under saturation can reduce tail latency. The mechanism: under CPU saturation, the broker's internal work queue backlog grows, and produce requests wait their turn in the queue — the tail of produce latency is dominated by queueing, not by the produce itself. Reducing request rate (by batching harder at the producer) reduces the backlog and the queueing tax. "Modest adjustments of the producer linger time can ease the saturation enough to allow tail latencies to drop significantly. From the client's perspective, it looks like you're inducing higher potential end-to-end latency by increasing linger, but you're actually reducing it because the scheduling backpressure is largely getting reduced, if not removed." — a first-principles canonical wiki statement of reducing request rate under saturation reduces latency.

Architectural framing

Batching as fixed-cost amortisation

The first-principles framing — request cost = fixed + variable, small-requests-cannot-scale-to-zero, batching-amortises-fixed — is canonicalised on the wiki as concepts/fixed-vs-variable-request-cost. The same framing underpins the Kafka pattern already on the wiki: patterns/batch-over-network-to-broker (producer-side transport economics) and concepts/pagecache-for-messaging (broker-side sequential I/O economics). The Redpanda post adds the explicit fixed/variable decomposition as the substrate argument, whereas Kozlovski's 2024-05-09 Kafka 101 framed the same mechanism at one level up ("groups messages together, reduces network overhead, single linear HDD write").

Trigger logic (pseudo-code)

Verbatim from the post, framed as two cases separated by the producer's max-in-flight state:

  • Not at max-in-flight cap: the batch dispatches as soon as either linger.ms elapses or the next message would exceed batch.size. Whichever fires first.
  • At max-in-flight cap: the producer cannot dispatch (no available request slot), so it keeps enqueueing into the current batch until the next record would overflow — at which point it closes the batch and queues it up to be sent when a slot frees. This is the mechanism by which broker backpressure inflates batch sizes: a saturated broker keeps the producer's in-flight slots full, which keeps the producer queueing into open batches past the configured batch.size threshold.

The seven factors

An expanded enumeration beyond the summary bullet #5:

  1. Message rate. Low-rate workloads (kilobytes per minute) cannot fill a batch unless linger.ms is very high. There is no amount of configuration that lets a 1 msg/sec producer form 16 KB batches with a 5 ms linger.
  2. batch.size. Upper limit on batch bytes per partition. Caps the fixed-cost amortisation.
  3. linger.ms. Wait-time trigger. Counter-balances batch.size — raising batch size without raising linger leaves bigger batches underfilled.
  4. Partitioning. "A batch contains messages for a single partition, so if we increase the number of partitions in a topic, we'd see fewer messages per partition. If messages have keys, then the partition for each message is set by the key and this then translates into a slower batch fill rate, which may then lead to smaller batches. It also increases the amount of client memory required, since there would be more batches held open." The sticky partitioner is the mitigation.
  5. Number of producers. Fan-out dilutes batch sizes: "HTTP servers behind a load balancer producing to a topic. Increasing the number of HTTP servers (producers) will result in fewer messages per producer, resulting in smaller batches."
  6. Client memory (buffer.memory). "If this is insufficient for all of the open batches (as could happen with a large maximum batch size), then if a new batch is required, one of the existing batches is sent to make space — likely before it hits either the maximum batch size or the maximum linger time." A second early-dispatch mechanism, orthogonal to the linger/batch-size triggers.
  7. Backpressure. The max-in-flight cap at the client, not the server — but the cap is set by server responsiveness. When the broker is saturated, produce responses lag, in-flight slots stay occupied, and the producer keeps enqueueing records past the configured batch ceiling.

CPU-saturation-driven latency inversion

The load-bearing production-ops insight is that producer-side batching economics change sign at broker CPU saturation:

  • Normal regime (broker CPU not saturated): more batching = more throughput + more per-record latency. Classic throughput-latency trade-off. linger.ms=0 minimises latency; linger.ms=5ms trades ~2.5 ms average producer latency for throughput.
  • Saturated regime (broker CPU near ceiling, internal work queue backlog): more batching = less tail latency, because the broker's queueing tax (time spent waiting for CPU) dominates per-request processing. "This may seem counterintuitive. Modest adjustments of the producer linger time can ease the saturation enough to allow tail latencies to drop significantly."

This is a rare first-principles wiki statement of request-rate reduction → latency reduction under saturation. Distinct from (though composing with) classic tail-latency-at-scale framing (which focuses on fan-out amplification and hedged-request mitigation).

Systems touched

  • systems/redpanda — subject of the explainer. First-principles statement of why Redpanda's Kafka-API-compatible broker benefits from the same batching economics as Kafka.
  • systems/kafka — Kafka-API semantics apply identically. The linger.ms / batch.size / buffer.memory / sticky-partitioner behaviour described is native to Kafka clients (KafkaProducer Java client, librdkafka, etc.) that Redpanda implements.

Concepts touched

Patterns touched

Operational numbers

Parameter Java-client default Redpanda guidance
linger.ms 0 ms (no batching) Raise with batch.size; check library defaults
batch.size 16 KB Raise with linger.ms; check library defaults
buffer.memory 32 MB Must accommodate batch.size × open_partitions

No specific target throughput / latency numbers disclosed in part 1 (those are promised for part 2: "In part two, we'll cover observability and tuning, optimal batch sizing, and the impact of write caching").

Caveats

  • Pedagogy voice, vendor blog. Redpanda Inc. markets a Kafka-API- compatible broker; the explainer's implicit comparison point is Kafka. The post is not a benchmark or incident retrospective; no production numbers beyond client defaults.
  • "Increase batch size + increase linger" is not a blanket prescription. The seven-factor framework implies that tuning batch.size in isolation is mostly futile — the bottleneck is often message rate, partition count, or producer fan-out, not the producer's configured ceiling.
  • Saturation-regime latency-inversion finding is qualitative. No specific p99 / p99.9 numbers, no broker CPU-utilisation curves — part 2 is promised to cover observability and tuning.
  • Compression + batching compounding claim is unquantified. "Compression ratio improves as you compress more messages at once" is uncontroversial but no specific bytes-saved-per-batch- size curve.
  • Sticky-partitioner mechanism is stated at effect-level, not mechanism-level. The post says the sticky partitioner concentrates writes on one partition until the batch.size threshold triggers rotation, but does not walk the client-side state machine that implements this.
  • buffer.memory pressure is one sentence. The case where insufficient buffer memory forces early dispatch is named but not walked — in practice this is a common production issue ("my batches are tiny and I can't figure out why" → buffer.memory is the answer).
  • The acks durability knob is absent. Part 1 is focused on throughput/latency batch tuning, not durability — the acks + min-insync-replicas framing is not discussed. This is a scope choice, not an omission flaw; durability and batching are orthogonal.
  • Redpanda-specific internals are not disclosed. The post is about Kafka-API client-side tuning, which applies identically to Kafka and Redpanda. Redpanda's broker-side internals (thread-per-core, Seastar, Raft) are not covered — those are separate substrate disclosures for future Redpanda ingests.

Cross-source continuity

  • Pairs with Kozlovski's Kafka 101 (pedagogy-altitude Kafka substrate). Kozlovski canonicalised "batching reduces network overhead, the broker writes a single linear HDD write"; Redpanda's post reconstructs the framing from first-principles request economics (fixed + variable) and the seven-factor effective-batch-size framework.
  • Composes with Voyage AI's token-count-based batching disclosure: that post names Kafka's byte/count-based batching as insufficient for GPU inference workloads where payload semantics matter. The Redpanda post is the inverse perspective — the seven-factor framework is the canonical answer to "why bytes/count batching is the right substrate for transport economics" (and therefore implicitly justifies layering a token-count aggregator in front of Kafka rather than replacing Kafka's batching logic).
  • Companions the generic-backpressure framing in concepts/backpressure at the broker-queue level: this post is one of the few wiki sources where backpressure is a source of bigger batches rather than a mechanism for slowing producers.

Source

Last updated · 470 distilled / 1,213 read