CONCEPT Cited by 2 sources

Effective batch size

Definition

Effective batch size is the actual batch size a streaming- broker producer achieves in production, as distinct from the configured ceiling (batch.size) the operator sets. Redpanda's 2024-11-19 batch-tuning explainer frames the gap verbatim:

"Because batching is enabled by setting only a few simple parameters, it can be tempting to think they're the only factors that matter. In reality, the actual batch sizes result from a range of factors, of which the producer configuration is a small, but crucial part." (Source: sources/2024-11-19-redpanda-batch-tuning-in-redpanda-for-optimized-performance-part-1)

The seven factors

A field-manual enumeration of what actually moves effective batch size, only two of which are on the producer's config surface:

1. Message rate. A producer at 1 msg/sec cannot form a 16 KB batch regardless of configured knobs unless linger.ms is unreasonably high. Configured batch.size is a ceiling, not a guarantee.
2. batch.size. The byte ceiling. When records arrive faster than linger.ms can accumulate, batch.size is the effective trigger.
3. linger.ms. The time ceiling. When records arrive slower than batch.size can fill in time, linger.ms is the effective trigger. Raising one without the other leaves the higher knob unused.
4. Partitioning. Batches are per-partition. More partitions ⇒ fewer records per partition ⇒ smaller batches. Keyed records hash to partitions deterministically, intensifying the dilution. The sticky partitioner is the per-client mitigation (concentrate writes on one partition until batch.size rotates to the next).
5. Number of producers. Scaling an HTTP-server fleet behind a load balancer horizontally divides a fixed aggregate write rate across more producers → each producer sees a smaller local rate → smaller batches per producer. Fan-out at the producer tier deoptimises batch formation.
6. Client memory (buffer.memory). Producer-side memory budget for pending batches. If total open-batch memory exceeds buffer.memory, the client must early-dispatch an existing batch to free space — dispatching below both batch.size and linger.ms thresholds. A common cause of "my batches are tiny and I can't figure out why" — the answer is usually "buffer.memory is too small for batch.size × open_partitions."
7. Backpressure. When the broker is saturated, in-flight request slots at the producer stay occupied (responses delayed), so the producer continues queueing records into open batches past the configured batch.size ceiling. Canonicalised as concepts/producer-backpressure-batch-growth — heavy broker load inflates producer batches.

Second-order effects

The seven factors interact:

• Adding brokers to a loaded cluster can decrease batch size. "Adding additional brokers to a loaded cluster can sometimes cause batch sizes to decrease since there is less backpressure." The backpressure-inflation effect (factor 7) eases as broker capacity grows, so batches shrink back toward the configured ceiling. Counter-intuitive: more broker capacity ⇒ worse batching under constant client configuration.
• Sticky-partitioner invariance to partition count. When the sticky partitioner is active, factor 4 (partitioning) is effectively neutralised — adding partitions does not reduce batch size per client because the client writes to a single partition at a time up to batch.size.
• linger.ms under saturation has inverted latency semantics. Under normal regime, higher linger.ms = higher latency; under saturation, higher linger.ms can reduce tail latency by cutting broker request rate and shrinking the internal work-queue backlog. See concepts/batching-latency-tradeoff.

Implications for tuning

A field-manual ordering:

1. Measure effective batch size before tuning batch.size / linger.ms. If effective batch size is well below batch.size, check factors 1, 4, 5, 6 first. Tuning the two knobs doesn't help if the limit is elsewhere.
2. Partition count is a batching knob. Halving partitions (when ordering requirements allow) can double effective batch size per client.
3. Producer fan-out dilutes batches. One fat producer forms bigger batches than ten thin producers at the same aggregate rate.
4. buffer.memory should be sized for peak open-partition fan-out. Rule of thumb: buffer.memory ≥ batch.size × max_open_partitions
5. headroom.
6. If batches mysteriously grow under load, don't fight it. The backpressure-inflation behaviour is protective (the producer is already throttled by max-in-flight); let it run until cluster capacity grows.

Seen in

• sources/2024-11-19-redpanda-batch-tuning-in-redpanda-for-optimized-performance-part-1 — canonical wiki source for the seven-factor framework. Redpanda's first-principles explainer on producer-side batching, Kafka-API compatible.
• sources/2024-11-26-redpanda-batch-tuning-in-redpanda-to-optimize-performance-part-2 — operations-manual companion. Canonicalises the broker-side measurement view via Prometheus private metrics + PromQL templates, the per-topic diagnosis discipline (aggregate- cluster effective batch size hides tiny-batch offenders), the 4 KB NVMe floor as the target minimum, and a real-customer production case study where three-round linger tuning on offending topics dropped p99 128 ms → 17 ms and enabled 2-cluster → 1-cluster consolidation at ~2.2× throughput per cluster.

Related

• systems/kafka, systems/redpanda — Kafka-API producers exhibit this behaviour identically.
• patterns/batch-over-network-to-broker — the producer-side pattern that effective batch size quantifies.
• concepts/kafka-partition — the partition boundary is the batch boundary.
• concepts/sticky-partitioner — mitigates factor 4.
• concepts/producer-backpressure-batch-growth — factor 7, backpressure inflates batches past batch.size.
• concepts/batching-latency-tradeoff — linger.ms semantics under normal vs saturated regime.
• concepts/fixed-vs-variable-request-cost — why batching matters at all.