Skip to content

CONCEPT Cited by 1 source

Preallocated memory budget

A performance optimization where a consumer module allocates a fixed-size memory region at startup sized for some upper bound of the data it will process, rather than allocating on-demand per input. Trades predictable latency + cache locality + no-GC for a load-bearing size cap.

Why preallocate

On CDN-scale hot paths (µs-budget per request):

  • No runtime allocation. malloc() / Box::new() / Vec::with_capacity() introduce latency variance.
  • No GC pressure. Languages with tracing GCs (Java, Go, JavaScript) pause. Languages without (Rust) still have the cost of destructor runs on drop.
  • Cache locality. A single contiguous region is predictable for hardware prefetchers.
  • Predictable memory footprint. Easier capacity planning; OOM conditions moved from "during traffic burst" to "at startup, loudly."

The hidden hazard

Preallocation turns the size cap into a load-bearing invariant. If any upstream producer ever generates a payload exceeding the cap, the consumer must:

  • (a) fail-open — log + fall back to a known-good prior payload.
  • (b) fail-closed — panic / error / drop traffic.
  • (c) silently truncate — correctness risk.

Absent an explicit fail-open path, fail-closed is the default via .unwrap() / assertion / out-of-bounds access. At CDN-scale, fail-closed means 5xx for every request that hits the module.

Canonical instance

Cloudflare's Bot Management module on FL2 preallocated for 200 features (see concepts/feature-file-size-limit) — well above the ~60 actually in use. On 2025-11-18, a ClickHouse permission migration caused the feature-file generator to produce ~doubled rows. The check on file size was a Rust .unwrap() on a bounds check; it panicked; every fl2_worker_thread died; ~3 hours of core-traffic 5xx. The load-bearing size invariant had no fail-open path.

See sources/2025-11-18-cloudflare-outage-n-november-18-2025.

Remediation shape

The discipline is not to avoid preallocation — preallocation is correct for CDN-scale hot paths. The discipline is:

  1. Validate at ingest — before the payload reaches the preallocated buffer, check size / shape / value ranges.
  2. Fail open on violation — fall back to a known-good prior payload; log the violation; alert operators.
  3. Never treat internally-generated input as trusted — see concepts/internally-generated-untrusted-input.

Captured in patterns/harden-ingestion-of-internal-config.

Seen in

Last updated · 200 distilled / 1,178 read