Skip to content

CONCEPT Cited by 1 source

Memory-bandwidth-bound

Definition

A computation is memory-bandwidth-bound when its throughput is limited by the rate at which data can be read from (or written to) main memory, rather than by the CPU's arithmetic or instruction throughput. The computation consumes data as fast as the memory subsystem can deliver it — adding more compute (wider SIMD, higher clock) won't help; only faster memory or reduced data movement will.

Significance

Reaching memory bandwidth is the theoretical ceiling for any operation that must touch every byte of its input exactly once. When an algorithm achieves memory-bandwidth throughput, it means:

  • The CPU is never idle waiting for instructions to execute
  • The pipeline is never stalled on branch mispredictions
  • All data arrives from cache/memory just in time for processing
  • No wasted work (redundant loads, unnecessary stores, useless computation)

Canonical wiki instance

GitHub's casefold ASCII fast path runs at >45 GiB/s on Apple M4 — essentially the machine's memory bandwidth ceiling for a single-pass read-modify-write sweep. This is achieved by:

  1. Eliminating all data-dependent branches (enabling full auto-vectorization)
  2. Processing 16 bytes per NEON vector instruction
  3. Performing exactly one read + one write per cache line
  4. No allocations, no function calls, no system calls in the hot loop

The two-pass variant (scan for ASCII, then convert) reaches ~23 GiB/s — half bandwidth because it reads the data twice. Still 7× faster than the naive branch-per-byte loop.

(Source: sources/2026-07-31-github-dont-stop-early-case-folding-source-code-at-memory-speed)

Diagnostic

An operation is likely memory-bandwidth-bound when: - Throughput scales linearly with memory bandwidth (faster RAM → faster code) - Throughput is independent of computational complexity (simpler arithmetic doesn't help) - Throughput matches the machine's stream benchmark or memcpy speed - Adding more compute lanes (wider SIMD) shows diminishing returns

Seen in

Last updated · 607 distilled / 1,851 read