Skip to content

CONCEPT Cited by 2 sources

Roofline model

Definition

The roofline model (Williams, Waterman & Patterson, 2009) is a visual performance model that bounds the attainable throughput of a kernel on a given machine as:

P_attainable = min( P_peak , AI × BW )

where P_peak is the hardware's peak compute throughput (FLOPs/s), BW is memory bandwidth (bytes/s), and AI (arithmetic intensity) is the kernel's FLOPs performed per byte moved. Plotting P_attainable against AI gives the "roofline": a diagonal memory roof (AI × BW) meeting a horizontal compute roof (P_peak) at the ridge point — the minimum AI at which a kernel can be compute-bound.

  • Left of the ridge → the kernel is memory (bandwidth) bound; only moving fewer bytes per FLOP (raising AI) helps.
  • Right of the ridge → the kernel is compute-bound; the arithmetic units themselves are the limit, and only more/faster FLOPs help.

It is the canonical lens for the compute-vs-memory-bound question, and the ridge point is where the two roofs cross.

Why it matters

The roofline separates two fundamentally different optimization problems. A kernel pinned to the memory roof will not get faster from wider SIMD or more cores — the fix is data reuse (raising arithmetic intensity) so the same loaded byte feeds more FLOPs. A kernel already on the compute roof can only be helped by reducing FLOPs or using faster arithmetic (e.g. lower-precision FMA/MMA). Crossing the ridge is necessary but not sufficient: clearing the DRAM roof only moves the bottleneck down to cache bandwidth and then FMA latency, which is why cache- and register-blocking matter even for a compute-bound kernel (concepts/cache-locality).

Cache hierarchy appears as multiple memory roofs: L1/L2 roofs typically sit 30–60× above the DRAM "fair share," so a kernel whose working set exceeds cache is bounded by the DRAM roof — every byte of a cache-busting input crosses the DRAM boundary at least once.

Canonical wiki instance: batch vector search (NEAREST BY)

Databricks used the roofline to explain why a fully Photonized but naive batch vector-search plan was still far from optimal. The NEAREST BY shape yields nq × nb scores from only nq + nb distinct vectors — each base vector is needed by every query — so there is enormous potential reuse, but the plan shape determines whether the kernel can exploit it. FLOPs are identical (2 × d × nq × nb) across plans; only the bytes moved change:

Plan Bytes moved Arithmetic intensity Scaling
Pairwise cross join (every operand loaded fresh, used once) 2 × 4 × d × nq × nb 1/4 O(1) — pinned to DRAM slope at 0.8% of peak
Fused GEMM (buffer Q, stream B) 4 × d × nb nq/2 O(nq) — crosses the ridge at nq = 64, rides the compute roof

The near-optimal distance/similarity SIMD kernels couldn't help the naive plan because "the pair explosion multiplies FLOPs and bytes equally" — batching alone can't raise AI when there's no reuse to exploit. The fix was a fused operator plan that lets the kernel exploit data reuse; arithmetic intensity then follows. (On a reference m6i.2xlarge: 204.8 GFLOP/s fp32 FMA per core; a 64-executor × 4-core cluster peaks at ≈ 52.5 TFLOP/s; constants are illustrative and shift with hardware.) (Source: sources/2026-10-05-databricks-nearest-by-join-scaling-vector-search-in-databricks-runtime)

Seen in

Merged aliases

  • arithmetic-intensity
  • ridge-point
  • operational-intensity
Last updated · 854 distilled / 2,448 read