Skip to content

CONCEPT Cited by 1 source

Cache-miss concentration

Definition

Cache-miss concentration is the principle that routing all cache misses from many edge POPs through a single intermediary (upper tier) data center maximizes that intermediary's cache hit ratio. Because the upper tier sees the union of demand from all edges, its effective working set covers far more of the origin's content than any single edge POP could.

Why It Works

  • N edge POPs each see a fraction of total demand → each has a partial working set → more misses.
  • One upper tier sees the sum of all N edges' misses → builds a complete working set → far fewer misses reach origin.
  • The concentration effect is multiplicative: doubling the number of edge POPs feeding one upper tier roughly doubles the cache's effective demand without doubling the content volume.

Trade-off

The benefit of concentration (higher hit ratios, fewer origin connections) is maximized with a single upper tier. Adding more upper tiers dilutes the effect:

"By concentrating cache misses at one data center, you get higher cache hit ratios, fewer connections to your origin, and lower latency on origin pulls."

— Cloudflare, Smart Tiered Cache for Public Cloud Regions (2026-07-10)

When Smart Tiered Cache can't pick a single winner (e.g., for anycast origins), it falls back to multiple upper tiers — "nothing breaks, you just lose the thing that made a single closest tier worth it."

Seen in

Last updated · 575 distilled / 1,757 read