Skip to content

CONCEPT Cited by 1 source

Tiered caching

Definition

Tiered caching (in the CDN context) is an architecture where cache misses from edge POPs do not go directly to the customer's origin server. Instead, they pass through one or more upper-tier (also called "mid-tier" or "shield") data centers that maintain their own caches. This additional cache layer absorbs repeat requests before they reach origin, reducing origin load and improving hit ratios.

Client → Edge POP (L1 cache) → Upper-Tier POP (L2 cache) → Origin

Why It Works

  • Cache-miss concentration: by funneling misses from many edge POPs through a single (or small set of) upper tiers, the upper tier sees the union of all edge demand and can satisfy a much larger fraction from its own cache.
  • Fewer origin connections: origin sees traffic from the upper tier, not from hundreds of edge POPs independently.
  • Better hit ratios: the working set at the upper tier is the aggregate of all edge demand — much more likely to have a given object cached.

Trade-offs

Benefit Cost
Higher cache hit ratios Extra network hop (edge → upper tier → origin)
Lower origin load Upper-tier failure becomes a single point of failure for misses
Simpler origin scaling Topology selection complexity

Smart vs. Static Topologies

  • Static tiered cache: administrator manually assigns edge POPs to upper tiers (often by geographic proximity).
  • Smart/dynamic tiered cache: the system continuously probes latency and selects the optimal upper tier per origin automatically (as in systems/cloudflare-smart-tiered-cache).

Distinct from Meta's "Tiered Cache as Planetary Memory"

Not to be confused with concepts/tiered-cache-as-planetary-memory, which borrows the metaphor for a storage hierarchy (GPU memory → SSD → regional blob → global blob). The CDN concept here is specifically about HTTP cache miss routing through an intermediate data center.

Seen in

Last updated · 575 distilled / 1,757 read