Skip to content

CLOUDFLARE 2026-07-10

Read original ↗

Improving Smart Tiered Cache for Public Cloud Regions

Summary

Cloudflare describes how their Smart Tiered Cache system — which selects a single optimal upper-tier data center between edge and origin to maximize cache hit ratios — fails when origins sit behind anycast or regional unicast IPs on public clouds (AWS, GCP, Azure, Oracle Cloud). The latency probes that normally identify the closest upper tier return ambiguous results because the cloud provider's front-end IP appears equally close to many data centers simultaneously. The fix: allow customers to provide a cloud region hint, which Cloudflare maps to the correct upper tier using periodically fetched cloud-provider IP-range files and a weighted-voting algorithm. Includes a novel anycast-detection mechanism based on the speed of light in fiber.

Key Takeaways

  1. Single upper tier concentrates cache misses — Smart Tiered Cache's core value: by funneling all cache misses through one data center, you get higher cache hit ratios, fewer connections to origin, and lower latency on origin pulls.

  2. Anycast origins break latency-based tier selection — when a cloud provider uses anycast or regional unicast for load balancers, the origin appears equally close to many Cloudflare POPs simultaneously. There's no single clear winner from latency probes, so the system falls back to multiple upper tiers (safe but suboptimal).

  3. Hairpinning is the worst-case failure mode — a Chicago upper tier fetching from a Singapore origin adds hundreds of milliseconds of cross-continental latency because the anycast IP misleads the probing system into thinking Chicago is "closest."

  4. Speed-of-light constraint detects anycast — if combined latencies from two checkpoint data centers to an origin are faster than light in fiber could physically travel between the two checkpoints, the origin must be responding from multiple locations. Physics provides a reliable signal where routing metadata cannot.

  5. Region hint as explicit disambiguation — rather than fully automating a problem that requires external knowledge (which cloud region an origin lives in), Cloudflare exposes a simple region-hint knob (e.g., aws:us-east-1) that feeds the topology selector.

  6. Weighted-voting tier selection — cloud provider IP-range files map regions to subnets; subnets map to upper tiers via continuous latency probes (refreshed every 15 minutes). Each matching subnet contributes a weighted vote; the upper tier with the strongest signal becomes the region's primary.

  7. Primary/fallback PoP separation — primary and fallback upper tiers always come from different points of presence, so losing one PoP can't eliminate both paths to origin.

  8. Geographic fallback for sparse-data regions — new or low-traffic cloud regions without enough probe data fall back to the geographically closest Tier-1 PoP, then silently upgrade to data-driven selection as origins come online.

  9. Incremental improvement arc — Smart Tiered Cache has been progressively extended: base (2021) → R2-aware (Nov 2024) → Load-Balancing-pool-aware (Jan 2025) → Public Cloud Regions (Jul 2026). Each step teaches the system about a new origin architecture.

Operational Numbers

  • Smart Tiered Cache is Cloudflare's most popular tiered cache topology (all plans, free).
  • Latency probes refreshed every 15 minutes.
  • Cloud provider IP-range files fetched every few hours.
  • Launched with AWS, GCP, Azure, Oracle Cloud support.

Systems & Concepts Extracted

Systems: - systems/cloudflare-smart-tiered-cache — the tiered cache topology system with latency-probe-based upper tier selection - systems/cloudflare-cache — the underlying HTTP cache layer on every POP

Concepts: - concepts/tiered-caching — the CDN pattern of routing cache misses through an intermediate "upper tier" before hitting origin - concepts/anycast — network routing where one IP is advertised from multiple POPs - concepts/latency-based-tier-selection — selecting the optimal intermediary based on real-time latency measurements - concepts/anycast-origin-detection — detecting when an origin responds from multiple locations using speed-of-light constraints - concepts/cache-miss-concentration — funneling all cache misses through a single upper tier to maximize hit ratio - concepts/hairpin-routing — traffic inefficiently routed to a distant intermediary only to fetch from a nearby origin

Patterns: - patterns/region-hint-for-anycast-origins — explicit cloud-region annotation to guide topology selection when probing fails - patterns/weighted-voting-tier-selection — subnet-level weighted votes to elect the best upper tier per cloud region - patterns/primary-fallback-pop-separation — ensuring primary and fallback tiers reside on different physical PoPs - patterns/latency-probe-based-topology — continuously probing latency to build and maintain a dynamic cache hierarchy

Caveats

  • Region hints are manual — customers must know which cloud region their origin is in and configure it per-IP.
  • Only anycast-detected origins can receive hints via the dashboard; non-anycast origins presumably don't need them.
  • The speed-of-light detection is a heuristic (fiber speed, not straight-line) and may have edge cases with very short probe distances.

Source

Last updated · 575 distilled / 1,757 read