Skip to content

CLOUDFLARE 2026-10-23 Tier 1

Read original ↗

Cloudflare: Origin Range Requests — serving large files without range-request tuning

Summary

Cloudflare rebuilt how its CDN cache satisfies HTTP range requests for large files and shipped the result as Origin Range Requests, a self-service feature on all plans configured by a single Cache Rule (origin_range_requests: { "mode": "on" }). The old implementation — modeled on nginx's cache-slicing — mapped every client request onto fixed-size ranges whose size was a per-zone static configuration knob, part of the cache key. That knob was a lose-lose tradeoff: a large range (R2 ran 64 MiB) meant a small seek that "wouldn't fill a floppy disk" could force Cloudflare to pull 128 MiB from origin and move it between data centers — often after the viewer had already left; a small range turned one request into a long line of tiny origin fetches, each holding memory while it waited. The move from nginx to a Pingora-based cache server — which already stores every cached file on disk in fixed-size chunks to support resumable downloads — let Cloudflare delete the setting entirely: the range size is just the chunk size. A range request now fetches origin ranges aligned to chunk boundaries, serves already-cached bytes directly, and a per-request coordinator composes one correct 206 Partial Content response out of cached data, an in-flight download it can attach to, and a freshly-fetched suffix — validating that every byte belongs to the same representation (If-Range, stable total size, consistent ETag) before composing. Moving R2 onto the new implementation halved p99 time-to-first-byte on both hits and misses, and a follow-up that lets the coordinator combine on-disk bytes with upstream bytes in a single response cut tiered-cache traffic for R2 and similar workloads ~6% week over week — petabytes less data moving between data centers per day.

Key takeaways

  1. The configurable range size was the root problem, not a tuning opportunity. The previous design (an nginx-style cache-slicing scheme) made the slice size a per-zone static knob that was part of the cache key, so "no value works equally well for a small seek, a sequential download, a large media file, and every origin and cache topology" — and changing it invalidated all cached entries. "Our internal cache limitations were directly exposed to customers, who should not need to understand our capabilities in-depth to have files served efficiently." (Source: sources/2026-10-23-cloudflare-origin-range-requests)
  2. Fixed-size chunks the cache already used became the unit of fetching. Pingora stores every cached file on disk in fixed-size chunks (originally to make interrupted downloads resumable). "If the cache already thinks in chunks, the range size doesn't need to be a setting at all, it can just be the chunk!" A request for bytes 150-250 fetches the aligned chunk containing them; the downstream response still contains only 150-250.
  3. The extra work is now bounded and predictable, not customer-tuned. A sub-chunk request can still require fetching the whole chunk containing it, and non-aligned ranges take on extra work — but "the extra work is bounded and predictable, and no longer determined by a customer-selected range size that can cause unpredictable effects on us and origins."
  4. The hard part is a coordinator, not a cache lookup. One request can span cached data, an origin response, and downloads already in progress. The edge: (1) looks up asset headers + range-coverage metadata, (2) reads what it can from disk or in-memory streaming, (3) fetches missing coverage from origin using aligned ranges, (4) writes the new bytes and records the completed range in metadata, (5) combines the pieces and sets Content-Range / Content-Length for the downstream response. "The internal origin response can be larger than the downstream response, but the downstream status and headers describe only the requested range."
  5. Concurrent requests attach to an existing download instead of starting a new origin fetch. "If an existing download covers another request's bytes, the second request can attach to that download instead of starting another origin fetch. This shares work during bursts for popular large files." — the range-aware form of request collapsing. The old nginx cache couldn't combine range caching with concurrent streaming: turning both on cut responses short, so each range had to be fully downloaded before anyone could read it, and "popular files needed a separate origin fetch for every range."
  6. Composition is gated on representation validation. Composed range responses are valid "only when all bytes belong to the same representation." The coordinator checks each origin response matches the requested range, the total asset size is stable, and the ETag stays consistent across subrequests. If-Range lets a client request a range only if the representation still matches a known validator; on mismatch Cloudflare returns the full representation instead of combining bytes from different versions. A satisfiable range → 206 Partial Content with accurate Content-Range; unsatisfiable → 416 Range Not Satisfiable; a response that can't safely be ranged → served as the complete representation.
  7. Decompression is an explicit don't-range-this hazard. Cloudflare "avoids composing ranges for a representation it is about to decompress because the range over a set of compressed bytes would be different from those selected by the range in a decompressed response" — in those cases it serves the transformed full response rather than advertising a range it cannot honor correctly.
  8. Production numbers (R2, the largest internal user). Old configured range size 64 MiB, so a small boundary-crossing read could cost 128 MiB of inter-data-center transfer. After migration: p99 TTFB dropped ~half on both cache hits and misses across R2's storage regions. A later change taught the coordinator to combine on-disk bytes with upstream bytes in a single response (before, a request was served either entirely from cache or entirely from upstream) → tiered-cache traffic for R2 and similar workloads dropped ~6% week over week ("petabytes less data moving between our data centers every day").
  9. Self-service, zero tuning. "There's no range size to tune and no need to contact support." Turn it on with a Cache Rule; Cloudflare handles chunking + coordination. Intended for progressive/seekable video (MP4), resumable downloads, software/package repositories, disk images + archives, AI training/inference workloads transferring large model files or fetching selected regions of model artifacts, and clients fetching metadata or a small region of a larger file. Cloudflare intends to make it the default way certain file types are served, modifiable by Cache Rules.
  10. The client still decides whether to range; Cloudflare only changes how it satisfies the request. "The client decides whether to request a range, and Cloudflare does not change that request." The feature also does not change cache eligibility (an uncacheable response stays uncacheable), and the origin must support byte ranges for partial origin fetches to help.

Systems / concepts / patterns extracted

  • Systems: Cloudflare Cache (the HTTP cache layer whose range-request handling was rebuilt — primary subject), R2 (largest internal user; the migration's headline beneficiary — ~½ p99 TTFB, ~6% tiered-cache traffic cut), Pingora (the proxy whose fixed-size on-disk chunk storage made the range size redundant), Tiered Cache (the hop whose cross-data-center traffic the ~6% reduction applies to), nginx (the previous cache-slicing model Cloudflare moved off of — named as prior art, no page).
  • Concepts: HTTP range request (NEW page — primary subject, ≥2 prior source mentions), cache hit rate (chunk-aligned storage + coordinator combining cached + upstream bytes), egress / backbone-transfer cost (the 128 MiB-for-a-seek waste and the ~6% petabyte-scale reduction), head-of-line blocking (old design: requests in the same range blocked each other regardless of how far apart their bytes were), request collapsing (attach-to-in-flight- download for popular large files), self-service infrastructure (no range size to tune, no support ticket), content negotiation (one URL → multiple valid representations; If-Range/ETag validation guards range composition against representation drift), goodput (bounded extra fetch work vs. a tunable multiplier).
  • Patterns: caching proxy tier (the byte-range-aware coordinator is a new facet of the edge caching proxy).
  • Recorded as prose + tags only (single-source named ideas; Lint candidates at ≥3 citations): fixed-size-cache-chunk / cache-slicing (the Cloudflare-specific chunk-aligned-origin-fetch implementation detail — nginx prior art + Pingora chunk storage, not yet a reusable minted pattern), range-request-coordinator (compose one response from cached + in-flight + freshly-fetched bytes), chunk-size-replaces-config-knob (delete the tuning parameter by reusing an internal storage primitive), don't-range-what-you'll- decompress, representation-validation-before-composition.

Operational numbers

  • Old R2 configured range size: 64 MiB → a boundary-crossing small read cost 128 MiB of inter-data-center transfer.
  • Old configured range sizes across all users: tens of KB to tens of GB.
  • p99 TTFB dropped ~half on both cache hits and misses across R2's storage regions after migration.
  • ~6% week-over-week reduction in tiered-cache traffic for R2 and similar workloads after the cached-plus-upstream combining change ("petabytes less data moving between our data centers every day").
  • Worked chunk example: a request for bytes 150-250 fetches the aligned chunk containing those bytes; downstream response still contains only 150-250.
  • Status-code contract: satisfiable → 206 Partial Content + accurate Content-Range; unsatisfiable → 416 Range Not Satisfiable; unsafe-to-range → full representation.

Caveats

  • No absolute latency figures (only relative: ~½ p99 TTFB, ~6% traffic) and no chunk-size value is disclosed.
  • Sub-chunk requests can still require fetching the whole containing chunk, and non-aligned ranges take on extra (but bounded) work — the feature trades a tunable, unpredictable multiplier for a fixed, predictable one, not zero overhead.
  • Does not change cache eligibility; the origin must support byte ranges for partial origin fetches to be useful.
  • Rollout framing: Cloudflare migrated existing old-range-caching users onto the new implementation starting with the worst-offending workloads before opening the self-service feature to all plans.

Source

Last updated · 854 distilled / 2,448 read