Skip to content

CONCEPT Cited by 2 sources

HTTP range request

Definition

An HTTP range request asks a server for part of a resource instead of the whole resource. The client sends a normal GET with a Range header naming a byte interval; if the server supports it, it replies 206 Partial Content and identifies the returned bytes with Content-Range:

GET /movie.mp4 HTTP/1.1
Host: example.com
Range: bytes=1048576-2097151

HTTP/1.1 206 Partial Content
Content-Range: bytes 1048576-2097151/524288000
Content-Length: 1048576

Standardized in RFC 7233 (now folded into RFC 9110 §14). The server advertises support with Accept-Ranges: bytes. Core response codes:

  • 206 Partial Content — a satisfiable range; Content-Range states the returned interval and the total length.
  • 416 Range Not Satisfiable — the requested interval lies outside the resource.
  • A server that cannot (or chooses not to) honor the range returns the complete representation with 200 OK — the client must tolerate this.

Why it exists

Range requests are how large objects are delivered efficiently. Canonical client uses:

  • A video player requests bytes around a seek position instead of the whole file (progressive / seekable MP4).
  • A download manager resumes after an interruption from where it left off.
  • A package manager fetches only part of an artifact.
  • A client inspects the end of a file (e.g. a ZIP central directory, a Parquet footer) without downloading everything before it.
  • An object store / NFS gateway transfers only the requested bytes to minimize data moved and cost — e.g. AWS S3 Files' byte-range reads (Source: sources/2026-04-07-aws-s3-files-mount-any-s3-bucket-as-a-nfs-file-system-on-ec2-ecs).

Because the client decides whether to request a range, a CDN or proxy in the middle does not change the request — it only changes how it satisfies it.

Representation validation (the correctness core)

A range is only meaningful relative to a specific representation of the resource. If the underlying object changes between subrequests, naively stitching together bytes from two versions produces a corrupt response. The guards:

  • ETag / strong validator consistency — a composing proxy must verify the entity tag (and total size) is stable across every subrequest it combines.
  • If-Range — the client sends a range conditional on a known validator (ETag or Last-Modified). If the representation still matches, the server returns 206; if it has changed, the server returns the full current representation instead of a partial one, so the client never splices two versions.

This makes range composition a validated operation, not a byte-offset copy — the single most important invariant a range-serving cache must hold.

Ranges and caching

Caching range requests naively wastes network bandwidth and disk I/O at CDN scale. Two families of approaches:

  • Cache-slicing (fixed-range mapping) — map each client request onto fixed-size byte ranges and cache those slices. The classic form is nginx's cache slicing, where the slice size is a static configuration knob and part of the cache key. The tradeoff is inherent: a large slice over-fetches (a small seek pulls a whole large slice from origin); a small slice fragments one request into many tiny origin fetches. Changing the slice size invalidates cached entries.
  • Chunk-aligned storage reuse — if the cache already stores files in fixed-size on-disk chunks (for resumable downloads), the "range size" need not be a setting at all — it is just the chunk size. The cache serves already- cached chunks directly and fetches only the chunk-aligned ranges it is missing. The extra work becomes bounded and predictable rather than determined by a customer-tuned multiplier. This is the design behind Cloudflare's Origin Range Requests (Source: sources/2026-10-23-cloudflare-origin-range-requests).

A range-serving cache quickly becomes a coordinator: a single downstream 206 can be composed from (a) bytes already on disk, (b) an origin response fetched now, and (c) an in-flight download another request is already pulling — which the second request can attach to instead of starting its own origin fetch (the range-granularity form of request collapsing). The internal origin fetch may be larger than the downstream response; only the downstream status/headers describe the requested range.

Hazards

  • Head-of-line blocking on shared slices. If concurrent requests that fall inside the same cache slice block each other — no matter how far apart their requested bytes are — a popular range serializes unrelated clients. See concepts/head-of-line-blocking.
  • Over-fetch amplification. A large fixed range turns a tiny seek into a large origin pull, multiplied across every hop (origin → edge → tier → client). A 64 MiB range made a sub-MiB read cost 128 MiB of transfer in Cloudflare's pre-migration R2 case (concepts/egress-cost).
  • Ranging over transformed bytes. A range over compressed bytes selects different content than the same range over the decompressed representation. A cache that is about to decompress must not advertise a range it cannot honor correctly — it serves the transformed full response instead.
  • Inconsistent Vary / error responses. The same representation-consistency rules that govern content negotiation apply: if the origin's validators or Vary behavior are inconsistent across cacheable responses, a range can be composed against the wrong representation.

Seen in

  • sources/2026-10-23-cloudflare-origin-range-requests — canonical wiki treatment. Cloudflare's Origin Range Requests rebuild: chunk-aligned origin fetches replace a per-zone range-size knob; a per-request coordinator composes one 206 from cached + in-flight + freshly-fetched bytes; If-Range + ETag + stable-size validation gate composition; decompression is an explicit don't-range hazard; 206/416/full-representation status contract. Production: R2 p99 TTFB ~halved; tiered-cache traffic −6% w/w.
  • sources/2026-04-07-aws-s3-files-mount-any-s3-bucket-as-a-nfs-file-system-on-ec2-ecs — byte-range reads as the transfer-minimizing primitive beneath an NFS-over-S3 gateway (transfer only the requested bytes; ~1 ms latency).
Last updated · 854 distilled / 2,448 read