Skip to content

CONCEPT Cited by 3 sources

Content negotiation

Definition

Content negotiation is the HTTP mechanism by which a single URL can return more than one correct representation, with the server selecting among them based on request headers. The client expresses preferences through Accept (media type), Accept-Language (language), Accept-Encoding (compression), and similar fields; the origin picks a representation and — critically — declares which request fields influenced its choice via the Vary response header (RFC 9110). Common uses: serving HTML to browsers and JSON to API clients from /catalog, English/French/German from one page, or gzip/Brotli/zstd bodies from one asset.

One URL, many representations is the whole point — and also the whole problem for anything that caches between client and origin.

Why it matters for system design

Content negotiation is where API design meets caching. Two forces pull against each other:

The load-bearing observation is that the origin knows which responses it can serve; the cache needs to know which requests can reuse each response. Vary communicates the first half (which headers matter) but not the second (which value differences matter) — so a correct-but-naive implementation is both valid and useless.

Mechanisms

Server-driven vs. the Accept-rewrite variant

  • Standard server-driven negotiation. Origin inspects request headers, returns the matching representation, and sets Vary to name the headers it used. Intermediary caches (CDNs) then key on those fields.
  • Edge/rewrite negotiation for agents. Vercel's markdown-for-agents work routes on the Accept header at the edge (a next.config rewrite) to serve a markdown representation of a page to agents while humans get HTML — content negotiation used as a token-efficiency lever rather than a language/format one. (Source: sources/2026-04-21-vercel-making-agent-friendly-pages-with-content-negotiation; see concepts/machine-readable-documentation.)

Normalization vs. passthrough (taming variant cardinality)

Because input cardinality (every possible Accept-Language string) vastly exceeds output cardinality (a handful of supported languages), CDNs offer to normalize negotiation headers before variant selection — lowercase, sort by quality value so client ordering doesn't matter, strip parameters, and fold regional tags to base languages — so equivalent requests share one cached entry. The alternative, passthrough, keys on raw bytes and is correct only when the value set is small and controlled. Bypassing cache entirely is the escape hatch for personalized headers. (Source: sources/2026-09-22-cloudflare-we-just-shipped-support-for-the-ugliest-part-of-http-vary; see systems/cloudflare-cache.)

Vary: *

A response of Vary: * declares that any aspect of the request — including information outside the HTTP message, like the client's IP — may affect the response. Caches treat it as uncacheable/bypass: they cannot safely reuse the response for a later request without contacting the origin.

Seen in

Last updated · 766 distilled / 2,225 read