Skip to content

CONCEPT Cited by 1 source

Shared-dictionary compression

Shared-dictionary compression is an HTTP-level compression technique in which server and client share a reference payload (the "dictionary") that both sides know about, so the server can send only the difference between the new response and the dictionary instead of compressing the new response from scratch.

Canonically expressed in RFC 9842: Compression Dictionary Transport with Cloudflare's 2026-04-30 beta as the first CDN-tier deployment.

The core insight

Traditional compression is stateless: every response is compressed as if the client had never seen anything before. Gzip builds a dictionary in real time by finding patterns inside the response being compressed. Brotli ships with a built-in static dictionary of common web patterns. Zstandard accepts a custom dictionary you pre-compute from representative content.

Shared-dictionary compression takes the custom-dictionary idea further: the previously cached version of the resource becomes the dictionary.

The deploy problem: a team ships a one-line fix to a 500 KB JS bundle. The bundler re-chunks, filenames change (bundler chunk invalidation), every user's browser sees a new URL, cache misses, full 500 KB re-downloaded. With shared-dictionary compression, the browser still has app.bundle.v1.js cached — the server compresses app.bundle.v2.js against v1 and sends only the diff. A 500 KB bundle with a one-line change becomes a few kilobytes on the wire.

The handshake (RFC 9842)

  1. First fetch: server serves the resource with a Use-As-Dictionary response header — "hold onto this, it will be useful as a dictionary for future requests matching this URL pattern."
  2. Subsequent fetch: browser sends Available-Dictionary: <hash> request header — "here's the dictionary I have cached."
  3. Server responds with Content-Encoding: dcb (delta- compressed Brotli) or dcz (delta-compressed Zstandard) and a body that is the diff against the cached dictionary.
  4. Browser decompresses the diff using its cached dictionary as reference, reconstructing the full new response.

See concepts/delta-compression-http for the encoding-level details.

Persistent savings across the release history

The dictionary-lineage compounds: v2 compresses against v1, v3 against v2, v47 against v46. "The savings don't reset, they persist across the entire release history." (Source: sources/2026-04-17-cloudflare-shared-dictionaries-compression-that-keeps-up-with-the-agent)

Empirical compression ratios

From Cloudflare's 2026-04 lab test + demo:

  • Lab test: 272 KB uncompressed bundle → 92.1 KB gzip (66 %) → 2.6 KB DCZ with v(n-1) as dictionary (97 % over gzip).
  • canicompress.com demo: ~94 KB SPA bundle, new deploy every minute, only a config block changes → diff on wire ~159 bytes (99.5 % over gzip).

Ratio is content-dependent — best-case is versioned bundles where most content is unchanged framework/library code (the canonical target). Ratios degrade for larger inter-version diffs (dependency bumps, refactors, feature work).

Constraints and failure modes

  • Same-origin constraint ([[concepts/same-origin-dictionary- scope]]): RFC 9842 dictionaries are only usable for responses from the same origin that served them. Closes most of the CRIME/BREACH attack surface that sank SDCH (2008-2017). Residual same-origin side-channels remain possible when sensitive data compresses alongside attacker-injected content.
  • Cache-variant explosion (concepts/cache-variant-explosion): cache keys must vary on Available-Dictionary + Accept-Encoding, so mid-deploy the edge holds multiple cache variants per URL (gzip / br / zstd / dcz-v1-dict / dcz-v2-dict / …).
  • Client-support gap: Chrome 130+, Edge 130+ advertise dcb/dcz; Firefox in progress, Safari not committed. Clients without support fall back silently to standard compression.
  • Origin-implementation complexity: dictionary generation + header emission + on-the-fly delta compression
  • fallback + cache-variant management is hard enough that Cloudflare argues "this is a coordination problem that belongs at the edge" (patterns/edge-managed-protocol-complexity).

Sibling concepts

  • concepts/git-delta-compression — delta compression in Git's object layer: candidate pairs picked by path-trailing- 16-chars heuristic, sort-and-window to keep it O(N·window) instead of O(N²). HTTP shared-dictionary compression simplifies the pairing: previous-version-of-this-URL is always the candidate dictionary. Sibling at a different layer.
  • concepts/delta-compression-http — the specific HTTP encoding wire format (dcb, dcz).

Seen in

Last updated · 200 distilled / 1,178 read