Skip to content

CONCEPT Cited by 1 source

Compression Plan

Definition

A compression Plan is the per-use-case configuration object that an OpenZL trainer produces from a shape description (via SDDL or a registered parser function) and sample data. The Plan encodes:

  • The transform sequence to apply before entropy coding (e.g. split header, AoS → SoA, per-field choice of delta / transpose / tokenize / …).
  • Transform parameters (block sizes, dictionary targets, entropy coder knobs).
  • Optionally, control points — per-frame branch points that read lightweight runtime statistics and pick a subgraph.

At encode time, the Plan is resolved against a specific data frame into a concrete Resolved Graph. The Resolved Graph is then embedded in the frame, so that the universal decoder can read and execute it without any out-of-band configuration. (Source: sources/2025-10-06-meta-openzl-an-open-source-format-aware-compression-framework.)

Why a "Plan" and not a "compression level"

Traditional compressors expose a compression level (-3, -9) — a single knob on a fixed curve. The OpenZL post is explicit about replacing this with Plans as serialized compressor graphs:

"Unlike traditional compressors, which offer configuration by setting a compression level, OpenZL offers configuration by serializing the compressor graph. This allows an immense amount of flexibility to select diverse tradeoffs." (Source: sources/2025-10-06-meta-openzl-an-open-source-format-aware-compression-framework.)

A compression level is scalar; a Plan is a DAG. The trainer can emit a Pareto frontier of Plans (each is a point in speed × ratio × decode-speed space) — or directly target the best Plan under a constraint.

Lifecycle

From Meta's described loop (Source: sources/2025-10-06-meta-openzl-an-open-source-format-aware-compression-framework):

  1. Describe. User provides shape via SDDL or parser function.
  2. Train. The OpenZL trainer runs a budgeted search — cluster finder groups like-behaving fields, graph explorer scores candidate subgraphs — and produces a Plan.
  3. Compress. At encode time, the encoder resolves the Plan into a Resolved Graph for the current frame (picking branches at any control points), records the Resolved Graph into the frame, emits the compressed bytes.
  4. Decode. Any OpenZL decoder reads the Resolved Graph from the frame and executes the inverse transforms. Same decoder binary for every Plan.

Continuous lifecycle via Managed Compression

Managed Compression is the runtime service that operates Plans at Meta fleet scale: monitor registered use cases, sample fresh data, periodically re-train, roll out new Plans when beneficial, keep old frames decoding unchanged. "When a plan improves, you just roll out the new plan, no new decompressor needed. Old data keeps decoding; new data get improved gains."

This is the offline-train-online-resolve pattern in operation — the Plan is the unit of configuration that flows through the rollout pipeline.

Plan vs Resolved Graph

A key distinction in the OpenZL architecture:

  • Plan — pre-training output; may contain control points (undecided branches).
  • Resolved Graph — per-frame realization of the Plan; all branches chosen, embedded in the frame, directly executable by the decoder.

The distinction is what lets a Plan adapt to per-frame variation without producing a new Plan (re-training). Exploration is bounded at encode time by the control-point structure of the Plan, preserving compression-speed guarantees.

Seen in

Last updated · 319 distilled / 1,201 read