Skip to content

SYSTEM Cited by 1 source

SDDL (Simple Data Description Language)

SDDL — Simple Data Description Language — is the declarative way to describe byte layouts to OpenZL for format-aware compression. "With the Simple Data Description Language (SDDL), you sketch how the bytes map to fields — rows, columns, enums, nested records." (Source: sources/2025-10-06-meta-openzl-an-open-source-format-aware-compression-framework.)

Project

  • API docs: facebook.github.io/openzl — SDDL API
  • Ships as part of OpenZL; open-sourced 2025-10-06.
  • Wiki role: stub. Entry exists to anchor the OpenZL architecture; deepening expected when Meta publishes a dedicated SDDL post or grammar reference is transcribed.

What SDDL is (and isn't)

  • It is: a parser-only description of how bytes map to typed fields — rows / columns / enums / nested records. Meta's framing is explicit: "SDDL is for parsing only; it just tells OpenZL the shape of your data."
  • It is not: a logic-bearing DSL. SDDL doesn't carry transforms, validation, or conditionals — only shape. The actual compression decisions come from the OpenZL trainer, which consumes the SDDL shape + sample data and produces a Plan.

Two ways to tell OpenZL the shape

From the post (Source: sources/2025-10-06-meta-openzl-an-open-source-format-aware-compression-framework):

  1. SDDL description — declarative; the simple path.
  2. Parser function — "you can write your own parser function directly using one of the supported languages, and register it with OpenZL to delegate the logic." Used when SDDL isn't expressive enough.

Meta explicitly names the expressiveness ceiling: "We also are actively working to extend SDDL to describe nested data formats more flexibly." As of 2025-10-06, complex nested formats fall through to the parser-function path.

Role in the compression pipeline

The canonical loop described in the OpenZL post:

"describe (SDDL) → train (produce a plan) → compress (emit frames with resolved graphs) → decode anywhere with the same binary."

SDDL lives in the describe step; its output (the shape) is trainer input. At decode time SDDL is not in the loop — the frame carries the resolved graph directly, and the universal decoder runs it without re-parsing any description.

Seen in

Last updated · 319 distilled / 1,201 read