Skip to content

PATTERN Cited by 1 source

Starlark configuration DSL

Use Starlark (Bazel's Python-subset configuration language) as the authoring surface for system configuration, evaluated at init-time in a VM to produce a Protobuf-structured config definition the system's core then consumes. Operators express complex composition (nested policies, conditional routing, inheritance) in a real (but hermetic) language instead of deeply-nested YAML/JSON.

Why a configuration program instead of a configuration file

  • Expressive composition. Engine trees, per-tenant policies, per- cluster rule-sets, conditional routing — these are inherently recursive / compositional. Flat key-value config or deeply-nested YAML either imposes artificial structure or is unreadable.
  • Reuse via functions and variables. Declare redis_foo = enginepb.Redis(...), reference it from multiple rule sites. A YAML author duplicates; a Starlark author abstracts.
  • Safer than Python. Starlark is deterministic + hermetic + no I/O + no import-time side effects + finite recursion — "config program" properties without "running arbitrary scripts in prod" risk.
  • Config-as-code ergonomics without binary redeploys. The core server binary stays stable; operators ship config-program changes separately. Engine-tree behavior can evolve as a config-only delivery.
  • Well-known, tested VM. Bazel, Buck, Tilt, Pants, Copybara all use Starlark; implementations are mature; existing IDE tooling works.

Canonical shape

# User-authored Starlark program.
def main():
    redis_foo = enginepb.Redis(...)
    redis_bar = enginepb.Redis(...)

    bar_router = enginepb.Router(
        rules=[
            enginepb.Rule(
                prefix=enginepb.Rule.Prefix(prefix="bar:"),
                engine=redis_bar,
            ),
            enginepb.Rule(  # passthrough / default
                engine=enginepb.Static(
                    reply=respcpb.Reply(message="rejected"),
                ),
            ),
        ],
    )

    cmd_router = enginepb.Router(
        rules=[
            enginepb.Rule(
                command=respcpb.Schema(name="GET"),
                engine=redis_foo,
            ),
            enginepb.Rule(
                command=respcpb.Schema(name="SET"),
                engine=bar_router,
            ),
        ],
    )

    return cmd_router

Server init runs main() in a Starlark VM; the returned object is a Protobuf message validated against the server's expected schema; server's data plane executes per that structure.

Load-bearing properties

  • The return value is typed Protobuf — config-program authoring is still constrained by a schema; it's not "runnable code that does whatever."
  • Primitives are imported from a Protobuf-derived namespace (enginepb.Redis, enginepb.Router, respcpb.Reply) — each primitive maps 1:1 to a runtime component. Composition in Starlark = composition of runtime components.
  • Evaluated once, at init-time. The program doesn't run on every command; it assembles the runtime tree, which is then frozen / interpreted by the core binary.
  • Hermetic + deterministic. Starlark has no filesystem / network I/O; the same program always renders the same Protobuf output. Safe to run in CI for validation, in staging for dry-run, in prod for activation — all producing identical structure.

Comparison to the alternatives

  • Deeply-nested YAML/JSON: forces all composition into the data tree. Reuse is copy-paste. Conditionals and computed rule sets require a templating layer (Helm, Kustomize, Jinja) that reintroduces unsafe Python-ish semantics without Starlark's discipline.
  • Full Python. Trades hermeticity + determinism for expressiveness. Import-time side effects, filesystem access, network calls make the config program's output depend on environment — the exact bug class Starlark was designed to eliminate.
  • In-binary DSL (code-as-config compiled into the binary). Requires a new binary deploy for every config change — the scenario Starlark config explicitly avoids.
  • Feature flags. Orthogonal, not substitute — flags turn individual behaviors on/off; Starlark authors the shape of the behavior graph.

Caveats

  • Learning curve. Starlark is Python-like but not Python — newcomers trip on frozen collections, load() semantics, lack of generic recursion, no while.
  • Debugging. A Starlark program producing wrong Protobuf output is harder to debug than an explicit YAML file; the indirection is load-bearing but real.
  • Runtime error surface. The server must validate the rendered Protobuf + surface structured errors clearly; an invalid engine- tree shape that crashes the server at init is a regression Starlark-as-config makes more likely without schema rigor.
  • Tooling investment. Editor integration / linting / dry-run rendering / diff-visualization all require in-house tooling since the Protobuf output is the semantic unit, not the Starlark source.

When to reach for this pattern

  • Composition-heavy config that YAML can't express cleanly (engine trees, policy hierarchies, routing DAGs).
  • Operator-authored runtime behavior where binary redeploys per change are unacceptable.
  • Correctness-critical config where hermeticity matters (same input → same output, no environmental drift).

Seen in

  • sources/2026-04-21-figma-figcache-next-generation-data-caching-platform — FigCache uses Starlark as the authoring surface for its engine tree: a "custom configuration system that models engine configuration as a Starlark program, dynamically evaluated at runtime in a virtual machine, which renders a Protobuf-structured configuration definition consumed by FigCache's backend." The quote from the post: "This enables operators to express complex runtime behaviors exclusively in configuration, without requiring heavyweight changes to core business logic in the server or deployments of updated server binaries. For example, command-type splitting, key-prefix–based routing and rejection, and proxying to distinct Redis clusters can be modeled as a composition of a few primitive engine building blocks."
Last updated · 200 distilled / 1,178 read