Skip to content

PATTERN Cited by 1 source

CI regression-budget gate

Shape

A CI regression-budget gate is the pattern of (1) identifying a scalar cost proxy that correlates with a real system property (build time, binary size, memory footprint, request latency, data-egress bytes, bundle size), (2) measuring it cheaply enough to run in CI on every PR, and (3) surfacing per-PR deltas with block/warn semantics.

The pattern is pre-emptive: stop the regression at authoring time before it hits production or a slow developer experience. Without the gate, regressions accumulate silently and require post-hoc bisect to localize.

Canonical realization: includes.py

Figma's systems/includes-py is the pattern applied to C++ build times:

  • Cost proxy: post-pre-processing bytes per translation unit (bytes are proportional to build time).
  • Cheap measurement: pure-Python crawl of the include DAG, no Clang invocation, runs in seconds.
  • CI gate: warn on PRs whose per-file byte delta crosses a significance threshold; block merge until addressed.
  • Outcome: 50-100 regressions / day caught at PR time that would otherwise have landed.

Invariants that make this pattern work

  1. The proxy actually proxies the cost. If the correlation is weak or monotone-but-noisy, the gate fires on irrelevant changes and loses trust. Figma's bytes-proportional model is measurable and direct.
  2. Measurement must be cheap. The gate runs in CI on every PR. If it doubles CI wall time, it will be routed around. Figma's "couple of seconds" measurement cost is well inside the budget.
  3. Actionable at the PR author's level. The author has to be able to look at the gate output and understand what to change. Figma pairs the gate with two canonical recipes: delete an unused include (systems/diwydu catches these separately) or forward-declare (patterns/centralized-forward-declarations makes the fix one line).
  4. Per-PR delta, not absolute value. A file already expensive shouldn't block every PR — only the ones that make it worse.
  5. Significance threshold, not zero-tolerance. Every CI gate wants to flag real regressions without flapping on noise. Figma's "significant" wording deliberately leaves room for calibration.

Why pre-emption beats post-hoc cleanup

The same article describes Figma's initial one-shot cleanup of unused includes in the largest files:

  • −31% compiled bytes
  • −25% cold build time

This is the upper bound on what any retrospective cleanup can deliver. But it erodes immediately without a gate: Figma's original problem statement was that "build times gradually reverted to their original pace" after earlier improvements (M1 Maxs, Ccache, remote caching). The gate is what makes the one-shot gains durable — 50-100 regressions/day would otherwise have climbed the cost back up.

Where it generalizes

The pattern is common across dimensions where cost is measurable per-artifact and summable:

  • Binary size: bundle-size CI checks on JS / Android / iOS PRs.
  • Docker image size: layer-diff gates on PRs touching Dockerfiles.
  • Cold-start memory: serverless cold-start budget gates on image builds.
  • Front-end perf metrics (Lighthouse budgets, Core Web Vitals gates) — the same shape applied to page weight and render-blocking bytes.
  • Data egress (patterns/chargeback-cost-attribution) — a cousin pattern. Chargeback attributes after-the-fact costs; the regression- budget gate prevents the cost landing in the first place.

Contrasts

  • vs. CI alerts / flake monitors: those react to failures or misbehavior; the regression-budget gate fires on a measured scalar changing in a direction known to be bad.
  • vs. post-merge rollback: gate catches at PR; rollback catches after impact. Gate is cheaper per incident.
  • vs. build-system caching (concepts/content-addressed-caching): caching makes re-runs cheaper but doesn't reduce the underlying cost; a regression still materialises on a cold build or when any transitive input changes. The gate attacks the root number; caching compounds with it.

Composition with detection tools

includes.py and systems/diwydu are a two-tool split on detection class:

Class Detector Fix
Included, never directly used DIWYDU (AST) Delete the #include
Directly used, but transitively huge includes.py (byte DAG) Forward-declare (patterns/centralized-forward-declarations) / split header

The gate is on the measurement tool, not the detection tool — the measurement layer is what gives per-PR actionability.

Seen in

Last updated · 200 distilled / 1,178 read