Skip to content

PATTERN Cited by 1 source

Static pipeline generation

Generate the CI pipeline YAML ahead of time — not at commit-arrival on the critical path. Move any per-commit conditional evaluation (what targets to run, which steps can skip) to a lightweight job-runtime lookup backed by an out-of-band hash computation. Canva took pipeline generation from 10+ min on the critical path to near-zero with this move (Source: sources/2024-12-16-canva-faster-ci-builds).

Intent

A "dynamic" pipeline — one generated per commit by running expensive analyses (e.g. bazel query, bazel-diff) — puts those analyses on the critical path. For a 900K-node build graph like Canva's, that analysis alone was taking >10 min per commit:

bazel-diff used to be generated and evaluated against Bazel queries (defined per step) in the pipeline generator. It took more than 10 minutes to generate the pipeline, which isn't good because it's on the critical path.

The fix is architectural: pre-compute the expensive shape offline, publish it, and let runtime consumers look it up.

Mechanism

Canva's pipeline-v3 architecture (Jan 2024):

  1. Static pipeline YAML. The pipeline structure itself is pre-generated (checked in or produced by an idempotent step before the commit-level flow). No git checkout on the critical path for pipeline generation.
  2. Conditional evaluation moves to job runtime. The per-commit "which targets are affected?" question is answered inside each job — so it runs in parallel with other steps rather than blocking them.
  3. bazel-diff runs out-of-band. Dedicated instances compute per-target input-hash manifests as soon as a commit is pushed.
  4. Manifest published to S3. Jobs download the manifest (seconds) and diff against the prior commit's manifest to decide which targets to run.
  5. Graceful fallback. If the download fails or the manifest is stale, let Bazel do its native incremental-build computation — slower, but correct.

Canva's evolution

Era Generator path Wall-clock
v1 TypeScript generator, bazel query + bazel-diff per step on critical path >10 min
v2 Drop bazel-diff from generation; let Bazel's native caching do the work 2–3 min
v3 (Jan 2024) Static pipeline + out-of-band hash manifest + job-runtime lookup near-zero

Between v1 and v3: pipeline generation went from being Canva's biggest per-commit tax to being a rounding error.

A parallel simplification: Canva rewrote the generator itself from TypeScript + complex conditionals to Starlark (Bazel's configuration language), which converts to YAML. The declarative Starlark file collapsed thousands of lines of TypeScript conditionals to a couple hundred — because "hermetic by design, no side effects" drops most of the fragility surface.

Preconditions

  • Build system with content-addressed cache so that "per-target input hash" is a meaningful atom (concepts/content-addressed-caching).
  • Deterministic target set. A static pipeline can't enumerate targets that only exist conditionally at per-commit granularity.
  • Fallback path. The manifest-based shortcut must degrade to a correct-but-slow baseline.

Trade-offs

  • Manifest freshness window. There's a window between commit push and manifest publication where jobs might hit the fallback. Canva considers this acceptable because the fallback is just "Bazel figures it out" — correct, just slightly less efficient than the manifest shortcut.
  • Out-of-band hash computation has its own cost. The dedicated instances aren't free, but they're not on the critical path and can be warm/pooled.

Composes with

Seen in

Last updated · 200 distilled / 1,178 read