CONCEPT Cited by 1 source

Per-PR ephemeral environment¶

Definition¶

A per-PR ephemeral environment is an isolated, named environment whose lifecycle is bound to a single pull request: created when the PR is opened, destroyed when it is closed (merged or abandoned). Its purpose is to let the PR's code be validated end-to-end without colliding with other in-flight PRs on a shared test substrate.

Lifetime is typically minutes to days (one open PR). Substrate can be compute, data, or both.

Why it beats the usual two alternatives¶

Without per-PR environments, teams have two options, both unsatisfying:

Shared test environment — causes contention: every PR writes/reads the same _test tables / DAGs / configs, so two concurrent PRs step on each other.
Separate servers per PR — correct but slow + expensive. E.g. MWAA needs ~30 minutes to provision a new Airflow server, plus compute cost per PR.

Per-PR ephemeral environments on shared infrastructure — via a naming/versioning scheme — are the middle path: isolation without the cost of duplicated compute.

Required machinery¶

Creation on PR-open — CI or a PR webhook spins up a uniquely-named env.
Naming scheme — the PR / feature-branch name flows into every resource name in the env (DAG ids, database names, URLs, …) so they don't clash.
1-to-1 compute ↔ data binding — isolating only compute (or only data) leaves cross-PR collisions on the other layer.
Cleanup on PR-close — otherwise the shared substrate fills up with orphaned envs. Usually a cron polling PR state.

Canonical instance on the wiki¶

sources/2022-06-09-zalando-accelerate-testing-in-apache-airflow-through-dag-versioning — Zalando applies this to Airflow + Spark data pipelines. Compute isolation is per-branch zip-packaged DAGs with DAG id rewriting (see concepts/pipeline-environment). Data isolation is per-branch Spark databases, populated via views-over-copies (see concepts/data-environment). Creation <1 min; cleanup via a cron that polls GitHub PR status.
patterns/ephemeral-preview-environments — the related application-stack shape (IaC-defined web/service stack per PR for end-to-end smoke tests; lifetime minutes to hours).

The two shapes share the structural idea — ephemeral, PR-scoped, isolated — but differ in substrate: pipeline/data vs. cloud application.

Seen in¶

sources/2022-06-09-zalando-accelerate-testing-in-apache-airflow-through-dag-versioning — per-PR pipeline + data environment on shared Airflow test server.