Skip to content

SYSTEM Cited by 1 source

TestContainers

TestContainers is an OSS library (Java first, now many languages) that gives a test the ability to declaratively spin up its own Docker containers as storage / service backends (Postgres, Kafka, LocalStack, Redis, Elasticsearch, a custom image, …), with lifecycle tied to the test. Upstream: testcontainers.com.

Why it matters for CI architecture

The canonical pre-TestContainers pattern for integration tests is a shared harness: one set of storage containers (e.g. localstack) shared by all tests in the suite. Consequences:

  • Not hermetic. Every test sees every other test's state.
  • Not parallelisable. Tests that need exclusive access serialise through the shared backend.
  • Flaky. Resource contention, test-to-test interference, unclean state leak between runs.
  • Not cacheable. The "inputs" of a test include the entire shared backend's accumulated state.

TestContainers inverts the model: each test declares the containers it needs, brings them up in a sandbox, uses them, tears them down. That's enough to re-characterise the test as hermetic, because every container is a declared input and nothing leaks across tests.

Canva's use

From the Canva retrospective:

The Developer Runtime team developed a framework for hermetic container orchestration using the TestContainers library, allowing each test to control its distinct set of storage requirements within the confines of a Bazel sandbox, ultimately allowing us to cache these tests.

Three levels of Canva's TestContainers-based hermeticity:

  1. Backend integration tests. Each test gets its own storage containers inside a Bazel sandbox. Tests become cacheable (concepts/content-addressed-caching) and parallelism-safe.
  2. Service-container tests. Each backend service has its own TestContainer image and a launch-validation test. Shifts deployment failures left to CI.
  3. Hermetic E2E tests. E2E environments compose service-container definitions. Rebuild only triggers when a service-in-the-test changes — cache hits on unaffected services.

Mechanism (in a Bazel test)

  • The test's Bazel rule declares the TestContainers library and any container images (pinned by digest).
  • Bazel sandboxes the test; TestContainers gets a scratch Docker daemon context.
  • Test body brings up containers, exercises the system, tears them down.
  • Result: same inputs → same outcome → cacheable on the input-hash key Bazel assigns.

Preconditions

  • Docker (or compatible runtime) on test workers. Usually requires worker image changes.
  • Pinned container images. Non-pinned images break hermeticity silently: redis:latest is not a fixed input.
  • Per-test resource budget. Spinning up containers per test costs memory + startup time. Canva pairs this with Bazel sandboxing and right-sized worker pools (patterns/instance-shape-right-sizing).

Seen in

Last updated · 200 distilled / 1,178 read