Skip to content

SYSTEM Cited by 1 source

TestContainers

TestContainers is an OSS library (Java first, now many languages) that gives a test the ability to declaratively spin up its own Docker containers as storage / service backends (Postgres, Kafka, LocalStack, Redis, Elasticsearch, a custom image, …), with lifecycle tied to the test. Upstream: testcontainers.com.

Why it matters for CI architecture

The canonical pre-TestContainers pattern for integration tests is a shared harness: one set of storage containers (e.g. localstack) shared by all tests in the suite. Consequences:

  • Not hermetic. Every test sees every other test's state.
  • Not parallelisable. Tests that need exclusive access serialise through the shared backend.
  • Flaky. Resource contention, test-to-test interference, unclean state leak between runs.
  • Not cacheable. The "inputs" of a test include the entire shared backend's accumulated state.

TestContainers inverts the model: each test declares the containers it needs, brings them up in a sandbox, uses them, tears them down. That's enough to re-characterise the test as hermetic, because every container is a declared input and nothing leaks across tests.

Canva's use

From the Canva retrospective:

The Developer Runtime team developed a framework for hermetic container orchestration using the TestContainers library, allowing each test to control its distinct set of storage requirements within the confines of a Bazel sandbox, ultimately allowing us to cache these tests.

Three levels of Canva's TestContainers-based hermeticity:

  1. Backend integration tests. Each test gets its own storage containers inside a Bazel sandbox. Tests become cacheable (concepts/content-addressed-caching) and parallelism-safe.
  2. Service-container tests. Each backend service has its own TestContainer image and a launch-validation test. Shifts deployment failures left to CI.
  3. Hermetic E2E tests. E2E environments compose service-container definitions. Rebuild only triggers when a service-in-the-test changes — cache hits on unaffected services.

Mechanism (in a Bazel test)

  • The test's Bazel rule declares the TestContainers library and any container images (pinned by digest).
  • Bazel sandboxes the test; TestContainers gets a scratch Docker daemon context.
  • Test body brings up containers, exercises the system, tears them down.
  • Result: same inputs → same outcome → cacheable on the input-hash key Bazel assigns.

Preconditions

  • Docker (or compatible runtime) on test workers. Usually requires worker image changes.
  • Pinned container images. Non-pinned images break hermeticity silently: redis:latest is not a fixed input.
  • Per-test resource budget. Spinning up containers per test costs memory + startup time. Canva pairs this with Bazel sandboxing and right-sized worker pools (patterns/instance-shape-right-sizing).

Zalando ZMS's use (Java / JUnit 5 / Spring Boot altitude)

A complementary, application-developer altitude picture of Testcontainers comes from Zalando Marketing Services's 2021 backend-testing post (). Where Canva's account is CI-framework altitude (how Bazel wraps Testcontainers for fleet-scale hermeticity), Zalando's is how a Java team actually wires it into a running Spring Boot project:

  1. Singleton container on a base class. A public static PostgreSQLContainer field on AbstractIntegrationTest, started in a static initialiser, inherited by every concrete *IntegrationTest subclass. One container per JVM, not one per class. See concepts/singleton-container-pattern and patterns/shared-static-container-across-tests.
  2. Spring wiring via @DynamicPropertySource (Spring 5.2.5+). Method references on the container (postgreSQL::getJdbcUrl) supply the dynamic JDBC URL / random port / generated password to the Spring test context — resolved at context-creation time, after .start() has run.
  3. Maven phase split: unit tests via Surefire in the test phase, ITs via Failsafe in integration-test behind a with-integration-tests profile. See patterns/failsafe-integration-test-separation.
  4. Concrete startup costs quoted (author's local machine): ~4 s Postgres, ~0.4 s H2, ~20 s Localstack. These numbers motivate the singleton amortisation pattern.
  5. @Testcontainers / @Container annotations (per-class) are an alternative but cannot be reused between test classes and are documented as tested only with sequential execution — the reason Zalando prefers the static-field idiom.
  6. Cleanup via JVM shutdown hooks + Ryuk. Containers are never explicitly .stop()ed; Testcontainers registers JVM shutdown hooks, and the Ryuk companion container reaps orphans when the JVM dies before hooks can fire.
  7. Not sufficient — pair with contract testing. Zalando explicitly flags that a MockServer/WireMock-backed IT doesn't catch real-API drift; see concepts/contract-testing.

Canonical images called out

  • Postgres via PostgreSQLContainer — the primary database-parity motivator.
  • systems/localstack — AWS (S3, Kinesis, DynamoDB, SQS, …) emulation; startup ~20 s on author machine.
  • systems/mockserver / systems/wiremock — HTTP peers for exercising corner cases (5xx, timeouts, malformed bodies).
  • Ryuk — orphan reaper shipped with Testcontainers itself.

Seen in

  • sources/2024-12-16-canva-faster-ci-builds — replaced shared-localstack harness with per-test TestContainers sandboxes; extended to service-container tests and hermetic E2E environments. CI-framework altitude.
  • — Zalando Marketing Services's Java/JUnit 5/Spring Boot idiom for integration tests: singleton PostgreSQLContainer on an AbstractIntegrationTest base class + @DynamicPropertySource
  • Maven Surefire/Failsafe phase split + Localstack/MockServer companions. Application-developer altitude.
Last updated · 542 distilled / 1,571 read