Skip to content

CONCEPT Cited by 1 source

FIRST test principles

Definition

FIRST is Robert C. Martin's acronym (from Clean Code) for the five properties every test — unit, component, integration — should satisfy:

  • F — Fast. A test should not take more than a second to finish. Slow tests get skipped, disabled, or un-run.
  • I — Isolated. No order-of-run dependency. Test A passing must not depend on test B having run first (or not having run).
  • R — Repeatable. A test's result must not depend on data in the environment: wall-clock time, locale, existing DB rows, leftover files, random ports.
  • S — Self-Validating. Pass / fail decided by the test itself, no human inspection of logs or diffs required.
  • T — Thorough. Cover every use-case scenario, including edge cases and error paths — not merely coverage-percentage targets.

Why FIRST matters for Testcontainers-style ITs

Integration tests are where FIRST is hardest to hold. Running a real Postgres or Localstack container breaks Fast by default (~4 s Postgres, up to ~20 s Localstack on Zalando's author machine — sources/2021-02-24-zalando-integration-tests-with-testcontainers). The response is the concepts/singleton-container-pattern: start the container once per JVM, amortise the cost across every test that uses it.

But the singleton choice then threatens Isolated and Repeatable: every test now runs against the same DB instance, so state leaks across tests. Zalando names two strategies:

  1. Unique IDs / names per test — generate random primary keys, usernames, S3 keys so no two tests collide. Trade-off: aggregate queries like SELECT COUNT(*) FROM users now see rows from other tests. Often acceptable if tests don't assert on aggregates.
  2. Explicit cleanup after each test — @AfterEach drops the rows / files / topics the test created. More developer effort; a forgotten cleanup produces order-dependent flakes.

Concurrent execution (parallelising tests within a class or across classes) makes Isolated + Repeatable even harder and forces stronger discipline than sequential.

Common FIRST violations

  • Slow — embedded-browser E2E, full-app boot per test, Docker pull on first run.
  • Isolated — test depends on another test having seeded data; tests share a mutable static field without @BeforeEach reset.
  • Repeatable — test asserts on "today's date", on the top 100 rows (may have shifted), on a port number (busy on CI), on a hash of an unstable iteration order.
  • Self-Validating — System.out.println + "look at the output"; assertions commented out "for debugging".
  • Thorough — only happy path tested; 4xx / 5xx / timeout / malformed input paths left uncovered. Coverage % can be high while FIRST-Thorough is low.

Seen in

Last updated · 476 distilled / 1,218 read