Skip to content

ZALANDO 2021-02-01

Read original ↗

Zalando — Stop using constants. Feed randomized input to test cases.

Summary

Vijaya Kandel (Zalando Mobile, iOS) distils a single testing discipline picked up at Jorge Ortiz's 2017 Swift Aveiro Clean Architecture workshop and applied inside Zalando's iOS codebase: never hand-type constants into test cases. Constants make tests pass trivially on implementations that hard-code the same value (the canonical return "Zalando" tampered-code example) — so the signature func set(value: String, for key: String) gets tested against one String instance when the contract says any. The fix is to drive every test input from Type.random, backed by an open-source Swift library (Randomizer) Kandel authored, which covers Standard Library types and lets user-defined structs conform to a Random protocol (manually or — aspirationally — via codegen). This makes every test run a different permutation of the input space and breaks hard-coded implementations immediately. Scope claim at the end: the technique is domain-agnostic — "the only requirement is Type.random."

Key takeaways

  1. Constant-input tests are near-tautological against hard-coded implementations. Kandel's worked example: a storage abstraction where get(for:) is tampered to return "Zalando" passes the test that set "Zalando" and read it back. The fix is to replace the hand-typed "Zalando" / "companyName" with String.random — then the hard-coded implementation fails because the random value differs from "Zalando" across runs (Source: sources/2021-02-01-zalando-stop-using-constants-feed-randomized-input-to-test-cases).

  2. Type.random is the single ergonomic primitive. Swift's type inference means .random at a use site resolves to the exact type's Random conformance. One uniform call shape (let x = Type.random) replaces ad-hoc fixture construction everywhere. Canonical in Randomizer Swift library (Source: sources/2021-02-01-zalando-stop-using-constants-feed-randomized-input-to-test-cases).

  3. User-defined structs conform by delegating to per-field .random. The LabelProps example extends the struct in the test target to conform to Random, returning LabelProps(text: .random, backgroundColor: .random, font: .random). This composes recursively — a struct's random instance is just its fields' random instances. Build-time codegen (analogous to how Swift synthesises Equatable) is flagged as future work, not shipped (Source: sources/2021-02-01-zalando-stop-using-constants-feed-randomized-input-to-test-cases).

  4. Store the generated value when the test needs to compare against it. Canonical idiom: let value = String.random; …; XCTAssertEqual(value, obtained). The randomness is captured into a local so the assertion has something to compare against. Applied to larger types (APIAccessibility.random stored so accessibilityModel.label / .hint can be asserted on the rendered view) (Source: sources/2021-02-01-zalando-stop-using-constants-feed-randomized-input-to-test-cases).

  5. Constrained types need constrained generators. String.random is insufficient when the function under test expects an Email, URL, Deeplink, or PhoneNumber represented as String. Two escape hatches: (a) extend String with String.randomEmail / randomURL for common cases; (b) create a concrete type (e.g. a Email wrapper) that conforms to Random with a generator that produces only valid instances. The second is structurally better — the type system enforces validity at the call site, not just in the generator (Source: sources/2021-02-01-zalando-stop-using-constants-feed-randomized-input-to-test-cases).

  6. Test names change meaning. The same test name (test_setValueCanBeRetrieved) goes from "this one specific case works" to "this universal property holds for any (String, String) pair" when the inputs become random. The source doesn't explicitly name this shift but the worked before/after pair demonstrates it — the test body is almost identical; only the inputs flip from literals to .random.

  7. The technique is domain-agnostic. Kandel closes: "The technique of permutation testing by using random input applies to all software testing; not just iOS development. The only requirement is Type.random." In wiki terms this is the property-based testing pattern applied at its simplest altitude — no invariants beyond input-output round-trip, no shrinkers, no seed replay, no generator combinators. A gateway drug to the fuller pattern (Source: sources/2021-02-01-zalando-stop-using-constants-feed-randomized-input-to-test-cases).

Systems / concepts / patterns extracted

Systems (new)

Concepts (new)

Patterns (extended)

  • patterns/property-based-testing — Kandel's post is a minimal instance: random input across many permutations, no explicit invariant beyond the set-then-get round-trip. Extends the wiki-canonical pattern with a Swift iOS implementer altitude Seen-in alongside the Dropbox CanopyCheck (Rust / sync-planner), AWS ShardStore (Rust / SOSP spec), and MongoDB TLA+-generated-test (conformance-checking) Seen-ins.

Operational numbers and scope signals

  • No hard operational numbers. The post is pedagogical, not a production retrospective. No disclosed fleet size, no regression-detection rate, no CI-time impact measurement. The only quantitative claim is implicit: "many permutations" across runs.
  • No run-count / budget disclosure. The post does not specify how many randomised invocations per test (per PBT norms, ≥10⁴ is common; the post's examples read as one invocation per CI run).
  • No shrinker mention. Randomizer doesn't ship a shrinker, so failing inputs are whatever the RNG produced — not the minimal reproducer that mature PBT libraries surface. Kandel doesn't flag this as a limitation.
  • No seed-replay mention. Related gap — failures can't be reliably reproduced without recording the seed. The library may support this; the post doesn't say.
  • Adoption claim is company-internal only. "At Zalando, we use this Randomizer library." No external-adoption numbers, no GitHub-star disclosure, no porting to other languages.

Caveats

  • Tier-2 borderline include. The post is about testing methodology rather than distributed systems, storage, streaming, or production-infra architecture — the wiki's core scope. But it is load-bearing on two fronts that justify inclusion: (a) it canonicalises a Zalando open-source library (Randomizer) that had not been documented on the wiki; (b) it extends the existing patterns/property-based-testing page with a Swift iOS implementer altitude Seen-in that no prior source on the wiki covers (the three pre-existing Seen-ins are Rust / Rust / TLA+). The testing-discipline content has sibling Seen-ins on the wiki (concepts/test-data-generation-for-edge-cases, concepts/automated-vs-manual-testing-complementarity) so the precedent is established.
  • 2017-era workshop provenance, 2021 publication. The underlying insight is attributed to Jorge Ortiz's Swift Averio 2017 workshop (Clean Architecture); Kandel is reporting a technique already years old by the time of publication. The novelty is the Swift library and the Zalando-internal adoption story.
  • Limited depth relative to mainstream PBT tooling. The post does not cover shrinking, seed recording, generator combinators, stateful testing, fault injection, or any of the higher-altitude PBT machinery that libraries like Hypothesis (Python), proptest (Rust), or fast-check (JS) provide. It is an entry-level pitch for the pattern rather than a full tour. Readers following up should land on patterns/property-based-testing for the fuller shape.
  • Mobile-UI-testing example is unusual for PBT. Most PBT canonicalisations target pure functions or narrow planner logic (Dropbox CanopyCheck scoped to planner only for exactly this reason). The LabelComponent / MockNode example randomises view-model input but asserts on a single deterministic view-model→view projection — a reasonable altitude but narrower than PBT's typical "thousands of runs per property" shape.
  • No engagement with shrinking or seed replay. These are core to making PBT failures diagnosable at scale; Randomizer as described does neither.

Source

Related

Last updated · 476 distilled / 1,218 read