Skip to content

CONCEPT Cited by 1 source

Undocumented-behavior drift on version upgrade

Definition

Undocumented-behavior drift is a class of version-upgrade regression where callers rely on behavior the old implementation provided but the official spec never promised. The new version fixes the bug (i.e. aligns the implementation to the spec), and every caller whose code depended on the laxness breaks.

The regression is not a breaking change by the spec's definition — the spec has been consistent — but it is a breaking change in practice, because a measurable fraction of production queries silently relied on the implementation's laxness.

Examples

  • Elasticsearch nested- sort path requirement — ES 7.x silently accepted sort clauses on nested fields without nested.path; ES 8.x rejects with query_shard_exception. (Canonical instance, Zalando 2023-11 ingest.)
  • MySQL reserved- words breakage on upgrade — new reserved words introduced in a MySQL minor version mean queries and DDL that happened to use those identifiers now fail parse-time.
  • ES 7→8 _type field removal — _type was deprecated since 7 but still returned; ES 8 fully removed it, so tests asserting its presence broke.
  • ES 7→8 date/epoch_second range-query numeric-bounds change — Elastic declared it a feature, not a bug, but it broke queries that passed bounds as numbers instead of strings.

Why it happens

  • Specs and implementations drift. Implementations tend to be more permissive than specs because adding a check is a change but leaving lax behavior is not.
  • Callers do not test against the spec — they test against production behavior. Production behavior is the implementation, not the spec.
  • Major version boundaries are where the maintainers feel licenced to align implementation to spec, so the drift accumulates in minor versions and surfaces at the major boundary.

Mitigation

  • Parity testing with both versions before cutover — see patterns/multi-version-testcontainers-parity-suite. The Zalando three-class test matrix is explicitly about catching this class of regression before the live cluster upgrade.
  • Read the per-version breaking-changes doc carefully, and search it for every API surface your code touches.
  • Beta-test the new version against a replay of production queries, not just synthetic test payloads.
  • File bugs on the upstream forum early; undocumented-behavior drift is usually well-known to the maintainers and already patched in docs or issue tracker — a forum post can turn a panic into a clear migration step.

Seen in

  • sources/2023-11-19-zalando-migrating-from-elasticsearch-7-to-8 — Zalando encountered four instances of this class during their ES 7→8 migration:
  • _type field removed from search response.
  • is_write_index no longer accepts null.
  • Range-query numeric bounds on date/epoch_second fields semantics change (Elastic classed it a feature).
  • action.destructive_requires_name default flipped false → true. Plus the nested-sort path requirement, explicitly a bug fix. All surfaced by the three-class Testcontainers parity suite.
Last updated · 501 distilled / 1,218 read