Skip to content

CONCEPT Cited by 3 sources

Backward compatibility

Backward compatibility is the property that interfaces accept and behave correctly on inputs + requests that were valid under prior versions of the interface. For read-path APIs with URL-shaped / bookmarkable / link-shareable inputs — search boxes, query endpoints, IDs, share links — backward compatibility is usually the primary constraint on any rewrite, not a secondary feature; breaking a shared URL is a visible user-facing incident without a rollback in many deployments.

This page is scoped to the rewrite side of backward compat: how a team replaces a subsystem that serves long-lived inputs without breaking them. The other face (introducing a new API with forward-compatible headroom) is a separate concept.

The three mechanisms, ranked by strength

1. Grammar-level inclusion (strongest)

One parser / one grammar accepts both old and new syntax as distinct rules of the same language. Old syntax remains a subset of valid inputs by construction, not by a fallback path. Tests check that every historical input still parses to a semantically equivalent intermediate representation.

Example: GitHub Issues' 2025 rewrite extended its PEG grammar so that the flat legacy query form (assignee:@me label:support new-project) and the new nested form (author:A AND (type:bug OR type:epic)) are both valid entry rules that parse to the same AST family. No separate legacy parser exists — the new parser is the only parser. (Source: sources/2025-05-13-github-github-issues-search-now-supports-nested-queries-and-boolean)

2. Test-suite inclusion (medium)

The existing unit + integration tests from the old subsystem are re-run unchanged against the new subsystem. The old tests become the behavioural contract; any failure under the new path is a regression to fix before rollout.

Stronger variant: run the tests with the new-path feature flag both on and off to verify that neither path has regressed. This is GitHub's second validation technique for the Issues-search rewrite:

"we ran the tests for the search endpoint both with the feature flag for the new search system enabled and disabled."

3. Dark-ship behaviour diff (empirical)

For a sample of live production traffic, run both the old and new subsystems, compare results, and log differences for triage. Catches regressions the test suite doesn't cover because real users construct inputs the test authors didn't anticipate.

GitHub's third validation technique on Issues search: 1% of real queries run against both paths; count-of-results differences (within ≤1 s) are logged. See patterns/dark-ship-for-behavior-parity.

Why layered matters

Each mechanism catches a different failure class:

Mechanism Catches
Grammar inclusion Parse-stage regressions; syntax rejections
Test-suite re-run Known semantic contract violations; explicit edge cases authored in tests
Dark-ship Real production queries the test suite missed; long-tail inputs

A single layer is never enough. Grammar inclusion can pass tests while the query-generation stage silently produces different backend queries. Test-suite re-run passes what's authored; real user inputs are a different population. Dark-ship only observes inputs that happen during the dark-ship window; historical inputs that are rare but important may not appear.

Bookmarkability as a failure mode

"Users have bookmarked this URL" is a pattern-setting observation: it means inputs outlive their authors' intent and their producers' code. This shows up in:

  • Search URLs with filter parameters
  • Stable entity IDs in permalinks
  • RSS feed URLs with query-string filters
  • API keys / tokens with embedded scoping
  • Third-party integrations that hardcode a URL pattern
  • Chatroom / doc links that reference an issue-search filter

When any of these are in scope, the "smallest change that breaks nothing" becomes the driving requirement, not a polite aspiration.

Contract-invariance checks under feature flags

A subtle failure mode specific to flag-gated rewrites: the contract is preserved with the flag off (fine, old code path) and with the flag on (also fine, intended new behaviour), but toggling the flag mid-request / mid-session breaks something. Running the test suite with the flag both on and off checks the two endpoints of the transition; it does not check the transition itself. For stateful / cache-laden paths, a separate flip-in-flight test is sometimes required. (The GitHub Issues rewrite is read-only, so this concern doesn't apply; it would for a write-path rewrite.)

Caveats

  • Backward compatibility is not free. The grammar that accepts legacy and new syntax is larger than a clean-slate new grammar; the test suite that re-runs old contracts accumulates over years. The maintenance cost is real and should be budgeted.
  • Some rewrites are genuinely incompatible. When the underlying data model changes or a security bug retracts a past behaviour, strict backward compat may be impossible or undesirable. In that case: staged rollout + explicit deprecation notice + migration period, not silent drift.
  • "Same result count" is not the same as "same results." Dark-ship with only count-equality (GitHub's first-iteration choice) catches large regressions but can miss result-set reordering or silent swap-outs within a consistent count. GitHub acknowledges this as a first-iteration simplification.

Seen in

  • sources/2025-05-13-github-github-issues-search-now-supports-nested-queries-and-boolean — canonical instance. GitHub Issues search rewrite layers all three mechanisms (grammar inclusion via parslet's PEG, test-suite re-run with flag on/off, dark-ship diff on 1% of live queries) to protect bookmarked / shared Issues search URLs at ~2 kQPS.
  • sources/2025-11-04-datadog-replication-redefined-multi-tenant-cdc-platform — backward compatibility as the compatibility mode of a Kafka Schema Registry in a CDC pipeline: "new schemas must still allow older consumers to read data without errors." In practice this limits schema changes to adding optional fields or removing existing ones. The registry enforces the rule at publish time; a companion offline migration-SQL validator blocks pipeline-breaking DDL (e.g. SET NOT NULL) before it's applied. This is backward compatibility as a data-contract property on serialised records, distinct from the bookmarkable-URL variant the GitHub Issues rewrite illustrates — same principle, different substrate.
  • sources/2026-01-19-cloudflare-what-came-first-the-cname-or-the-a-record — backward compatibility as "the spec allows it, the install- base doesn't tolerate it" on the DNS-wire-format surface. RFC 1034's "order of RRs is not significant" reading technically permits CNAME records after A records in an answer section, but a meaningful population of deployed stub resolvers (glibc getaddrinfo, Cisco Catalyst DNSC) depends on CNAME-first. Cloudflare's 2025-12 memory optimisation exercised the spec-permitted-but-not-tolerated degree of freedom and detonated on the long tail of unreplaceable clients. Cloudflare's stated position: "we believe it's best to require CNAME records to appear in-order before any other records" — the install- base's constraint, not the spec's, is the shipping bar. This is the clearest public instance on the wiki of the gap between RFC-permits and ship-permissible. Canonical mitigation: test the ambiguous invariant.
Last updated · 200 distilled / 1,178 read