CONCEPT Cited by 1 source
Quasi-experimental methods¶
Definition¶
Quasi-experimental methods are causal-inference techniques for estimating treatment effects when randomization at the level of interest is infeasible. The quasi- prefix indicates the study resembles an experiment (treatment vs control with a causal claim) without the randomised assignment that makes a true experiment work.
Typical quasi-experimental designs include: - Difference-in-differences — compare before/after change in the treated population to before/after change in a comparable control population. - Synthetic control — construct a weighted combination of untreated units that best tracks the treated unit's pre-period trajectory; post-period divergence estimates the effect. - Regression discontinuity — exploit a cutoff in an eligibility rule to compare just-above and just-below observations. - Interrupted time series — estimate the counterfactual by extrapolating the pre-intervention trend. - Propensity score matching / weighting — balance treated and control on observable confounders.
Why experimentation platforms offer them¶
Some product decisions have no unit at which randomization works: - Country-level comparisons — users in country A cannot plausibly be randomised relative to users in country B; the "variants" differ in many confounded ways (language, payment mixes, supply). - Market-wide treatments — ridesharing, delivery, and marketplace settings where treating some users interferes with outcomes for others (SUTVA violation — see concepts/user-split-experiment). - Regulatory or one-off launches — the treatment is being applied to the whole population on a date; there is no "control group" to randomise.
Zalando cites comparing performance between two countries as the canonical example (Source: sources/2021-01-11-zalando-experimentation-platform-at-zalando-part-1-evolution). Octopus (see systems/octopus-zalando-experimentation-platform) does not force A/B testing for every use case; instead it ships guidelines + software packages to help analysts pick the right causal inference tool.
Implications for platform design¶
A mature experimentation platform is not an A/B-only tool. Its job is to help owners pick the right inference method for the decision they're making, which means:
- Support for quasi-experimental designs as first-class primitives alongside user-split A/B.
- Guidelines on when each is valid — randomization-feasible? SUTVA-violating? Long-horizon? — that read as a decision tree, not a bibliography.
- Software packages (DiD, synthetic control, regression discontinuity implementations) so analysts use the same well-audited code across the org.
Related designs¶
Wiki siblings from Lyft's 2026 post on marketplace experiment design: - concepts/region-split-experiment — a geographic quasi-experiment formalised as a study design - concepts/switch-back-experiment — time-alternating randomisation (a partial middle ground) - concepts/surrogacy-causal-inference — short-term mediator → long-term outcome reasoning, often used in combination with quasi-experimental methods
Seen in¶
- sources/2021-01-11-zalando-experimentation-platform-at-zalando-part-1-evolution — Zalando Octopus ships quasi-experimental tooling + guidance for use cases that can't be cleanly A/B-tested (country comparisons)