Skip to content

CONCEPT Cited by 1 source

Stability threshold rollout tiers

Definition

Stability threshold rollout tiers is the practice, for a staged production rollout, of pre-committing to explicit stability thresholds and the response actions at each — before the rollout begins. Each tier pairs a measurable stability floor with a named operational response, so that during the rollout, the team isn't debating what to do while observing the first regression; the decision has already been made.

The canonical wiki instance is Shopify's 2025 rollout of their React Native new-architecture migration (see sources/2025-09-12-shopify-migrating-to-react-natives-new-architecture).

Shopify's three-tier playbook

Stability target: 99.95% crash-free sessions. Three response tiers:

Tier Trigger Response
1 stability > 99.80% fix forward on next weekly release
2 99.00–99.80%, or a broken critical flow with a known fix pause rollout + hotfix
3 stability < 99.00%, or a broken critical flow without a quick fix rollback

Rollback is treated as a last resort — not because of any stability fear, but because of operational cost: losing at-scale production signal sets the migration back by weeks. Pause + hotfix preserves the rollout's forward progress while fixing what broke.

Why pre-commit

Three reasons:

  1. Decision quality under pressure. During a live rollout with stability degrading, ad-hoc debate about whether to pause is noisy. A pre-committed threshold lets you execute without second-guessing.
  2. Organisational alignment. Leadership, product, and on-call all agreed to the same tiers in advance. No one is surprised when the rollback happens.
  3. Guards against sunk-cost thinking. If you're mid-rollout and already invested days of engineering into monitoring it, you're motivated to keep it alive. A pre-committed threshold puts the decision on a number, not an argument.

Thresholds must be app-specific

99.95% / 99.80% / 99.00% are Shopify numbers. For a different app — different baseline, different user tolerance, different severity-of-crash mix — the numbers are different. The concept is the structure: have named thresholds, each paired with a named response, before you ship the rollout.

How this couples to platform-asymmetric rollout

The stability-tier thresholds work in tandem with concepts/platform-asymmetric-rollout-control: Shopify uses Android's fine-grained halt capability to run early-signal exposure against the thresholds, and iOS's slower control surface shapes the schedule that the thresholds protect. Both concepts are load-bearing; neither is sufficient alone.

Last updated · 470 distilled / 1,213 read