Skip to content

PATTERN Cited by 1 source

Staged progressive deployment with metrics

Pattern

Divide a deployment into multiple sequential stages (each representing a distinct fleet segment like an ASG or K8s cluster), gate progression between stages on service health metrics, and automatically roll back any stage that regresses — preventing issues from propagating to the full fleet.

Mechanism

  1. Define pipeline stages (e.g., canary ASG per AZ → production ASG per AZ).
  2. Deploy to the first stage.
  3. Monitor key service metrics at that stage against health thresholds.
  4. If metrics are healthy through the soak period, advance to the next stage.
  5. If metrics degrade, automatically roll back — only the affected stage and downstream stages are impacted.
  6. Each stage can use a different rollout executor appropriate to its substrate (Instance Refresh, Karpenter, custom).

Distinction from simple canary

While a canary deployment is a single small→large expansion, staged progressive deployment supports multiple intermediate stages with independent metric gates, giving operators more granular control over rollout sequencing and blast radius per-stage.

Slack's implementation

Gondola pipelines support multiple stages, with the Egress Team example running "separate canary and production ASGs in each availability zone, with deployment stages ordered so updates flow sequentially. Gondola updates each stage, monitors key metrics, and rolls back automatically if problems are detected, preventing issues from spreading" (Source: sources/2026-07-14-slack-shipyard-next-generation-ec2-platform).

Seen in

Last updated · 585 distilled / 1,765 read