Skip to content

PATTERN Cited by 1 source

Red-black GPU deployment

Intent

Deploy a new model version to GPU infrastructure with zero downtime and atomic rollback capability, without requiring consumers to coordinate their config changes with the deployment.

Mechanism

  1. Deploy new model version alongside the current one on fresh GPU instances.
  2. New instances pass health checks (model loaded, warm, GPU ready).
  3. Traffic shifts in phases — new version scales up, old scales down at the same rate.
  4. If any phase fails, trigger atomic rollback (traffic returns to old version).

Applicability

Use when the model's I/O interface is stable — same tensor shapes, same API contract. The consumer doesn't need to change anything when the model upgrades.

Limitation

A coordination gap exists when the new version changes the I/O schema. The consumer can't update its config until the new model is fully live, so during the migration window the consumer sends "old-format" requests to the "new" deployment, and those fail. For breaking changes, use patterns/versioned-concurrent-deployment instead.

Recommendation from Netflix

Embed variable configurations (e.g., tensor shapes) directly into the inference model to make it version-agnostic, keeping the deployment on the cheaper red-black path.

Seen in

Last updated · 585 distilled / 1,765 read