Skip to content

CONCEPT Cited by 1 source

Red-black deployment (GPU serving)

Definition

Red-black deployment is a zero-downtime deployment strategy where a new version is deployed alongside the current one; once the new instance passes health checks, traffic shifts in phases — the new version scales up while the old scales down at the same rate. If any step fails, the system triggers an atomic rollback.

This is functionally equivalent to blue-green deployment, with Netflix using "red-black" terminology in their GPU serving stack.

GPU-specific considerations

GPU deployments take longer to bring up than CPU services (large model downloads, GPU initialization, KV cache warmup), making the coordination window between old and new versions longer and more failure-prone.

Coordination gap

Netflix documents a specific failure mode: when a new model version changes the I/O schema (e.g., new tensor dimensions), the upstream consumer cannot update its config until the new model is fully live. During the migration window, "old" requests hit the "new" deployment and fail. This gap motivates the versioned deployment alternative.

Recommendation

Embed variable configurations (e.g., tensor shapes) directly into the inference model to make it version-agnostic, allowing the cheaper red-black path. Reserve versioned deployment for unavoidable breaking interface changes.

Seen in

Last updated · 585 distilled / 1,765 read