PATTERN Cited by 1 source

Versioned concurrent deployment

Intent

Deploy model versions with breaking I/O interface changes without dropping requests, by maintaining independent per-version deployments that serve simultaneously.

Mechanism

1. Maintain an independent deployment for every (modelId, modelVersion) pair.
2. Multiple versions serve simultaneously — the new version comes up without affecting the old.
3. Consumer waits for the new version to be fully ready before switching its config.
4. Old version keeps serving legacy traffic during the transition.
5. Platform cleans up older deployments after inactivity but always preserves the latest.

Trade-offs

• Pro: Decouples model deployment from consumer updates — no coordination window where requests fail.
• Con: Temporary increase in GPU cost during the overlap period (both versions run simultaneously).

When to use

Reserved for cases where a breaking interface change (e.g., new tensor dimensions, different tokenizer, incompatible I/O schema) is unavoidable. Prefer patterns/red-black-gpu-deployment for non-breaking changes.

Seen in

• sources/2026-07-17-netflix-in-house-llm-serving — described as the fallback strategy for breaking changes in Netflix's MSS