CONCEPT Cited by 1 source
Warm pool instances¶
Warm pool instances are compute instances that a serverless substrate keeps alive between jobs instead of deprovisioning them immediately after a job finishes. When a new job lands, it reuses a warm instance from the pool, skipping the provisioning + boot + image pull stages of a cold start.
Warm pools are the explicit knob a serverless substrate exposes when cold-start is too expensive for a latency-sensitive workload but the platform's default behaviour is on-demand provisioning.
The trade-off¶
- Pure on-demand — no idle cost, but cold-start tax on every job.
- Warm pool — pays to keep instances alive for some retention window, then deprovisions on inactivity. Cold-start → zero while the pool is warm.
The knob is pool size and retention TTL. Larger / longer → lower cold-start probability, higher idle cost. Smaller / shorter → higher cold-start probability, lower idle cost. Tune to workload shape.
When warm pools are the right answer¶
Warm pools are the stopgap when cold-start is too expensive and other techniques fall short:
- Lazy image loading (SOCI) helps cold-start by shrinking the first-pull tax, but can't help workloads whose startup cost is dominated by runtime init (JVM warmup, Python import, model load into memory), not image fetch.
- Image-size optimisation has diminishing returns past a point.
- Warm pools skip everything by not tearing down the instance at all.
Lyft / LyftLearn 2.0 instance¶
Lyft's LyftLearn 2.0 migration onto SageMaker observed that SOCI (for SageMaker training / batch jobs) "wasn't available" at migration time, so for the most latency-sensitive workloads — models that retrain every 15 minutes — they adopted SageMaker warm pools to keep instances alive between retraining runs. Outcome: Kubernetes-like startup latency on a fully serverless substrate, without paying for pre-warming a whole idle cluster as they had in LyftLearn 1.0 (Source: sources/2025-11-18-lyft-lyftlearn-evolution-rethinking-ml-platform-architecture).
Related pattern¶
patterns/warm-pool-zero-create-path — the generalised shape
(pre-create-and-queue instances; serve create by dequeueing),
originally documented from Fly.io Sprites. Lyft's SageMaker warm
pool is the ML-platform instance of the same shape.
Seen in¶
- sources/2025-11-18-lyft-lyftlearn-evolution-rethinking-ml-platform-architecture — Lyft used SageMaker warm pools to cover the cold-start gap for 15-minute-retrain models.