CONCEPT Cited by 1 source
Persistent connection amplification¶
Definition¶
Persistent connection amplification is the phenomenon where protocols that maintain long-lived connections (gRPC, WebSocket, HTTP/2 streams) amplify infrastructure-level load distribution problems that would be invisible with short-lived stateless HTTP.
The mechanism: with stateless HTTP, connections churn constantly — even if DNS resolves to a single origin, the next request might get a different resolution. With persistent protocols, connections are established once and held open for the session lifetime. All connections established during a DNS TTL window accumulate on whichever origin was resolved at that moment, and they stay there. Mild unevenness becomes catastrophic concentration.
Why this matters at scale¶
The bitdrift case (Source: sources/2026-07-15-aws-bitdrift-scaled-121-million-grpc-connections-cloudfront) is the canonical wiki production instance: 121 million mobile devices establishing persistent gRPC connections through CloudFront to origin NLBs. With Weighted routing returning 1 IP per DNS response, all connections from a TTL window landed on a single NLB and stayed there. With stateless HTTP, those same connections would have naturally redistributed on the next request cycle. The persistent protocol turned a minor DNS-level inefficiency into an 80% error rate.
The asymmetry: - Stateless HTTP: DNS routing imbalance → mild latency variance, self-healing on next request cycle. - Persistent gRPC/WebSocket: DNS routing imbalance → permanent connection concentration → capacity exhaustion → cascading failure.
Structural similarity to gRPC L4 load balancing problem¶
This is a cousin of the well-known gRPC L4 load-balancing problem: L4 load balancers pick a backend once per TCP connection, so gRPC's long-lived connections create traffic skew. The DNS routing variant is the same shape at a higher altitude — DNS picks an origin once per TTL, and persistent connections accumulate there.
Both problems have the same structural fix: move the distribution decision closer to the per-request (or per-connection-establishment) granularity — L7 per-request routing for intra-cluster, Multi-Value Answer (multiple IPs) for DNS-level.
Seen in¶
- sources/2026-07-15-aws-bitdrift-scaled-121-million-grpc-connections-cloudfront — 121M gRPC connections; Weighted routing returned 1 IP → all connections concentrated on single NLB per TTL window → 80% error rate. Fix: Multi-Value Answer routing (8 IPs) → zero errors.