CONCEPT Cited by 1 source
DNS TTL concentration¶
Definition¶
DNS TTL concentration is the mechanism by which DNS TTL (Time To Live) windows synchronize all resolving clients to the same answer, causing temporal concentration of traffic onto a single origin endpoint.
When a DNS routing policy returns a single IP per response (e.g., Route 53 Weighted routing), all clients that resolve during the same TTL window will get the same IP. For a CDN like CloudFront with many edge nodes, this means the entire edge fleet converges on one origin for 60 seconds (typical TTL), then potentially a different one for the next 60 seconds — but never distributing across multiple origins simultaneously.
Mechanism¶
- Weighted routing returns 1 IP per DNS query.
- CloudFront edge node A resolves origin → gets
10.0.1.100(NLB-1). - CloudFront edge node B resolves origin within same TTL window → also gets
10.0.1.100. - All edges route all origin traffic to NLB-1 for the TTL duration (60s).
- After TTL expires, next resolution might return NLB-2 — but the cycle repeats: all edges converge on one.
This is a distinct thundering-herd shape: the synchronizer is the DNS TTL window itself, not a failure event, deploy, or cache wipe. The herd forms because all resolvers are temporally aligned by the shared TTL boundary.
Why adding origins doesn't help under this mechanism¶
This is the counter-intuitive lesson from the bitdrift case: scaling from 2 to 6 NLBs had no effect because the routing policy still returned only 1 IP per response. More origins don't help when the routing policy's per-query cardinality is 1 — all edges still converge on whichever single IP the policy selects for that TTL window. (Source: sources/2026-07-15-aws-bitdrift-scaled-121-million-grpc-connections-cloudfront)
Fix¶
Switch to a routing policy that returns multiple IPs per response (Route 53 Multi-Value Answer: up to 8). This breaks the TTL-concentration mechanism because each resolution immediately distributes across all available origins.
Seen in¶
- sources/2026-07-15-aws-bitdrift-scaled-121-million-grpc-connections-cloudfront — 121M devices, 60s TTL, Weighted routing → all CloudFront edges converge on 1 NLB per TTL window → catastrophic overload.