PATTERN Cited by 1 source

Request scattering

Intent

Prevent synchronized bursts (self-DDoS) in large-scale distributed systems by introducing controlled temporal dispersion into scheduled or periodic operations.

Problem

When many components (Lambda functions, cron jobs, polling loops) share an identical schedule expression (e.g., rate(5 minutes) aligned to the clock), they all fire simultaneously. At small scale this is invisible; at large scale (thousands+ of instances), the burst overwhelms downstream APIs, resembling a DDoS attack.

Solution

A standardized internal library enforces three dispersion mechanisms:

1. Jitter — random delay added to each invocation's start time, spreading the burst across the full scheduling interval.
2. Randomized batch offsets — groups of related functions start at different sub-intervals within the period.
3. Staggered updates — deployments and configuration refreshes roll out in waves rather than fleet-wide.

The rule: "Never do the same thing at the same time everywhere."

Implementation approaches

• Hash-based offset: delay = hash(tenant_id) % interval — deterministic, evenly distributed, no coordination needed.
• Random jitter: delay = random(0, interval) — simple but less predictable in aggregate distribution.
• Sliding window: divide the interval into N slots, assign each function to a slot based on a consistent hash.

When to apply

• Any periodic/scheduled workload at scale (>100 instances with the same schedule).
• Polling loops hitting a shared API.
• Health checks, heartbeats, and keep-alive pings in large fleets.
• Cache TTL expiry (use jittered TTLs to prevent simultaneous invalidation).

Seen in

• sources/2026-06-29-aws-lessons-learned-from-scaling-to-1-million-lambda-functions — ProGlove built a standardized library enforcing jitter + staggered offsets across 1M+ Lambda functions after experiencing self-DDoS from clock-aligned schedules.