Skip to content

CONCEPT Cited by 14 sources

Rate limiting

Definition

Rate limiting caps how many requests (or how much work) a caller can send to a resource in a time window. Requests over the limit are rejected (HTTP 429), delayed, or queued. Common algorithms are the token bucket, leaky bucket, fixed window, and sliding window. A quota is the same idea over a longer window (per day or month), often tied to a plan or a budget.

Rate limiting does three jobs. These jobs are often confused:

  1. Protection. It keeps a backend below its capacity so one caller (or a retry storm) cannot overload it. This is a form of admission control, near concepts/backpressure.
  2. Fairness. It shares a common capacity across tenants or consumers, so one noisy consumer cannot starve the others.
  3. Accounting. Per-key counters are also a metering surface for billing, cost attribution, and budgets.

Where it is enforced matters. An edge or gateway limit (patterns/central-proxy-choke-point) stops traffic before it costs backend work. A distributed limit inside a worker fleet must coordinate counters across workers. A limit that outlives its reason becomes a source of outages (see the GitHub source below).

Seen in

Last updated · 854 distilled / 2,448 read