Skip to content

CONCEPT Cited by 2 sources

Database throttler

Definition

A database throttler is a service or component that pushes back on an incoming flow of database operations so that the database system stays healthy under load. It is a backpressure primitive specialised to databases: the client asks the throttler before doing work, and the throttler's answer is a go / no-go / retry later.

"A throttler is a service or component that pushes back against an incoming flow of requests to ensure the system has the capacity to handle all received requests without being overwhelmed." — Shlomi Noach, Anatomy of a Throttler, part 1 (PlanetScale, 2024-08-29)

Primary target workload

Throttlers in the database world are usually built for asynchronous, batch, massive operations:

  • ETLs / data imports / data exports
  • Online DDL (schema changes running against production tables)
  • Mass data purges (GDPR deletes, expiry cleanup)
  • Resharding / rebalancing workflows
  • Background analytics scans

These operations span minutes to days and cannot be allowed to single-handedly tank the database's production serving capacity.

"The throttler will push back on those operations that can span minutes, hours, or days of operation. Other forms of throttling may push back on OLTP production traffic. This discussion applies equally to both."

Why databases need this, not a generic rate limiter

Generic rate limiters assume a fixed service time per request: N requests per second ≈ N × (service time) ≈ a known load. That assumption fails in databases — cost per query depends on:

  • Query scope — small point-lookup vs full-table scan.
  • Hot/cold spots — contended row vs untouched partition.
  • Page-cache state — in-memory vs fetched-from-disk.
  • Data overlap — many queries on the same page vs on distinct pages.

A throttler therefore pushes back on system-health signals — metrics that summarise the system's ability to absorb more load right now — rather than on raw request rate.

Check-then-proceed contract

The canonical shape (see patterns/collaborative-throttler-check-api):

subtask_size = tuned_constant  # e.g. 100 rows
for batch in work:
    while not throttler.ok():
        sleep(backoff_period)
    apply(batch)

subtask_size is tuned between:

  • Small enough that one subtask cannot by itself tank the database.
  • Large enough that per-check throttler overhead is amortised and net progress is meaningful.

Two throttler shapes

Shape Contract Where enforcement lives
Collaborative Client asks, client respects answer. In the client. Lightweight, requires trust.
Barrier Sits in the request path between app and database; rejects directly. In a proxy / gateway. Heavier, enforces compliance.

MySQL-world tooling (gh-ost, pt-online-schema-change, Vitess throttler) defaults to collaborative. A client that ignores the throttler is not stopped from submitting work — only shamed.

Seen in

  • concepts/backpressure — parent primitive. A database throttler is the special case of backpressure where the producer is a batch job and the consumer is a multi-tenant database.
  • concepts/queueing-theory — every health metric the throttler reads is a summary of some queue (changelog, commit, run-queue, connection pool). The throttler is the mechanism by which the operator caps the depth of those queues.
  • concepts/replication-lag — the most-used MySQL throttling signal.
  • concepts/symptom-vs-cause-metric — every load-predicting metric is a symptom; the throttler is the mechanism that translates symptom-level thresholds into client-level admit/reject decisions.
  • patterns/multi-metric-throttling — the recommended architectural shape.
  • patterns/throttler-observability-substitute — the consequence: once throttling is in place, the database stops being the root-cause surface for throttled traffic.
Last updated · 319 distilled / 1,201 read