PATTERN Cited by 1 source
Singular vs distributed throttler¶
Pattern¶
A throttler — the service that rejects / accepts client requests based on system-health metrics — can be deployed as one monolithic process (singular) or as many collaborating processes (distributed). The choice is a topology-level decision that shapes availability, metric staleness, connection counts, and cost.
The design space¶
The Noach post enumerates five points on the axis:
| Variant | Throttler count | Metric source | HA story |
|---|---|---|---|
| Pure singular | 1 | Direct connections to every DB + OS metric source | Hope it stays up; fail-open client policy |
| AZ-parallel singular | 1 per AZ, each oblivious | Direct, per-AZ scope | Losing one AZ loses one throttler, not the decision path globally |
| Active-passive singular | 1 active + N standby | Direct | Standby steps up on active failure; may probe while passive |
| Singular + per-host agent | 1 central + N host agents | Agents poll locally, throttler polls agents over HTTP | Agents are independent; throttler is the SPOF; has layered staleness cost |
| Fully distributed (per-host / per-shard) | Many | Each throttler scrapes local metrics; peers gossip on demand | No single point; more coordination overhead |
"A singular throttler is an all-knowing service that serves all requests … it is a simple, monolithic, synchronous approach."
— Shlomi Noach, Source: sources/2026-04-21-planetscale-anatomy-of-a-throttler-part-2
Why singular is attractive at first¶
- Simple data model — one process knows everything.
- Synchronous probing — no polling-layer staleness compounding.
- Easier to reason about — one source of truth.
Why singular doesn't scale¶
- Connection count ceiling. "There's only so many connections it can maintain while running high-frequency probing."
- Security / environmental restrictions. Many environments don't permit one process to hold persistent probe connections to every DB + host.
- Single point of failure. The fail-open vs fail-closed choice becomes load-bearing on the client.
- Regional locality. Probing hosts in another AZ costs round-trips + crosses fault boundaries.
The intermediate step: per-host agents¶
Add a metrics agent on each host; the singular throttler polls an HTTP API instead of holding direct connections.
"In adding this agent component, even if it's the simplest script to scrape and publish metrics, we've effectively turned our singular throttler into a distributed multi-component system."
Cost: layered polling staleness — agent 1 Hz + throttler 1 Hz = up to 2 s stale worst-case instead of 1 s.
Distributed variants¶
Per-AZ independent throttlers — each AZ has its own throttler; clients target their AZ's throttler; no cross-AZ metric flow.
Per-AZ throttlers that gossip — one AZ's throttler advertises its metrics to peers; peers grab all AZ metrics in one pull. "This is a glorified and scaled implementation of the agent architecture above, where all components are throttlers."
Functional partitioning — disjoint services get dedicated throttlers; no cross-service probing; bounded fan-out per throttler.
Per-host / per-service (Vitess-style) — one throttler per tablet / MySQL server; see patterns/throttler-per-shard-hierarchy for the aggregation story.
Decision criteria¶
- Fleet size + connection budget — past some count, singular is infeasible.
- Security boundaries — if central probing is forbidden, agents or per-host throttlers are mandatory.
- Metric-staleness tolerance — monolithic keeps staleness tightest; agent-mediated doubles it; peer-gossiping adds more.
- Availability target — active-passive helps a bit; AZ-parallel helps more; per-host eliminates the single- point altogether.
- Cross-scope needs — if shard-scope and host-scope metrics are both needed, the hierarchy pattern wins.
Compositions¶
The pattern composes with other throttler patterns:
- + patterns/throttler-per-shard-hierarchy — the Vitess canonical shape: per-host throttlers roll up to a shard-primary for shard-scope queries.
- + patterns/idle-state-throttler-hibernation — orthogonal axis: any topology can hibernate during idle.
- + patterns/host-agent-metrics-api — the bridge from pure singular toward distributed.
Seen in¶
- sources/2026-04-21-planetscale-anatomy-of-a-throttler-part-2 — canonical wiki introduction. Shlomi Noach walks the entire design space from pure singular through active-passive and agent-mediated to the fully distributed Vitess tablet throttler. Cross-shard throttler communication is deliberately excluded as a sustainability-of-fan-out measure.