Skip to content

PATTERN Cited by 2 sources

Multi-metric throttling

Problem

A throttler that keys its admit/reject decision on a single metric is vulnerable to every failure mode that metric doesn't capture. The canonical MySQL example:

  • Throttle on replication lag → catches replica falling behind, misses primary commit-queue stalls.
  • Throttle on threads_running → catches concurrency spikes, misses replication-lag degradation on the write path.
  • Throttle on pool exhaustion → catches connection-level backpressure, misses slow-query degradation.

Every one of these metrics is a symptom of some underlying queue; a single metric summarises only one chain.

Solution

"A throttler should be able to push back based on a combination of metrics, and not limit itself to just one metric. We've illustrated some metrics above, and every environment may yet have its own load predicting metrics. The administrator should be able to choose an assorted set of metrics the throttler should work with, be able to set specific thresholds for each such metric, and possibly be able to introduce new metrics either programmatically or dynamically."

— Shlomi Noach, Anatomy of a Throttler, part 1

The pattern:

  1. Enumerate candidate metrics. Replication lag, commit delay, pool usage, load average, custom workload metrics, query queues, lock waits.
  2. Operator selects a subset appropriate to their deployment.
  3. Per-metric thresholds are set independently (5 s lag, 100% pool usage, 50 ms commit delay, etc).
  4. Throttler admits only if all metrics are below their thresholds. Any single exceeded metric → reject.
  5. Extensibility hook for adding new metrics dynamically — custom queries, external probes, plugged-in signals.

Why combination, not override

The simpler alternative would be "pick one metric" or "hierarchical override (replication lag first, fall back to pool usage)". The combination approach wins because:

  • Failure modes don't stack predictably. The one metric that catches this degradation may not be the one that caught last degradation. Combinations catch orthogonal cases.
  • False-positive isolation. A single metric can be noisy due to measurement quirks; requiring multiple metrics to agree reduces spurious throttling.
  • Separation of concerns. Replication lag is an async-apply signal; pool exhaustion is a sync-blocking signal; commit delay is a write-path signal. A throttler over all three respects the independent concerns of each.

Implementation shape

Pseudo-interface:

throttler.addMetric(name="replication_lag", threshold=5.0, fetcher=fetch_replication_lag)
throttler.addMetric(name="pool_usage",     threshold=1.0, fetcher=fetch_pool_usage)
throttler.addMetric(name="commit_delay",   threshold=0.05, fetcher=fetch_commit_delay)

def check():
    for metric in throttler.metrics:
        if metric.value() > metric.threshold:
            return Reject(metric.name)
    return Ok()

Real implementations (e.g. Vitess) generalise the fetcher slot to support arbitrary custom SQL probes defined by the operator — "a metric" becomes "whatever returns a number when you run this query".

What not to put in the metric list

  • Arbitrary rate limits (e.g. "max 1000 queries per second"). Rate limits are not health signals; see concepts/database-throttler on why databases can't use generic rate limiters.
  • Instantaneous CPU %. Too noisy; use load average or run-queue latency instead.
  • Metrics with no natural threshold. Either work to find the threshold (reading it from system configuration, like pool size) or choose a different metric.

Seen in

  • sources/2026-04-21-planetscale-anatomy-of-a-throttler-part-1 — canonical articulation of this pattern. Noach proposes it as the recommended architectural shape for throttlers at the end of his metrics walk-through.
  • sources/2026-04-21-planetscale-announcing-vitess-21 — production shipping of multi-metric throttler in Vitess 21: "The tablet throttler has been redesigned with new multi-metric support. With this, the throttler now handles more than just replication lag or custom queries, but instead can work with multiple metrics at the same time, and check for different metrics for different clients or for different workflows." Pairs structurally with the same-day Anatomy-of- a-Throttler series — that series canonicalises the design space; this release canonicalises the shipping primitive. v20 backward-compat preserved; compat removed in v22.
  • systems/vitess-throttler — canonical implementation of this shape. Vitess's throttler accepts custom metric definitions via configurable SQL probes, with per-metric thresholds, on top of its default replication-lag metric.
Last updated · 319 distilled / 1,201 read