Skip to content

CONCEPT Cited by 1 source

Self-censoring forecast

A self-censoring forecast is a predictor that scores its own recent accuracy and only emits a prediction when that accuracy has been high. When recent error exceeds a threshold, the predictor returns "no prediction" rather than an untrustworthy one; downstream consumers fall back to another predictor or to a default policy.

Named directly in MongoDB's 2026-04-07 predictive auto-scaling retrospective:

"So we added a 'self-censoring' mechanism to our prototype: the Long-Term Forecaster scores its own confidence based on its recent accuracy, and only trusts its prediction if its recent error has been small." (Source: sources/2026-04-07-mongodb-predictive-auto-scaling-an-experiment)

Why it matters

A point-forecast-plus-always-act policy breaks when the forecast goes bad. For autoscaling that means scaling to the wrong size (over-provision → wasted cost; under-provision → customer latency incident). The self-censoring gate is the minimal intervention that makes acting on the forecast safe — whenever the model's own recent accuracy fails, withhold the action.

The mechanism is deliberately pessimistic: it's willing to miss some scaling opportunities (false negatives on the confidence gate) in exchange for avoiding incorrect scaling actions (false positives). In a fallback architecture the missed opportunities are picked up by a second predictor (MongoDB's Short-Term Forecaster) or by the reactive backstop; the dangerous false positives are suppressed at source.

Construction

The MongoDB post is vague on the exact formula ("scores its own confidence based on its recent accuracy") but the standard shape is:

# At prediction time t:
recent_errors = [error(i) for i in last_N_predictions]
recent_error = some_aggregation(recent_errors)   # mean, max, percentile
confidence = monotone_decreasing(recent_error)
if confidence < threshold:
    return NO_PREDICTION
else:
    return model.predict(t)

Axes to tune:

  • Window size — how many past predictions count.
  • Aggregation — mean is standard; max or p95 is more pessimistic.
  • Threshold — calibrated on held-out data to achieve a target false-positive rate.
  • Decision granularity — per replica set (MongoDB's case, because each replica set has its own workload shape) vs. global threshold.

Relationship to uncertainty quantification

Self-censoring is one flavour of uncertainty quantification. Two flavours differ in where the uncertainty signal comes from:

  1. Validation-accuracy flavour (self-censoring): recent held-out error on this series tells you if the model is currently reliable. MongoDB's approach.
  2. Model-output flavour (predicted-distribution-width): the model itself emits a distribution rather than a point; width of the distribution is the uncertainty signal. Google RLM-on-Borg's approach (sources/2025-07-29-google-simulating-large-systems-with-regression-language-models).

Both are valid; validation-accuracy doesn't require the model to support distributional output and works on a wider class of predictors (boosted trees, point-forecast time-series models). Predicted-distribution gives per-prediction granularity rather than per-time-window.

Relationship to fallback patterns

Self-censoring is the gate primitive for fallback-structured control loops:

In both cases the gate is doing the same job: deciding "is it safe to act on the faster / less authoritative prediction?". Self-censoring is a concrete implementation of that gate when the predictor isn't distributional.

Risk framing

The gate's thresholds encode the architect's asymmetric risk framing:

  • Wrong-forecast-then-act cost high (e.g., scale down a production DB incorrectly) → tight threshold, lots of censoring, rely more on backstop.
  • Wrong-forecast-then-act cost low (e.g., proactively pre-warm a cache) → loose threshold, act more often, rarely fall back.

MongoDB's production predictive scaler ships scale-up only — the gate is calibrated for the lower-risk direction; the higher-risk scale-down action is left to the reactive backstop.

Seen in

Last updated · 200 distilled / 1,178 read