Skip to content

CONCEPT Cited by 6 sources

Fail-open vs fail-closed

A design choice for what a module does when its input is corrupt, out-of-range, or fails an invariant:

  • Fail-closed — refuse to serve. Return 5xx / drop the request / panic the worker. Safer in security contexts (default-deny); dangerous in availability contexts (a single bad input takes out every request that reaches the module).
  • Fail-open — log the error, fall back to a known-good prior state or pass traffic without scoring, continue serving. Safer in availability contexts; dangerous in security contexts if the module was the only thing enforcing a policy.

The choice is not universal; different modules on the same hot path can and should differ. The discipline is to make the choice explicitly per module, not by accident.

The implicit-fail-closed trap

Many crashes are fail-closed by default, not by design:

  • .unwrap() on a Rust Result panics the worker.
  • A nil-index in Lua throws an exception the request handler doesn't catch.
  • An assertion in C++ aborts the process.

The programmer chose a terse syntax; the runtime chose fail-closed. The architecture never explicitly chose.

Canonical Cloudflare instances

The 12-05 post names the stated remediation "Fail-Open" Error Handling: "if a configuration file is corrupt or out-of-range (e.g., exceeding feature caps), the system will log the error and default to a known-good state or pass traffic without scoring, rather than dropping requests. Some services will likely give the customer the option to fail open or closed in certain scenarios."

The 11-18 post names the same project earlier: "Reviewing failure modes for error conditions across all core proxy modules."

Trade-off articulation

  • WAF / Bot Management / security modules — fail-open is controversial: you may serve traffic the module would have blocked. But at scale, serving-without-scoring is usually better than 5xx for everyone — the 5xx case denies service to the entire customer base, including the legitimate users the security module exists to protect.
  • Some customers may prefer fail-closed — explicit per- customer choice (from the 12-05 post: "give the customer the option to fail open or closed in certain scenarios").

Fail-stale as a stronger default

The 2026-05-01 Code Orange: Fail Small is complete post extends the binary fail-open-vs-fail-closed framing to a ternary ladder by introducing fail-stale as the preferred default:

We will now use the last known good configuration where possible ("fail stale"), and if that isn't possible we have reviewed each failure case and implemented "fail open" or "fail close" depending on whether serving traffic with reduced functionality is preferable to failing to serve traffic.

Ordering: fail stale (correct-behaviour-on-outdated-data) > fail open (degraded-behaviour) > fail closed (unavailable). Fail stale strictly dominates fail open whenever a last-known-good version exists — the "where possible" caveat is honest about the precondition (the module has to maintain a reference to the prior valid version, not just a single buffer).

Seen in

  • sources/2026-09-24-databricks-how-i-built-agent-based-security-reviews-on-databricks — conservative-default (fail-closed) applied to agentic decisioning. Databricks' multi-agent security-review system makes the fail-closed choice explicit for uncertain AI decisions rather than letting the model optimistically infer an approval: "When evidence is missing or contradictory, the system does not guess. It defaults to the more conservative risk tier, posts a specific clarification request, or hands the request to a reviewer... It does not find its way to approval." The risk-assessment agent "defaults to a higher tier when the picture is incomplete," and "an assertion with no evidence is treated as missing information." This is the same per-decision explicitness this page argues for — a security-shaped default (deny / escalate) chosen deliberately for a high-cost-of-wrong-approval context, and a novel instance of the choice living inside an LLM-agent pipeline rather than a proxy hot path. Pairs with HITL (the escalation target) and patterns/specialized-agent-decomposition (the conservative default is owned by one bounded agent).
  • sources/2026-09-09-aws-testing-application-resilience-with-amazon-sqs-and-aws-fault-injection-service — a self-inflicted fail-closed lockout via IAM's explicit-deny-wins rule. Injecting a resilience-test deny on SQS, if you deny sqs:* the explicit Deny overrides every Allow — including the automation's own permission to remove the policy later, so the cleanup step is denied and the queue is locked out with no in-band recovery path. The fix is the same discipline this page argues for, applied to authz: make the failure mode an explicit design choice — scope the deny to data-plane actions only (leaving management actions serving), add a validation step that refuses any deny covering management actions, and add a self-expiring DateLessThan condition so the deny fails open on a timer even if cleanup never runs. See systems/aws-iam.
  • sources/2026-08-12-meta-how-were-building-scam-alert-on-whatsapp-with-end-to-end-encryption-and-verifiability-guarantees — a fail-closed-on-the-privacy-invariant instance: WhatsApp Scam Alert's client refuses to transmit metrics unless the aggregator TEE binary matches the published ledger and the job's DP ε/δ + k-anonymity parameters meet local guardrails — fail-closed applied to confidentiality, not just availability.
  • sources/2026-05-01-cloudflare-code-orange-fail-small-complete — canonical wiki instance of the three-way ladder and the "fail stale" preferred default. Completes the 2025-12-05 stated "fail-open" remediation by naming a stronger default.
  • sources/2025-11-18-cloudflare-outage-on-november-18-2025
  • sources/2025-12-05-cloudflare-outage-on-december-5-2025

Example: fail-open on unmeasurable layout in a checker

GitHub's alt-text plugin makes the choice explicitly in its repeated-alt rule. When it cannot measure an image's bounding box, the run-grouping check fails open — it continues rather than emit a possibly-wrong grouping. Here the driver is not availability but false-positive cost in a linter: "a missing finding is invisible; a wrong one isn't." This is the same per-module explicitness this page argues for — a security-shaped default (don't emit) chosen for a quality-checker context. (Source: sources/2026-08-24-github-your-alt-text-passes-automated-checks-that-doesnt-mean-its-any-good)

Merged aliases

  • throttler-fail-open-vs-fail-closed- default-closed-routing
  • fail-stale
  • warn-mode-vs-enforce-mode
Last updated · 766 distilled / 2,225 read