Skip to content

CONCEPT Cited by 1 source

Keep hazmat away from complex code

A secure-design heuristic, stated by Fly.io's Thomas Ptacek as the explicit rationale for building tkdb:

"Root secrets for Macaroon tokens are hazmat, and a basic rule of thumb in secure design is: keep hazmat away from complicated code." (Source: sources/2025-03-27-flyio-operationalizing-macaroons.)

What "hazmat" means here

Secret material whose compromise would give an attacker broad authority — root HMAC keys, signing keys, long-lived credentials. The more complex the code that can reach that material, the more ways an attacker has to extract it (memory-safety bugs, RCE, injection, misconfiguration, dependency compromise).

The architectural move

Narrow the code surface that can touch the hazmat to the smallest, simplest service you can credibly build, and make it do only that. At Fly.io, tkdb is ~5,000 lines of Go (vs the rest of the Fly.io control plane being a much larger codebase), runs on isolated hardware, only speaks the token-authority RPC protocol, and doesn't serve any other product feature.

Contrast: putting the token authority inside the primary API cluster would have colocated root secrets with every unrelated feature-flag / billing / deployment / integration code path that ships in that cluster. Any vulnerability anywhere in that much bigger blob would put the secrets within reach.

  • concepts/blast-radius — the same instinct framed as a containment metric.
  • Defense in depth — hazmat isolation is one layer; it doesn't replace the others.
  • Minimum necessary privilege — tkdb itself only mints / verifies / revokes; it doesn't execute customer code or terminate user traffic.

Seen in

Last updated · 200 distilled / 1,178 read