SYSTEM Cited by 1 source

Pet Semetary (petsem)

Pet Semetary ("petsem") is Fly.io's in-house Vault replacement — the service that manages user secrets for applications running on Fly.io (e.g., Postgres connection strings, API keys, etc.). It is Fly.io's "second dedicated security service," alongside tkdb.

"Petsem manages user secrets for applications, such as Postgres connection strings. Petsem is its own Macaroon authority (it issues its own Macaroons with its own permissions system), and to do something with a secret, you need one of those Petsem-minted Macaroon." (Source: sources/2025-03-27-flyio-operationalizing-macaroons.)

What makes it wiki-worthy: the third-party-caveat

privilege-separation pattern

The architecturally interesting piece is how flyd gets to inject secrets into a booting Fly Machine without having a read-every-user's-secret Macaroon. Fly.io's design:

1. petsem mints its own Macaroons (own authority, own permissions system).
2. The Fly primary API holds a petsem Macaroon that allows writes but not reads — "there's no API call to dump your secrets, because our API servers don't have that privilege."
3. flyd (orchestrator, running on every worker in the fleet) needs to read secrets at Machine-boot time. So it gets a "read secret" petsem Macaroon — with a third-party caveat attached.
4. That caveat is dischargeable only by talking to tkdb and proving (with normal Macaroon tokens) that you have permissions for the org whose secrets you want to read.

Result: "Once again, access is traceable to an end-user action, and minimized across our fleet." No single flyd ever holds an unqualified read-all-secrets credential; every secret read is bound to proof that a valid user action authorized it.

Canonical wiki instance of patterns/caveat-for-privilege-separation.

Why it's not merged with tkdb

"We have at this point a second dedicated security service (Petsem), and even though they sort of rhyme with each other, we've got no plans to merge them. Rule #10 and all that."

Rule #10 of how complex systems fail ("All practitioner actions are gambles") — Fly.io's explicit application is that combining two narrow, stable security services into one larger one is a bet not worth taking. The "allergic to microservices" disclaimer in the same post is the counterpoint: narrow-purpose security services justify the carve-out.

Seen in

• sources/2025-03-27-flyio-operationalizing-macaroons — canonical wiki instance; petsem as secrets service, privilege-separation pattern, explicit "no plans to merge" with tkdb.

Related

• systems/tkdb — sibling security service; petsem relies on tkdb to discharge its third-party caveats.
• systems/flyd — the orchestrator that holds the caveated read-secret Macaroon petsem issues.
• systems/fly-machines — the execution environment petsem injects secrets into at boot.
• concepts/third-party-caveat — the primitive petsem uses for privilege separation.
• concepts/discharge-token — what petsem's caveat demands.
• patterns/caveat-for-privilege-separation — the canonical wiki pattern.
• companies/flyio.