Skip to content

CONCEPT Cited by 1 source

Tokenized secret

Definition

A tokenized secret is a placeholder credential — opaque to the client — that a trusted egress hop substitutes for the real secret at the moment the outbound call leaves the platform. The client (in Fly.io's framing, an LLM running an integration workflow) never sees the real credential; it only sees and operates on the placeholder. Because the substitution happens on a hardware-isolated hop the client cannot reach, a compromise of the client cannot expose the real secret.

Canonical wiki statement

Fly.io, 2025-04-08:

The pact the robots have with their pet humans is that they'll automate away all the drudgery of human existence, and in return all they ask is categorical and unwavering trust, which at the limit means "giving the robot access to Google Mail credentials". The robots are unhappy that there remain a substantial number of human holdouts who refuse to do this, for fear of Sam Altman poking through their mail spools.

But on a modern cloud platform, there's really no reason to permanently grant Sam Altman Google Mail access, even if you want his robots to sort your inbox. You can decouple access to your mail spool from persistent access to your account by tokenizing your OAuth tokens, so the LLM gets a placeholder token that a hardware-isolated, robot-free Fly Machine can substitute on the fly for a real one.

(Source: sources/2025-04-08-flyio-our-best-customers-are-now-robots)

The system Fly.io cites for the substitution is its own tokenized-tokens work from 2024.

The architectural move

A traditional OAuth integration:

[LLM] ── real OAuth token ──▶ [Google Mail API]

An LLM compromise = real token compromise.

A tokenized-secret integration:

[LLM] ── placeholder ──▶ [Hardware-isolated Fly Machine] ── real OAuth token ──▶ [Google Mail API]
                           (performs substitution at egress)

An LLM compromise only reveals the placeholder. The real token lives on the isolated Machine and is never surfaced.

Requirements on the substitution hop

For the substitution model to be worth the architectural cost, the hop that owns the real secret has to be:

  1. Hardware-isolated from the LLM — the LLM can't touch this host's memory, read its secrets, or exfiltrate via a shared kernel. Fly's substitution Machine is a Firecracker-backed Fly Machine on separate workers from the client.
  2. Robot-free — no LLM code runs on this hop; only the substitution logic. Otherwise the separation is theoretical.
  3. Narrow in scope — the substitution hop should only accept outbound calls whose placeholder it recognises, to a specific destination, with specific shape. A general-purpose "swap any token" proxy is a privilege escalator.
  4. Short-lived / auditable — ideally the placeholder is single-use and auditable so that loss of a placeholder is a bounded incident.

Contrast with short-lived / ephemeral credentials

  • concepts/ephemeral-credentials — the credential itself is short-lived (minutes). Loss is bounded by TTL.
  • concepts/short-lived-credential-auth — Fly.io's framing of STS-style federated credentials: "dead in minutes, sharply limited blast radius, rotate themselves, fail closed."
  • Tokenized secret — the client never holds the real credential at all, regardless of TTL. Useful when the real credential is long-lived by nature (OAuth refresh tokens, API keys the third party issues) or when the third party doesn't support short-lived auth flows.

The three postures compose: a tokenized OAuth token can itself be short-lived at the substitution hop, with ephemeral credentials on top of that for the client-to-hop authentication.

Why this matters for RX

RX requires a secret-handling posture that doesn't force the user to decide "do I trust OpenAI with my Gmail?" as a one-time yes/no. Tokenized secrets let the user grant the LLM access scoped by substitution hop — the LLM gets a placeholder that only works through a specific, auditable path. Revoking the placeholder (or narrowing the substitution scope) is an independent control from "log into Google and revoke the OAuth grant."

The post's broader framing is that the integrations-between- APIs market (Zapier, Make, IFTTT) is fundamentally about managing secrets across APIs, and tokenization is the right architectural primitive for that market whether or not robots are the primary driver:

There are several big services that exist to knit different APIs together, so that you can update a spreadsheet or order a bag of Jolly Ranchers every time you get an email. The big challenge about building these kinds of services is managing the secrets. Sealed and tokenized secrets solve that problem. There's lot of cool things you can build with it.

(Source: sources/2025-04-08-flyio-our-best-customers-are-now-robots)

Limits and open questions

  • Does not protect against the substituted call doing harm. If the LLM's placeholder has mail-send authority, the substitution hop will happily send mail on the LLM's behalf. Tokenization controls who has the secret; it doesn't control what the secret is used for. That's an authorisation concern on the substitution hop's allowlist, not a tokenization concern.
  • Does not protect the substitution hop from exfiltration. If the substitution hop is compromised, the real token is compromised. The argument is that the substitution hop is a narrow, well-audited Fly Machine; the client (LLM) is broad, poorly-audited, and exposed to prompt injection.
  • Token lifecycle on the substitution hop is not engaged with in this post. The substrate (Fly's tokenized-tokens post, 2024) owns that detail; this 2025 post only cites it.
  • No public framework for tokenized-secret scope. Fly's pitch is "placeholder for real" — but the placeholder's permissible uses, rotation, revocation, auditing are all substrate-level concerns not surfaced here.

Seen in

Last updated · 200 distilled / 1,178 read