Skip to content

PATTERN Cited by 1 source

Tokenized-token broker

Shape

A tokenized-token broker is a hardware-isolated intermediary that holds the real credential (OAuth token, API key, refresh token, mail-spool grant) and substitutes it into outbound calls from a less-trusted client. The client carries only a placeholder token and cannot recover the real secret even under compromise.

┌──────────────┐  request with placeholder    ┌──────────────────────┐  request with real token   ┌────────────┐
│  LLM client  │ ─────────────────────────▶   │  Broker (isolated)   │ ─────────────────────────▶ │  API       │
│  (untrusted) │                              │  holds real secret   │                            │  (Google,  │
│              │                              │  substitutes placeholder│                         │   Stripe,  │
└──────────────┘                              │  → real at egress    │                            │   …)       │
                                              └──────────────────────┘                            └────────────┘

The broker owns the secret; the client never does. A client compromise exposes the placeholder, not the real credential.

Canonical wiki statement

Fly.io, 2025-04-08:

On a modern cloud platform, there's really no reason to permanently grant Sam Altman Google Mail access, even if you want his robots to sort your inbox. You can decouple access to your mail spool from persistent access to your account by tokenizing your OAuth tokens, so the LLM gets a placeholder token that a hardware-isolated, robot-free Fly Machine can substitute on the fly for a real one.

(Source: sources/2025-04-08-flyio-our-best-customers-are-now-robots)

The broker in Fly.io's instantiation is a robot-free Fly Machine — a Fly Machine on the same Firecracker hypervisor substrate as the client, but on separate worker hardware, running only the substitution logic. The underlying secret-tokenization system is Fly's tokenized-tokens work from 2024.

Required properties

For the broker shape to be worth the architectural hop:

  1. Hardware isolation from the client. Not just a different process; a different hardware-isolated host. The client can only reach the broker through the broker's published API; it cannot read its memory.
  2. Minimal surface on the broker. The broker accepts only outbound calls whose placeholder it recognises, to a specific destination, with a specific shape. A general-purpose "swap any token" proxy is a privilege escalator.
  3. No client code on the broker. If the client can inject code onto the broker, the isolation is theoretical. In Fly.io's framing this is why the substitution Machine is "robot-free" — only substitution code runs there.
  4. Auditable substitution policy. The set of allowed (placeholder, destination, call-shape) tuples has to be explicit. Someone human has to approve each one.
  5. Placeholder lifecycle. Placeholders should be revocable independently of the underlying credential and (ideally) single-use / auditable. If a placeholder leaks, the blast radius is bounded.

Steps

  1. User grants the underlying credential once via the normal flow (OAuth dance, API key copy-paste). Credential lands on the broker, not the client.
  2. Broker issues a placeholder token bound to that credential, with a declared substitution policy.
  3. Client (LLM, agent, integration worker) gets the placeholder. Places it in its credential store, environment, or tool-call arguments.
  4. On outbound call, the client constructs a request as if the placeholder were the real credential.
  5. The request routes through the broker (typically via platform egress rules, so the client can't bypass it).
  6. Broker validates the placeholder against the policy: correct destination? correct call shape? not revoked?
  7. Broker substitutes the placeholder for the real credential and forwards the request to the third-party API.
  8. Response comes back through the broker to the client.

What this is not

  • Not OAuth scope narrowing. OAuth scopes narrow what the user can do through the token. Tokenization narrows who can hold the token (or its proxy). The two are complementary.
  • Not a VPN or egress proxy. A VPN puts the client's traffic on a controlled network. Tokenization puts the secret on a controlled host. A VPN doesn't protect against the client leaking the secret; tokenization doesn't protect against the broker leaking the traffic.
  • Not a capability URL. Capability URLs embed the permission in an opaque URL. Tokenized secrets embed nothing in the placeholder; the placeholder is looked up on the broker.
  • Not short-lived credentials. Short-lived credentials (concepts/ephemeral-credentials, concepts/short-lived-credential-auth) narrow the time window; tokenization narrows the holder. Compositional: the real credential on the broker can itself be short-lived.

Adjacencies on the wiki

  • patterns/init-as-credential-broker — Fly.io's fly-init brokers AWS OIDC federation tokens to the guest process; same substrate-and-broker shape, different credential type (AWS STS session credentials instead of OAuth to a third-party). The tokenized-token broker extends this pattern from "AWS credentials" to "any credential with a placeholder."
  • patterns/oidc-role-assumption-for-cross-cloud-auth — Fly.io → AWS via OIDC federation; the broker shape applied to the federated-identity axis specifically. Tokenized tokens generalise past identity-federation to any third-party credential.
  • patterns/credentialed-proxy-sandbox — Cloudflare's framing for Agent Lee: the agent runs in a sandbox; the credentialed proxy tier holds the secrets and mediates outbound calls. Same shape.
  • External: HashiCorp Vault transit-secrets / dynamic secrets. Long-standing non-LLM instance — applications get a Vault-backed token that Vault unwraps at egress. The Fly.io post is the LLM-era variant of an older pattern.

Known uses

  • Fly.io (2025-04-08 blog post + 2024 tokenized-tokens system) — robot-free Fly Machines as the broker substrate; canonical wiki instance for the LLM-agent variant.
  • Cloudflare Agent Lee (2026-04-15) — credentialed-proxy sandbox shape, via patterns/credentialed-proxy-sandbox; brokers outbound calls from agent-hosted code.
  • HashiCorp Vault (pre-LLM) — dynamic secrets + transit-secrets engine; the original enterprise broker shape.

Operational risks

  • Broker compromise = full secret loss. If the broker host is compromised, all placeholders it knows about are compromised. This collapses the isolation. Mitigation: narrow per-broker scope (one broker per integration, single-tenant brokers, HSM-backed brokers).
  • Substitution policy drift. If the allowlist of permitted substitutions grows without discipline, the broker becomes a generic token-swap tool and the isolation degrades to theoretical.
  • Replay of placeholders. If the broker doesn't track placeholder usage, a leaked single-use placeholder might be spent before the legitimate client sees the response.
  • Latency. Every outbound call takes an extra hop. For chatty APIs this can be material.
Last updated · 200 distilled / 1,178 read