Skip to content

SYSTEM Cited by 1 source

oidc.fly.io (Fly.io OIDC IdP)

oidc.fly.io is Fly.io's in-house OpenID Connect identity provider. It issues short-lived OIDC tokens to Fly Machines and only to Fly Machines — Fly's OIDC vault runs on isolated hardware, and the issuer URI is per-org (https://oidc.fly.io/<org_name>). Counterparties (AWS, GCP, Azure, or any OIDC-compliant cloud) register it as an Identity Provider once, then constrain trust via audience + subject claims. (Source: sources/2024-06-19-flyio-aws-without-access-keys)

Per-org issuer, standard OIDC discovery

Counterparties verify tokens against the standard OIDC discovery endpoint:

https://oidc.fly.io/<org>/.well-known/openid-configuration

which "references a key Fly.io manages on isolated hardware." Anyone can verify a Fly.io OIDC token; only Fly.io can sign one.

Token shape

Tokens carry the standard OIDC claims plus Fly-specific identity claims:

  • iss — https://oidc.fly.io/<org_name> (per-org).
  • aud — caller-provided (sts.amazonaws.com for AWS STS).
  • sub — <org>:<app>:<machine> (the identity-scoping pivot).
  • app_id, app_name, org_id, org_name.
  • machine_id, machine_name, machine_version.
  • image, image_digest — the Docker image identity of the Machine's workload.
  • region — Fly region (e.g. yyz).

See sources/2024-06-19-flyio-aws-without-access-keys for the full example payload.

API surface

Tokens are minted via the Fly Machines API at /v1/tokens/oidc. The Machine calls this endpoint through init's Unix-socket proxy at /.fly/api — which attaches the Machine- scoped Macaroon token the guest code never sees. flyd knows which Machine is calling and refuses requests that don't match the Macaroon.

The caller can set arbitrary audience strings, so the same IdP can vend tokens for AWS STS, GCP Workload Identity Federation, Azure workload federation, or any other OIDC-compliant RP.

Sub-field scoping

Because sub is <org>:<app>:<machine>, AWS IAM trust policies (and equivalents on other clouds) can scope access along the full hierarchy:

  • Whole org → "example:*".
  • Specific app, any machine → "example:weather-cat:*".
  • Specific machine → "example:weather-cat:ancient-snow-4824".

Canonical sub-field scoping instance.

Security model

Two layers of protection for the minting path:

  1. The Machine's Macaroon token is Machine-bound. flyd verifies the requester identity; the Macaroon is useless off-Machine.
  2. The Macaroon never reaches the guest. init is the only component in the Machine that holds it; init exposes a proxy on /.fly/api that attaches the Macaroon on outbound requests and strips it from responses. Application code in the Machine has no way to exfiltrate the Macaroon — "you could rig up a local privilege escalation vulnerability and work out how to steal the Macaroon, but you can't exfiltrate it productively."

The OIDC token itself is short-lived ("dead in minutes" verbiage used for the downstream STS credential — the OIDC token life isn't explicitly disclosed but is on the same scale) and bound to a specific Machine via sub.

Fly.io's positioning

For us, the gold standard for least-privilege, conditional access tokens remains Macaroons, and it's unlikely that we're going to do a bunch of internal stuff using OIDC. We even snuck Macaroons into this feature. (Source: sources/2024-06-19-flyio-aws-without-access-keys)

OIDC is Fly's answer for external RP federation (AWS / GCP / Azure / any OIDC-compliant service) — not a replacement for Fly's internal Macaroon-based authz. The two coexist: a Machine's Macaroon gates access to the Machines API, which in turn gates issuance of the OIDC token the Machine uses outbound.

Seen in

  • sources/2024-06-19-flyio-aws-without-access-keys — named as the per-org IdP; full token payload disclosed; well-known OIDC discovery endpoint named; isolated-hardware-managed keys; sub- field shape <org>:<app>:<machine>; Machines-API /v1/tokens/oidc endpoint; arbitrary-audience design for any OIDC-compliant counterparty.
Last updated · 200 distilled / 1,178 read