Skip to content

PATTERN Cited by 1 source

Init as credential broker

Pattern

Make the init binary of a VM / micro-VM / container detect credential-requesting environment variables, fetch the required identity material from the platform's metadata service, write it to disk / environment variables in the shape the relevant SDK expects, and then launch the user's entrypoint. The application needs no code change — the SDK's standard credential-provider chain picks up the right file / env var automatically.

The pattern depends on three existing pieces working together:

  1. A platform-side identity service that can issue short-lived tokens (OIDC IdP, IMDS, Kubernetes projected token volume).
  2. A cloud SDK that already supports a standard credential- provider chain that reads from files / env vars (AWS SDK, GCP SDK, Azure SDK all do).
  3. An init binary with access to the platform's metadata service and the authority to set env vars for its children.

Canonical wiki instance

Fly init on Fly Machines:

If, when init starts in a Fly Machine, it sees an AWS_ROLE_ARN environment variable set, it initiates a little dance; it:

  1. goes off and generates an OIDC token, the way we just described,
  2. saves that OIDC token in a file, and
  3. sets the AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_SESSION_NAME environment variables for every process it launches.

The AWS SDK, linked to your application, does all the rest.

(Source: sources/2024-06-19-flyio-aws-without-access-keys)

Developer-facing UX

Before (static access keys):

# fly.toml
[env]
AWS_DEFAULT_REGION = "us-east-1"
# and secrets:
# fly secrets set AWS_ACCESS_KEY_ID=AKIA...
# fly secrets set AWS_SECRET_ACCESS_KEY=...

After (init as credential broker):

# fly.toml
[env]
AWS_DEFAULT_REGION = "us-east-1"
AWS_ROLE_ARN = "arn:aws:iam::123456123456:role/cat-bucket"
# No secrets at all.

The Docker image is unchanged. The application code is unchanged. The SDK is unchanged.

Why this works

The AWS SDK's credential-provider chain

The AWS SDK already looks for AWS_WEB_IDENTITY_TOKEN_FILE as one of the canonical inputs to its credential-provider chain (documented at aws/sdkref credentialProviderChain). When the env var is set and points to a readable file containing a JWT, the SDK automatically calls AssumeRoleWithWebIdentity at startup to exchange the JWT for STS credentials, then refreshes them as they near expiry.

This is the same hook EKS IRSA (IAM Roles for Service Accounts) uses inside Kubernetes Pods, and the same hook GitHub Actions uses in its OIDC support.

The init binary can see and set the environment

Init is process 1. It reads the Machine's env (set by the platform control plane), makes decisions, and execs the entrypoint with a modified environment. The application gets the env it expects, plus the additional vars init decided to add.

The metadata service is accessible from init

init has access to /.fly/api (the Unix socket proxy) with the Machine-scoped Macaroon attached. It can call the OIDC token endpoint before any user code runs.

Why this is the right layer for the broker

Alternatives and why they're worse:

  • Do it in the application. Requires every application owner to wire up the OIDC exchange themselves; defeats the "no code change" win. Also spreads the Macaroon handling into application code.
  • Do it in an LD_PRELOAD shim. Possible but fragile; requires cooperation with the entrypoint's dynamic linker setup; doesn't handle static binaries.
  • Do it in a sidecar. Adds a process, a socket, and another surface. The init binary already has the privileges needed and runs exactly once per Machine boot.
  • Do it in the platform control plane (and pre-fill env). Credentials aren't known until runtime (OIDC token is minted against the specific Machine + time); pre-filling static env from the control plane doesn't work.

Generalisations

The pattern isn't Fly-specific. Same shape appears in:

  • EKS Pod Identity / IRSA — the kubelet projects a service- account token into the Pod; the AWS SDK reads AWS_WEB_IDENTITY_TOKEN_FILE (the same env var Fly uses) and calls STS.
  • GKE Workload Identity — metadata server inside Pod returns GCP creds via the GCP SDK's metadata-client path.
  • EC2 IMDS — the classic instance-profile credential broker, built into the SDK's provider chain rather than into init.

Fly's contribution is recognising the init layer as the natural place to do the AWS_ROLE_ARN detection + env-var export for a platform that doesn't have kubelet / AWS-host-agent infrastructure.

When to use

  • You operate a compute platform (VMs, micro-VMs, containers).
  • You want application owners to get platform-issued credentials with zero code change and minimal configuration.
  • Your runtime has a process-zero you control (init).
  • Downstream cloud SDKs support env-var / file-based credential pickup (AWS, GCP, Azure, Vault all do in some form).

When not to use

  • Static / bare-metal hosts without a platform-managed init — you'll want a sidecar agent (e.g., HashiCorp Vault Agent, AWS sm-agent) instead.
  • The SDK doesn't support file/env-var pickup — you'd need to inject a credential-library shim.

Seen in

  • sources/2024-06-19-flyio-aws-without-access-keys — canonical wiki instance. The post is explicitly framed as "one init feature remains to be disclosed, and it's cute"; the full five-step init dance is disclosed; the AWS SDK integration is a single environment-variable flip (AWS_WEB_IDENTITY_TOKEN_FILE).
Last updated · 200 distilled / 1,178 read