Skip to content

FLYIO 2025-10-08

Read original ↗

Fly.io — Kurt Got Got

Summary

Fly.io security postmortem (2025-10-08) disclosing that Kurt Mackey, the CEO, was phished and the company's @flydotio Twitter/X account was taken over for ~15 hours. The attacker sent a plausible-looking "Twitter alert" email, Kurt pulled credentials from systems/1password|1Password and logged into a lookalike domain members-x.com (a malicious proxy for x.com), and the attacker immediately rotated the email-of-record to achilles19969@gmail.com, revoked tokens, and set up fresh 2FA — locking Fly.io out until X.com's recovery process completed ("15 hours for a 2FA reset isn't outside industry norms"). The attacker posted a low-effort crypto-airdrop scam and eventually deleted the account's history. Nothing in Fly.io's infrastructure was at risk because everything internal — logs, Stripe billing, infra metrics — is gated behind Google SSO with phishing-resistant MFA; Twitter was excluded from that regime because it had been treated as a deprioritised legacy shared account after the 2023-2024 "might not be a long-term thing for us at all" window during which Fly.io decamped to Mastodon and Bluesky. The post is more a policy narrative than a technical deep-dive, but it canonicalises three reusable architectural claims: (1) phishing-training doesn't work at scale — "under continued pressure, everybody clicks" ([people.cs.uchicago.edu phishing-training paper]); (2) the fix is phishing-resistant authentication via FIDO2/WebAuthn/Passkeys whose "origin- and channel-binding" breaks the malicious-proxy flow that phishing relies on; (3) the operational pattern is "everything behind an IdP that requires phishing-proof MFA" (Fly.io uses Google as IdP), which reduces the authentication-attack surface to "whatever you forgot to put behind the IdP" — which in Fly.io's case was a legacy social account. A secondary hygiene datum: 1Password's browser plugin would have refused to autofill on members-x.com because the URL didn't match the saved x.com entry — manual credential copy-paste defeated that guardrail. Incident-response timing is documented: Fly.io knew within 45 seconds (because the ATO simultaneously emailed all account-email-recipients a change-of- email notice), immediately audited 1Password access logs, and revoked access for anyone who had recently pulled the credentials on the worst-case assumption that an endpoint had been owned. The post concludes that blast-radius was limited enough ("not-very-plausible crypto scam that presumably generated $0") that Fly.io ran a watchful-waiting response rather than escalating — canonical instance of patterns/incident-response-calibrated-to-blast-radius. Twitter is now moved behind Passkeys. Tier-3 source; inclusion is marginal but the article explicitly articulates the origin- binding mechanism of FIDO2 and the "everything-behind-an-IdP" pattern, and serves as a real production anecdote for the phishing-resistant-MFA concept.

Key takeaways

  1. Twitter ATO via phishing; Fly.io infra not at risk. "Our Twitter got owned. We knew within moments of it happening. We know exactly how it happened. Nothing was at risk other than our Twitter account (and one Fly.io employee's self-esteem)." The attack is scoped to the one account that sat outside Fly.io's IdP-gated regime. Canonical instance of account takeover as a phishing outcome (Source: sources/2025-10-08-flyio-kurt-got-got).

  2. Phishing-training alone doesn't work — the fix is phishing-resistant authentication. "Contrary to one popular opinion, you don't defeat phishing by training people not to click on things. I mean, tell them not to, sure! But eventually, under continued pressure, everybody clicks. There's science on this. [linked to Ho et al. OAKLAND 2025 phishing training paper] The cool kids haven't done phishing simulation training in years. What you're supposed to do instead is use phishing-resistant authentication." Canonical statement of the training-is-not-sufficient claim on the wiki (Source: sources/2025-10-08-flyio-kurt-got-got; concepts/phishing-resistant-authentication).

  3. Phishing-resistant auth = origin + channel binding. "Phishing-resistant authentication works by mutual authentication (or, if you're a stickler, by origin- and channel-binding). Phishes are malicious proxies for credentials. Modern MFA schemes like FIDO2 break that proxy flow; your browser won't send real credentials to the fake site." The mechanism claim on the wiki: origin binding is the load-bearing property that defeats reverse-proxy phishing kits (Source: sources/2025-10-08-flyio-kurt-got-got; concepts/phishing-resistant-authentication).

  4. "Everything behind an IdP with phishing-proof MFA" is the operational pattern. "This is, in fact, how all of our infrastructure is secured at Fly.io; specifically, we get everything behind an IdP (in our case: Google's) and have it require phishing-proof MFA. You're unlikely to phish your way to viewing logs here, or to refunding a customer bill at Stripe, or to viewing infra metrics, because all these things require an SSO login through Google." Canonical instance of patterns/phishing-resistant-mfa-behind-idp: the discipline is not "every app implements FIDO2" but "every app is downstream of a single IdP that implements FIDO2" — which both reduces the integration surface (N×1 instead of N×M) and gives you a single chokepoint for off-boarding, auditing, and policy (Source: sources/2025-10-08-flyio-kurt-got-got; concepts/sso-authentication).

  5. Twitter fell outside the IdP regime because it was a legacy social account, not infra. "Twitter had been a sort of legacy shared account for us, with credentials managed in 1Password and shared with our zoomer contractor†." The account's deprioritisation was deliberate — in 2023-2024 Fly.io "decamped for Mastodon" and "later to Bluesky" and "there was a window of time… where it looked as if Twitter might not be a long term thing for us at all." Canonical instance of concepts/legacy-shared-account on the wiki: accounts that get demoted below the IdP threshold because the platform is deprioritised become the attacker's entry point when the platform unexpectedly regains relevance (Source: sources/2025-10-08-flyio-kurt-got-got).

  6. Password manager as a phishing guardrail — defeated by manual copy-paste. "The 1Password browser plugin would have noticed that 'members-x.com' wasn't an 'x.com' host." Kurt pulled the credential manually rather than autofilling, which bypassed the domain-matching check. Canonical instance of the patterns/password-manager-as-phishing-guardrail pattern (and its failure mode: manual copy-paste) on the wiki. The domain-matching behaviour of the password-manager browser plugin is a poor-man's origin check — weaker than FIDO2's cryptographic origin binding, but free if the credential is only ever autofilled (Source: sources/2025-10-08-flyio-kurt-got-got).

  7. Incident detection in ~45 seconds via a side-channel email notification. "We knew our X.com account had suffered an ATO because a bunch of us simultaneously got another email saying that the @flydotio account's email address now pointed to achilles19969@gmail.com. Our immediate response was to audit all accesses to the login information in 1Password, to cut all access for anybody who'd recently pulled it; your worst-case assumption in a situation like this is that someone's endpoint has been owned up." The email-of-record- change notification that the platform sent to every account- email was the detection signal, not the ATO itself. Response playbook: audit the credential store access log + revoke recent pullers on worst-case-endpoint-compromise assumption. Canonical first-party incident-detection-and-triage datum for ATO on the wiki (Source: sources/2025-10-08-flyio-kurt-got-got).

  8. Recovery time was platform-gated (~15 hours), not company- gated. "The attacker immediately revoked all tokens and set up new 2FA, so while we were quickly able to reset our password, we couldn't lock them out of our account without an intervention from X.com, which took something like 15 hours to set up. (That's not a knock on X.com; 15 hours for a 2FA reset isn't outside industry norms.)" Operational datum: attacker-set 2FA + attacker-set email → victim cannot self-recover, the recovery path routes through the platform's human-reviewed support channel, and industry-normal latency is ~O(10+ hours) for that path (Source: sources/2025-10-08-flyio-kurt-got-got).

  9. Watchful-waiting response because blast-radius was bounded. "In the grand scheme of things, the attack was pretty chill: a not-very-plausible crypto scam that presumably generated $0 for the attackers, 15+ hours of brand damage, and extra security engineering cycles burnt on watchful waiting. Our users weren't under attack, and the account wasn't being used to further intercept customer accounts." Canonical instance of patterns/incident-response-calibrated-to-blast-radius: the response intensity is scaled to the scope of what's actually at risk, not to the conspicuousness of the attack (Source: sources/2025-10-08-flyio-kurt-got-got).

  10. "x.com" is a structurally phishable domain. "Kurt complains that 'x.com' is an extremely phishable domain." Short, generic, one-character-different-from-lots-of-things, not discriminable-by-eye at a glance — the domain itself makes origin-discrimination harder for humans (which is why machine-enforced origin binding matters more when human origin-discrimination is degraded) (Source: sources/2025-10-08-flyio-kurt-got-got).

Systems extracted

System Role in this post
systems/1password Fly.io's shared-credential store; the attacker's victim (but only because the credential was pulled manually and copied into a lookalike domain — autofill would have blocked)
systems/twitter-x The compromised platform; 2FA-reset latency ~15 hours
systems/google-workspace-sso Fly.io's IdP for all internal infra; requires phishing-proof MFA
systems/okta Named as the exemplar category of enterprise IdP (Fly.io uses Google specifically; Okta is adjacent)
systems/stripe Named as the kind of downstream app behind Google SSO — "to refund a customer bill at Stripe… requires an SSO login through Google"

Concepts extracted

Concept Claim canonicalised
concepts/phishing-resistant-authentication Defined as mutual-auth / origin-and-channel binding; contrasted with password+OTP which is phishable via reverse-proxy kits; training is not a substitute
concepts/account-takeover-ato Defined: attacker gains control of an account, typically by combining phishing + immediate 2FA/email rotation to lock out the legitimate owner; recovery is platform-gated
concepts/origin-bound-credential Mechanism: credential is cryptographically bound to the origin it was registered against; browser refuses to sign challenges for the wrong origin; breaks the malicious-proxy pattern
concepts/legacy-shared-account Pattern: accounts on platforms that were deprioritised (or for which the company considered leaving) become deprioritised in security posture and are the entry point when the platform unexpectedly regains relevance
concepts/fido2-webauthn Named as the canonical phishing-resistant-MFA primitive — stub if not present
concepts/passkey-authentication The Fly.io fix at the end: "our Twitter access is Passkeys now" — second canonical wiki instance
concepts/sso-authentication Fly.io's Google SSO is the chokepoint; canonical instance of "everything behind an IdP that requires phishing-proof MFA"

Patterns extracted

Pattern What it names
patterns/phishing-resistant-mfa-behind-idp Everything internal is downstream of one IdP that enforces FIDO2/WebAuthn MFA; reduces integration from N×M (apps×authenticators) to N×1; single chokepoint for off-boarding and audit
patterns/password-manager-as-phishing-guardrail The browser-plugin's domain-matching check is a free origin-check — defeats autofill phishing. Failure mode: manual copy-paste bypasses it
patterns/incident-response-calibrated-to-blast-radius Scale response to what's actually at risk, not to the conspicuousness of the attack. Fly.io ran watchful-waiting because customer accounts weren't targeted and the misuse was low-leverage

Operational numbers & timeline

Metric Value Note
Detection latency ~45 seconds via email-of-record-change side channel
Password reset seconds self-serve, but moot because attacker rotated 2FA
Account recovery ~15 hours platform-gated (X.com support)
Attacker revenue (reported) ~$0 low-plausibility crypto airdrop scam
Brand-damage window 15+ hours
1Password-sharing scope employees + 1 contractor shared legacy-social credential store
Fly.io Twitter status afterward Passkeys enforced

Caveats

  • Self-reported incident narrative. No external attestation.
  • Short post, light on technical detail. The architectural claims are policy-level ("everything behind an IdP", "use FIDO2"), not implementation-level. No code, no diagrams, no specific IdP configuration shown.
  • Tier-3 source + first-party postmortem — inclusion is justified by the clean articulation of the origin-binding mechanism and the explicit naming of the "everything-behind-an-IdP" pattern, both of which are reusable across any org's security posture. Borderline case; notes flagged.
  • Phishing-training-doesn't-work is a strong-form claim cited to a single paper (Ho et al., OAKLAND 2025). Related literature is more nuanced — training measurably reduces click-through rates in the short term, but the effect decays, and the absolute rate is never zero. Fly.io's framing is directionally right but worth pairing with the phishing-training literature if citing.
  • "Everything behind an IdP" is a governance claim, not an implementation. The attack surface becomes "whatever you forgot to put behind the IdP", which is exactly what bit Fly.io. The pattern's weak point is coverage enforcement — the legacy-shared-account failure mode is the generalisable lesson.
  • Twitter/X.com's 15-hour recovery latency is characterised as "not outside industry norms" by Fly.io; no independent benchmark is given. Treat as anecdotal.
  • 1Password's browser-plugin-as-phishing-guardrail is a secondary control, not primary. It relies on the user autofilling rather than copy-pasting. FIDO2 on the authenticator side is the primary control.

Source

Last updated · 542 distilled / 1,571 read