Skip to content

FLYIO 2025-04-08 Tier 3

Read original ↗

Our Best Customers Are Now Robots

Fly.io developer-blog retrospective by Thomas Ptacek disclosing that over the past ~6 months the growth-driving users on the Fly.io platform are LLM-driven coding agents ("robots") running vibe-coding sessions, not the human CLI-first developers Fly.io built the platform for. Post is explicitly self-deprecating ("it's going to read like a brochure") but clears the Tier-3 bar on four load-bearing architectural disclosures: (1) the start vs create Machine lifecycle split named as the specific primitive robots consume, (2) the Fly Volume (filesystem) as the robot- iteration storage primitive where Postgres would be the human default, (3) MCP's long-lived SSE connections as the reason Fly's request-routing control is a robot attractant, and (4) tokenized secrets as the identity-plane robot ergonomic Fly is leaning into. Introduces Robot Experience (RX) as a product-design axis to sit alongside UX and DX.

Summary

Thomas Ptacek's thesis is that Fly.io's growth over the last six months has shifted from human developers to LLM-driven coding agents. He names four ways in which the Fly.io platform turned out to be "robot bait" without meaning to: (1) compute lifecycle — Fly Machines have a distinct start path (re-start an already- created Machine, lightning fast, not billed while stopped) separate from create (Docker-container-to-Machine boot, slower, billed). Vibe-coding sessions "stir up a frenzy of activity for a minute or so, but then chill out for minutes, hours, or days" — the stop → start cycle fits that shape exactly. Fly has "never been able to explain effectively to humans" this distinction, "but the robots are getting a lot of value out of it". (2) Storage — LLMs build Machines up "incrementally", adding packages, editing source code, installing systemd units after boot, which Fly's product thinkers would have told humans not to do. This means LLMs want a filesystem (Fly Volumes, "the one form of storage we sort of wish we hadn't done") and object storage (Tigris), not Postgres. (3) Networking — MCP is the robot interop protocol; modern MCP uses long-lived SSE connections; multitenant MCP servers need session affinity so connections hit the same stateful instance; Fly's dynamic request routing is therefore "possibly a robot attractant". (4) Identity / secrets — the robot pact is "automate all the drudgery in return for categorical and unwavering trust", which at the limit means "giving the robot access to Google Mail credentials". Fly pitches tokenized tokens as the architectural answer: the LLM gets a placeholder token that a hardware-isolated, robot-free Fly Machine can substitute for the real OAuth token at egress. Ends by introducing RX (Robot Experience) as a new product-design axis alongside UX and DX, and by naming managed Postgres (MPG) as the "most important engineering work happening today at Fly.io" — still DX, not RX — with an explicit "but more robots are coming."

Key takeaways

  1. Fly Machines have a distinct start primitive separate from create, and start is the fast path the robots consume. "There are two ways to start a Fly Machine: by creating it with a Docker container, or by starting it after it's already been created, and later stopped. Start is lightning fast; substantially faster than booting up even a non-virtualized K8s Pod. This is too subtle a distinction for humans, who (reasonably!) just mash the create button to boot apps up in Fly Machines. But the robots are getting a lot of value out of it." Canonical wiki datum for start-vs-create lifecycle and the start-fast-create-slow lifecycle pattern. (Source: sources/2025-04-08-flyio-our-best-customers-are-now-robots)

  2. Robot workloads are bursty-then-idle for hours, which start/stop fits but Lambda invocation lifecycle doesn't. "A vibe coding session generates code conversationally, which is to say that the robots stir up frenzy of activity for a minute or so, but then chill out for minutes, hours, or days. You can create a Fly Machine, do a bunch of stuff with it, and then stop it for 6 hours, during which time we're not billing you. Then, at whatever random time you decide, you can start it back up again, quickly enough that you can do it in response to an HTTP request." The comparison Fly draws is explicit — a Fly Machine is "Lambda-like" in start latency but "EC2-like" in how long it can run. Canonical shape for vibe coding workload as neither transaction nor batch. (Source: sources/2025-04-08-flyio-our-best-customers-are-now-robots)

  3. LLM workflows build Machines incrementally, making a filesystem the right storage abstraction — not Postgres. "A typical vibe coding session boots up a Fly Machine out of some minimal base image, and then, once running, adds packages, edits source code, and adds systemd units (robots understand systemd; it's how they're going to replace us). This is antithetical to normal container workflows, where all this kind of stuff is baked into an immutable static OCI container. But that's not how LLMs work: the whole process of building with an LLM is stateful trial-and-error iteration." And: "As product thinkers, our intuition about storage is 'just give people Postgres'. And that's the right answer, most of the time, for humans. But because LLMs are doing the Cursed and Defiled Root Chalice Dungeon version of app construction, what they really need is a filesystem, the one form of storage we sort of wish we hadn't done. That, and object storage." Canonical wiki datum for stateful incremental VM build as the robot storage-shape argument. (Source: sources/2025-04-08-flyio-our-best-customers-are-now-robots)

  4. MCP's long-lived SSE connections to multitenant servers require session affinity — which Fly's dynamic request routing already provides. "To interface with the outside world (because why not) LLMs all speak a protocol called MCP. MCP is what enables the robots to search the web, use a calculator, launch the missiles, shuffle a Spotify playlist, &c." And: "Complicating things somewhat, more recent MCP flows involve repeated and potentially long-lived (SSE) connections. To make this work in a multitenant environment, you want these connections to hit the same (stateful) instance. So we think it's possible that the control we give over request routing is a robot attractant." Canonical wiki datum for MCP long-lived SSE as a session-affinity requirement and session affinity for MCP SSE as the architectural pattern. (Source: sources/2025-04-08-flyio-our-best-customers-are-now-robots)

  5. Tokenized secrets decouple LLM access from permanent credential grant via a placeholder-for-real substitution at a hardware-isolated egress hop. "On a modern cloud platform, there's really no reason to permanently grant Sam Altman Google Mail access, even if you want his robots to sort your inbox. You can decouple access to your mail spool from persistent access to your account by tokenizing your OAuth tokens, so the LLM gets a placeholder token that a hardware-isolated, robot-free Fly Machine can substitute on the fly for a real one." Canonical wiki datum for tokenized secret and the tokenized-token broker pattern; also generalises beyond the robot use-case to any API-bridging integration service ("There are several big services that exist to knit different APIs together […] The big challenge about building these kinds of services is managing the secrets. Sealed and tokenized secrets solve that problem."). (Source: sources/2025-04-08-flyio-our-best-customers-are-now-robots)

  6. RX (Robot Experience) is proposed as a new product-design axis alongside UX and DX. "One of our north stars has always been nailing the DX of a public cloud. But the robots aren't going anywhere. It's time to start thinking about what it means to have a good RX. That's not as simple as just exposing every feature in an MCP server! We think the fundamentals of how the platform works are going to matter just as much. We have not yet nailed the RX; nobody has. But it's an interesting question." Canonical wiki datum for RX as product-design axis. DX is still the primary engineering focus (managed Postgres / MPG is "the most important engineering work happening today at Fly.io"), but the post is the first wiki instance of a Tier-3 cloud platform explicitly calling out RX as a separate product-shaping concern. (Source: sources/2025-04-08-flyio-our-best-customers-are-now-robots)

  7. The MCP-ing of Fly's own API is named as a planned extension. "For instance: it should be easy to MCP our API. The robots can then make their own infrastructure decisions." Not shipped at publication; forward-looking. But the architectural premise — that the Fly Machines API is already the right MCP surface for robots to self-provision compute — is the load-bearing assumption for all of the RX framing above. (Source: sources/2025-04-08-flyio-our-best-customers-are-now-robots)

  8. The underlying hypervisor is explicitly shared with AWS Lambda. "Not coincidentally, our underlying hypervisor engine is the same as Lambda's." Fly.io's positional framing relative to the Lambda–EC2 axis: "Like a Lambda invocation, a Fly Machine can start like it's spring-loaded, in double-digit millis. But unlike Lambda, it can stick around as long as you want it to: you can run a server, or a 36-hour batch job, just as easily in a Fly Machine as in an EC2 VM." Canonical wiki datum connecting Fly Machines to Lambda on the hypervisor axis while establishing the workload-duration contrast. (Source: sources/2025-04-08-flyio-our-best-customers-are-now-robots)

Systems extracted

  • systems/fly-machines — named as the compute primitive the robots consume; start vs create split, same hypervisor as AWS Lambda, double-digit-ms start latency, stick-around-as- long-as-you-want runtime. Underlying hypervisor is Firecracker (non-GPU Machines per the 2025-02-14 retrospective — "our underlying hypervisor engine is the same as Lambda's" makes Firecracker the specific substrate shared with Lambda).
  • systems/fly-volumes — named as the filesystem the robots want for stateful-incremental VM build; "the one form of storage we sort of wish we hadn't done." Self-described regret paired with robot-driven value.
  • systems/flyd — implied as the orchestrator delivering the start/stop/create primitives (not named in this post but the Machines API is flyd's RPC surface).
  • systems/fly-proxy — the dynamic-request-routing fabric Fly.io positions as a robot attractant for MCP SSE workloads.
  • systems/tigris — named as the object-storage half of the LLM storage answer; partnered storage tier alongside Fly Volumes for stateful-incremental-build workloads.
  • systems/tokenized-tokens — Fly.io's tokenized-tokens / sealed-secrets system (referenced to tokenized tokens, 2024). A hardware-isolated Fly Machine substitutes a real secret for a placeholder at egress time. First wiki mention of the system.
  • systems/model-context-protocol — named as the robot interop protocol with long-lived SSE connections; now a Fly.io-specific routing-attractant datum.
  • systems/aws-lambda — named as the share-the-hypervisor ancestor of Fly Machines on the start-latency axis; explicitly contrasted on the workload-duration axis.

Concepts extracted

  • concepts/robot-experience-rx — new product-design axis alongside UX and DX, introduced by name in this post.
  • concepts/fly-machine-start-vs-create — the lifecycle-split primitive at the heart of the post's compute claim.
  • concepts/vibe-coding — Fly's own gloss on what the robots are doing ("when those vectors can be interpreted as source code, we call this process 'vibe coding'").
  • concepts/mcp-long-lived-sse — MCP's session-affinity requirement; Fly's framing of it as a routing-attractant property of its platform.
  • concepts/tokenized-secret — placeholder-for-real secret substitution at a hardware-isolated hop; Fly.io names this as the architectural answer to the robot-credential-grant problem.
  • concepts/stateful-incremental-vm-build — LLMs build Machines up incrementally (packages, source edits, systemd units) after boot, unlike the immutable-OCI-container human default.
  • concepts/scale-to-zero — extended in this post to the minutes-to-hours-of-idle-between-vibe-coding-bursts shape (Machine stopped, not billed, start-ready on demand).
  • concepts/cold-start — the start path contrasted against create: "substantially faster than booting up even a non-virtualized K8s Pod."
  • concepts/fast-vm-boot-dx — the boot-latency property Fly.io positions as load-bearing on both the DX and RX axes (double-digit ms, Lambda-shared hypervisor).
  • concepts/agentic-development-loop — the workload shape the post describes is itself an agentic-loop consumer.

Patterns extracted

Operational numbers

  • Machine start latency: "double-digit millis" / "lightning fast" / "substantially faster than booting up even a non-virtualized K8s Pod" — same order of magnitude as Lambda invocation start, explicitly because Fly.io runs Lambda's hypervisor (Firecracker).
  • start-after-stop long-idle window: "stop it for 6 hours, during which time we're not billing you" — disclosed as a typical vibe-coding gap.
  • Observed growth-shift horizon: "over the last 6 months or so" — the time window in which robot usage became the dominant growth driver on Fly.io (post dated 2025-04-08, so roughly late 2024 → early 2025).

Caveats

  • Not an architecture paper. Opinion / product-framing post by Thomas Ptacek with the self-aware disclaimer that "a lot of this is going to read like a brochure." Load-bearing architectural disclosures are the start/create lifecycle split, the tokenized-tokens reference, and the MCP-SSE routing framing — everything else is positioning.
  • No usage numbers. The "best customers are now robots" headline is asserted but not quantified — no customer counts, revenue mix, Machine-hours, start call rates, or per-session Machine counts. Just "a funny thing has happened over the last 6 months or so. If you look at the numbers, DX might not matter that much. That's because the users driving the most growth on the platform aren't people at all. They're… robots."
  • Tokenized-tokens system deferred to the 2024 post. The substrate system Fly is pitching (tokenized OAuth secrets / sealed secrets) is not architecturally walked in this post — the reader is pointed to tokenized tokens (2024) for the design. This wiki's systems/tokenized-tokens page is a minimum-viable stub grounded in the 2025-04-08 reference — a future ingest of the 2024 tokenized-tokens post will flesh out the substrate.
  • MCP-ing the Fly API is forward-looking. "It should be easy to MCP our API." Not shipped at publication. The RX framing depends on the assumption that the Machines API is the right MCP surface, but Fly.io has not yet published an MCP server fronting it.
  • No robot-workload failure mode disclosed. The post is bullish on robot adoption. It does not discuss abuse patterns, runaway-loop cost protection, robot-driven quota incidents, or the operational tax of being on the receiving end of LLM-agent traffic.
  • RX is named, not defined. "We have not yet nailed the RX; nobody has." The post proposes RX as a design axis and gives a few data-point requirements (long-lived MCP SSE, tokenized secrets, start/stop, filesystem storage) but does not codify what an RX-scored cloud platform looks like.
  • Partial overlap with the 2025-02-14 GPU retrospective. The "developers don't want GPUs, they want LLMs" thesis (concepts/developers-want-llms-not-gpus) from two months earlier is consistent with — and foundational to — this post's robots-as-customers framing. This 2025-04-08 post is the platform-demand-side half of the retrospective started on 2025-02-14.

Source

Last updated · 200 distilled / 1,178 read