Skip to content

SYSTEM Cited by 3 sources

Fly WireGuard mesh (6PN)

Fly.io runs an internal IPv6 WireGuard mesh — referred to internally as 6PN ("Six Private Network") — connecting every Fly Machine across hosts and regions inside a given org. Described in detail in the Fly post IPv6 + WireGuard = peering. Each Fly Machine gets an fdaa:… IPv6 address on the mesh the moment it boots.

Role in FKS

Under FKS, the WireGuard mesh is the direct replacement for the Kubernetes CNI layer. Per the primitive-mapping table in the FKS beta post:

Networking/CNI → Our internal WireGuard mesh connects your pods together

Observable side-effects in the K8s API:

  • ClusterIP Services default to IPv6 (IP Families: IPv6, IP Family Policy: SingleStack).
  • Endpoint addresses are fdaa:… WireGuard-mesh IPs, e.g. [fdaa:0:48c8:a7b:228:4b6d:6e20:2]:8080.
  • The fly machine list tooling reports the same fdaa:… IP on the Machine record.

This is a concrete instance of ipv6-service-mesh — service-to-service traffic rides an encrypted IPv6 underlay rather than the CNI+iptables substrate Kubernetes reference clusters use.

Seen in

  • sources/2024-03-07-flyio-fly-kubernetes-does-more-now — CNI substitute under FKS; cluster DNS and ClusterIP Services are IPv6-first because of this substrate.
  • sources/2024-03-12-flyio-jit-wireguard-peers — named as separate from the external customer-facing gateway mesh this post is actually about. The 6PN mesh (this page) is always-on, IPv6, Fly-Machine ↔ Fly-Machine, internal; the gateway mesh is transient, IPv4/IPv6, external flyctl ↔ gateway, per-CI-job — a different substrate with different scaling characteristics.
  • sources/2024-07-30-flyio-making-machines-move — 6PN as the transport for inter-worker iSCSI during stateful-Machine migration, AND the source of the migration's most-painful complication. 6PN addresses are themselves an instance of embedded-routing-header-as-address — "It functions by embedding routing information directly into IPv6 addresses. This is, perhaps, gross. But it allows us to route diverse private networks with constantly changing membership across a global fleet of servers without running a distributed routing protocol." The cost: "the embedded routing information in a 6PN address refers in part to specific worker servers" — migration requires a new 6PN address, DNS updates, and a in-init compatibility bridge for literal-address consumers like Fly Postgres cluster configs. Canonical embedded-routing-in-ip-address + hardcoded-literal-address-antipattern instance.
Last updated · 766 distilled / 2,225 read