Skip to content

CONCEPT Cited by 1 source

Packet sniffing as event source

Packet sniffing as event source is the move of manufacturing a control-plane event by observing the data plane — when the primitive you care about exposes traffic but not a subscription API for the event you want.

The shape

  1. The native API (kernel subsystem, SDK, whatever) exposes configuration RPCs and state queries, but no event stream for the "something just happened" you need to react to.
  2. The event you need to react to is reliably signaled by a specific packet on the wire — either inherently visible (cleartext header bit) or made visible by a hook your infrastructure controls (proxy, tunnel terminator).
  3. Attach a packet filter — BPF on a packet socket, eBPF on XDP / TC, a userspace hook in a controlled daemon — that matches only that packet, cheaply.
  4. Treat every filter match as the missing subscription event and drive the control plane from it.

Canonical instance — Fly.io JIT WireGuard

Linux WireGuard's Netlink configuration interface has no "handshake received" event. "Note that there's no API call to subscribe for 'incoming connection attempt' events. That's OK! We can just make our own events. WireGuard connection requests are packets, and they're easily identifiable, so we can efficiently snatch them with a BPF filter and a packet socket." (Source: sources/2024-03-12-flyio-jit-wireguard-peers)

Filter:

udp and dst port 51820 and udp[8] = 1

Three primitives: UDP transport, destination port = 51820 (WireGuard default), payload byte at offset 8 equal to 1 (the WireGuard handshake-initiation type byte). Every match is a missing-Netlink-event substitute. Fly.io drives their whole JIT peer provisioning pipeline from this.

WebSocket tunnel variant

When WireGuard rides on WebSockets (Fly.io's customer default for networks that can't talk end-to-end UDP), the raw UDP packets are no longer on the wire in the usual sense — but they are on the wire inside a daemon Fly.io owns. "We own the daemon code for that, and can just hook the packet receive function to snarf WireGuard packets." (Source: sources/2024-03-12-flyio-jit-wireguard-peers) Same event semantics, different hook point.

Why it's a legitimate design move (not a hack)

  • Cost is bounded. A filter on an already-flowing stream is O(bytes-per-second) overhead; matched packets are rare relative to transport data; the hot path doesn't change.
  • Event is ground-truth. The packet that triggers the event is the thing the control plane cares about. No "maybe a peer connected" ambiguity — the handshake is the connection.
  • Decouples from the primitive's API cadence. If the native primitive adds a subscription API later, you can move off sniffing; if it never does, sniffing keeps working.
  • Filters are cheap to specify. WireGuard's plaintext type-byte makes Fly.io's filter one-byte-wide. More generally, protocols with a small cleartext header surface are tractable.

When it doesn't work

  • Fully-encrypted packet type identification. If the thing you want to react to is distinguishable only after decryption (and you don't control decryption), you can't filter on it cheaply — at best you filter on encryption metadata.
  • No observable packet. Some events — connect() completes, peer goes idle — don't correspond to a specific packet on the wire.
  • Middlebox-transparent protocols. If the protocol is designed so middleboxes can't tell what's going on (some QUIC trust-boundary framing), the filter surface shrinks to almost nothing.

Relationship to concepts/in-kernel-filtering

Adjacent concept. In-kernel filtering uses eBPF to drop high-volume events before they leave the kernel — the point is suppression, the kernel is the fast path. Packet-sniffing as event source uses (e)BPF to detect rare events that would otherwise have no signal — the point is amplification of the missing signal into a control-plane action.

Both rely on the same underlying primitive (a BPF program on a data-plane hook) but sit at opposite ends of the filter design axis: suppress-noise vs. extract-signal-from-noise.

Seen in

Last updated · 200 distilled / 1,178 read