Skip to content

SYSTEM Cited by 5 sources

SQLite

SQLite is a widely-used embedded, file-backed relational database. Single-file database format, in-process library, no network layer, no separate server.

The wiki records it where it appears as a load-bearing substrate in ingested posts. At Fly.io's gateways SQLite is the local store for WireGuard peer state (pre- and post-JIT rewrite) and the rate-limit cache on the control-plane-lookup path under JIT peer provisioning:

"We keep a rate-limited cache in SQLite, and when we see new peers, we'll make an internal HTTP API request to fetch the matching peer information and install it." (Source: sources/2024-03-12-flyio-jit-wireguard-peers)

The wiki notes on SQLite at this kind of use — embedded local store, control-plane lookup cache, persistence substrate for a small-but-important set of per-host state — apply directly: single file, easy to snapshot, fast on local NVMe, O(ms) lookup, transactional.

Fly explicitly frames SQLite as a non-capacity-wall primitive in the WireGuard context:

"You could store every WireGuard peer everybody has ever used at Fly.io in a single SQLite database, easily. What you can't do is store them all in the Linux kernel." (Source: sources/2024-03-12-flyio-jit-wireguard-peers)

I.e. the kernel is the capacity wall; SQLite is happily below it.

Seen in

  • sources/2024-03-12-flyio-jit-wireguard-peers — as rate-limited lookup cache + local peer-state store on the gateway.
  • sources/2025-02-12-flyio-the-exit-interview-jp-phillips — two contrasting roles for SQLite at Fly.io surfaced in one source:
  • corrosion2 uses CRDT-structured SQLite as its read-side, fleet-queryable state-distribution store. Generic SQL queries from any consumer are the feature, not the risk.
  • flyd deliberately avoids SQLite and picks BoltDB — JP Phillips's defence is that the SQL surface is a foot-gun for an orchestrator's authoritative state (see concepts/bolt-vs-sqlite-storage-choice).
  • Hypothetical per-Fly-Machine SQLite is JP's "if I had to do it over" alternate design (see patterns/per-instance-embedded-database); the open problem is schema management across a fleet of per-instance databases.
  • sources/2025-03-27-flyio-operationalizing-macaroons — fourth SQLite role on the Fly.io wiki: root-of-trust store for the tkdb Macaroon token authority. SQLite + LiteFS + Litestream — the three-layer stack. "We use SQLite for a lot of our infrastructure, and this is one of the very few well-behaved databases we have." The closing line "a total victory for LiteFS, Litestream, and infrastructure SQLite. Which, after managing an infrastructure SQLite project that routinely ballooned to tens of gigabytes and occasionally threatened service outages, is lovely to see" is an implicit contrast to corrosion/corrosion2. tkdb's dozens-of-megs DB is the well-behaved counter-example — narrow schema, low write rate, small data, single-team ownership, blast-radius-controlled. Canonical wiki instance of patterns/sqlite-plus-litefs-plus-litestream.
  • sources/2025-05-20-flyio-litestream-revamped — fifth SQLite role on the wiki: SQLite's VFS extension surface as the read-replica integration point for revamped Litestream. The 2025-05-20 post is also the canonical wiki disclosure of the LTX file format (sorted per-transaction page-range files) that replaces raw-WAL shipping — formalizing SQLite's page-level state as a first-class replication substrate, not just a WAL byproduct. Positioning: SQLite as the agentic-coding PITR / fork primitive — "the robots that write LLM code are going to like SQLite too" — with Fly.io-owned tooling (Litestream + LiteFS) underneath.
  • sources/2025-10-02-flyio-litestream-v050-is-here — SQLite-ecosystem library disclosure. Litestream v0.5.0 switches from mattn/go-sqlite3 (CGO-dependent) to modernc.org/sqlite (pure-Go port). Ben Johnson frames it as "we've settled the age-old contest between mattn/go-sqlite3 and modernc.org/sqlite in favor of modernc.org" — a downstream-ecosystem signal that modernc.org/sqlite is considered production-grade for at least the write-log-tailing workload Litestream runs. First wiki instance of this library-pick; relevant for any Go service that wants SQLite without CGO (cross-compile from macOS to Linux Just Works, static binaries, smaller container images).
Last updated · 200 distilled / 1,178 read