Skip to content

PlanetScale

PlanetScale Engineering blog is a Tier-3 source on the sysdesign-wiki. PlanetScale is a managed relational-database vendor originally built on top of Vitess (MySQL-protocol sharding substrate spun out of YouTube) and now offering both MySQL and Postgres products. The PlanetScale system page covers product scope, integrations, and positioning separately.

Tier classification

Tier 3 — PlanetScale's blog straddles marketing / developer-education / engineering deep-dive. A substantial fraction of posts are product-announcement / launch / pricing content that fails scope. A significant minority are genuine database-internals education written by authors like Ben Dicken (ex–Turso, ex–Pure Storage) and Sam Lambert (CEO, ex-GitHub) — these consistently clear the Tier-3 bar via detailed coverage of storage engines, index structures, query optimisation, contention patterns, and production war stories from previous employers.

Skip rules specific to PlanetScale

  • "Introducing…" / "Now available…" / pricing announcements — skip unless there is a substantive architecture section alongside the announcement.
  • Migration-guide marketing ("Moving from $competitor to PlanetScale") — skip unless backed by a concrete architectural narrative.
  • Generic cloud-database industry commentary — skip.
  • Dicken database-internals / Burlacu architecture / Lambert ex-GitHub war-story posts — default include. These are Tier-3-clearing by construction.

Recent articles

  • 2026-04-21 — Bring your data to PlanetScale (Phani Raju, originally 2021-11-17 per page timestamp, re-fetched 2026-04-21) — twentieth PlanetScale first-party ingest and earliest public architectural description of PlanetScale Database Imports on the wiki — ~4.5 years before Matt Lord's 2026-02-16 petabyte-scale deep-dive. Two canonical new wiki pages: (1) Vitess unmanaged tablet as the composition primitive under imports — "PlanetScale's data imports are powered by Vitess's Unmanaged Tablet technology." The 2026 post assumes this as background; the 2021 post is the canonical wiki attribution. A Vitess tablet that attaches to an externally-managed MySQL (process / storage / backups operated by someone else) and speaks the MySQL replication protocol to tail its binlogs — lets Vitess's workflow machinery (VReplication, MoveTables, routing rules) operate against foreign MySQL endpoints without requiring them to be rehomed into Vitess first. (2) Database-as-data-router as the earlier bidirectional-routing cutover shape that predates the atomic-swap SwitchTraffic design — during the validation phase, the customer points their application at PlanetScale, which terminates the wire protocol, serves reads locally, and transparently forwards writes back to the source over the replication channel. Writes flow back into PlanetScale via ongoing CDC. Author explicitly contrasts the pattern with dual writes: "This step of routing your application's reads and writes through PlanetScale allows you to safely validate that your application is fully compatible with PlanetScale without taking downtime and without fragile application-level migration mechanisms like dual writes." Cutover is "Enable primary mode" — a direction reversal, not an atomic swap. After cutover the source is kept in sync via reverse replication so rollback is another direction flip. Detach is the explicit tear-down step. Canonical three-stage lifecycle: validate (destination-as-proxy, writes forwarded to source) → primary mode (direction reversed, reverse-replication running) → detach (source disconnected, credentials deleted). Also names the pre-copy schema-compatibility gate that surfaces incompatibilities before any row copy: "We will notify you of any schema incompatibility or connection errors that might interfere with using PlanetScale." Six architectural axes: (1) unmanaged-tablet substrate; (2) binlog-enabled MySQL + read/write creds as prerequisites; (3) pre-copy schema-compatibility check; (4) snapshot + binlog-catch-up data motion; (5) database- as-data-router validation phase with writes proxied back to source + dual-write-is-inferior framing; (6) direction- reversal cutover + reverse-replication-for-rollback + detach lifecycle. Phani Raju becomes the third named 2021-era PlanetScale voice on the wiki (after the 2022-12 HN-thread linking and the main 2026-era voices). Cross-source continuity: extends the 2026-02-16 Matt Lord import deep-dive by canonicalising the substrate (unmanaged tablet) and the earlier product-UX shape (database-as-data-router) that the 2026 post takes as background. Both shapes share the underlying primitives (VReplication, routing rules, reverse replication, unmanaged tablet) but compose them into different cutover ceremonies — atomic SwitchTraffic vs operator-driven direction reversal. Also canonical first-party attribution that PlanetScale Imports has been in the product since
  • Scope: on-scope Tier-3 — Phani Raju product- launch voice but with architectural density ~60-70% of body (VReplication + unmanaged tablet + binlog replication

  • routing rules + reverse replication + detach lifecycle — every framed feature backed by an architectural primitive disclosure). Clears Tier-3 bar decisively on two orthogonal-to-existing-coverage axes: unmanaged-tablet substrate not previously canonicalised; bidirectional- routing cutover variant not previously canonicalised. Wiki pages created (3): sources/2026-04-21-planetscale-bring-your-data-to-planetscale, systems/vitess-unmanaged-tablet, patterns/database-as-data-router. Wiki pages extended: systems/vitess (new Seen-in entry + tags), systems/vitess-movetables (new Seen-in entry + tags + related), systems/planetscale (frontmatter + related), concepts/online-database-import (new Seen-in entry + tags + related), patterns/routing-rule-swap-cutover (new Seen-in entry framing sister-pattern relationship + tags + related).

  • 2026-04-21 — Benchmarking Postgres (Ben Dicken, originally 2025-07-01 per page timestamp, re-fetched 2026-04-21) — nineteenth PlanetScale first-party ingest and canonical wiki disclosure of PlanetScale's benchmarking methodology. Methodology / disclosure-voice post accompanying the 2025-07-01 PlanetScale for Postgres launch's "consistently outperform every Postgres product on the market" claim. Six architectural axes: (1) Canonicalises Telescope as PlanetScale's internal benchmarking harness — "We built an internal tool, 'Telescope', to be our go-to tool for creating, running, and assessing benchmarks." First canonical wiki instance of patterns/custom-benchmarking-harness at the multi- vendor-comparative-OLTP-benchmarking altitude. (2) Three benchmarks, three questions: latency (SELECT 1;), TPCC (Percona sysbench-tpcc at 500 GB, TABLES=20, SCALE=250), OLTP read-only (sysbench oltp_read_only at 300 GB). (3) Reference target: i8g M-320 (4 vCPU, 32 GB, 937 GB NVMe) with primary + 2 replicas across 3 AZs. (4) RAM:CPU-asymmetry rule: match RAM first, let CPU asymmetry cut in the vendor's favour when ratios differ — "we opted to match the RAM, giving them double the CPU count used by PlanetScale. This is an unfair advantage to them". (5) Availability-posture equalisation: competitor cost models include replicas to match PlanetScale's default 3-AZ posture. Canonical wiki instance of the new concepts/price-performance-ratio concept. (6) Reproducibility commitment: full configs, reproduction instructions at /benchmarks/instructions/tpcc500g and /benchmarks/instructions/oltp300g, feedback address benchmarks@planetscale.com. Methodology-voice acknowledgment of bias caps: AZ-placement not controllable across vendors; RAM:CPU asymmetry called out explicitly. Canonical wiki pages created (4): sources/2026-04-21-planetscale-benchmarking-postgres, systems/telescope-planetscale, concepts/price-performance-ratio, patterns/reproducible-benchmark-publication. Scope: on-scope Tier-3 — Ben Dicken database-vendor- methodology voice, architecture density ~80% of body (methodology + configs + accounting rules + bias acknowledgment), clears Tier-3 bar per "Dicken database-internals / Lambert ex-GitHub war-story posts — default include" skip-rule. The post is pedagogy-meets- methodology-disclosure, not a raw results post — companion to the 2025-10-14 Postgres 17 vs 18 post which supplies the actual comparative numbers.

  • 2026-04-21 — Behind the scenes: How schema reverts work (Holly Guevara + Shlomi Noach, originally 2022-03-24, re-fetched 2026-04-21) — eighteenth PlanetScale first-party ingest and canonical wiki walkthrough of PlanetScale's instant-schema-revert mechanism. The post is the complementary "emergency escape hatch" to the earlier expand-migrate-contract "discipline" framing: the disciplined approach avoids needing a revert; PlanetScale's mechanism rescues the team that didn't follow it. Architectural load-bearing claim: VReplication "does not terminate upon migration completion" — uniquely among online-DDL tools. At cut-over the shadow table + replication stream stay alive, the stream is re-primed in the inverse direction (new → old) so the old-schema table keeps up with every post-cut-over write, and a revert is a second freeze- point swap of two already-in-sync tables — not a data copy, not a backup restore. Canonical new concepts concepts/shadow-table, concepts/cutover-freeze-point, and concepts/pre-staged-inverse-replication and patterns patterns/shadow-table-online-schema-change

  • patterns/instant-schema-revert-via-inverse-replication originate here. Four-step online-DDL shape canonicalised: build empty shadow with new schema, apply DDL to shadow, backfill + track concurrent writes through binlog, cut over under a brief write lock — shared by pt-online-schema-change, gh-ost, and Vitess. Five VReplication design properties named as distinguishing factors: copy-and-changelog progress both tracked (not just backfill); per-transaction GTID mapping; GTID-driven interleaving; transactional sidecar-state coupling; non-termination after cut-over. Walked example (ALTER TABLE users DROP COLUMN title) makes the column-projection asymmetry explicit: post-cut-over rows ("Savannah") survive the revert but reappear with NULL in the restored column — "expected and something you can clean up after the revert, if necessary." Cross-source continuity: pairs with the 2026-02-16 zero-downtime-migrations post (same VReplication substrate, data-motion scale) and the patterns/reverse-replication-for-rollback pattern from it — one architectural principle ("keep the inverse replication alive past cut-over so nothing is a one-way door") at two scales (data-motion cutover + online DDL). Introduces Holly Guevara as a new named PlanetScale voice (joining Dicken, Lambert, Van Wiggeren, Martí, Lord, Gangal, Sigireddi, Hazen, Noach, Barnett, Gupta, Raju). Wiki pages created (6): source + 3 concepts + 2 patterns. Wiki pages extended (10): systems/vitess-vreplication, systems/vitess (eighth Vitess-internals axis: online-DDL + schema revert), systems/mysql, systems/planetscale, concepts/online-ddl, concepts/gtid-position, concepts/consistent-non-locking-snapshot, concepts/binlog-replication, patterns/snapshot-plus-catchup-replication, this companies/planetscale. Scope disposition: on-scope Tier-3 → clears. Architecture content is

    90% of the body; zero marketing content; retrospective-product post with deep internals. URL rule + H1 + wikilink compliance all observed.

  • 2026-04-21 — Behind the scenes: How we built Password Roles (Phani Raju, originally 2022-07-27, re-fetched 2026-04-21) — seventeenth PlanetScale first-party ingest and first canonical wiki disclosure of Vitess's authorization subsystem (seventh distinct Vitess-internals axis after expression eval, data motion, query routing / transactional writes, load admission / throttler, fork management, and the backup / observability tooling from Vitess 21). Architecturally the post is about side-stepping Vitess's native static-config table-ACL — a JSON configuration file loaded by vttablet at boot with table_groups declaring table_names_or_prefixes + readers / writers / admins user lists, reloaded from disk via --table-acl-config-reload-interval. The static-file shape targets deployments with a pre-defined user set, a small number of ACL files (one per cluster, not per tenant), and maintenance-scheduled updates — none of which apply to PlanetScale. Four concrete failure modes of pushing ACL state onto every vttablet pod: (a) refresh-interval latency vs "credentials work immediately" user story; (b) per-pod race conditions during rollout; (c) pod-restart has no authoritative source for ACL reconstruction; (d) per-customer state explosion. Architectural fix canonicalises three new wiki framings: (1) the concepts/external-credential-store concept — per-password 4-field record (display_name, role, password_sha1_hash, password_sha2_hash) stored in a dedicated DB outside the data plane; (2) concepts/on-the-fly-security-principal-rewrite — rewriting the authenticated identity at an ingress service before the request reaches the downstream static-authz check, so the downstream check can stay invariant across tenants; (3) concepts/table-group-acl — naming the Vitess-style three-role (readers / writers / admins) table-group authz shape as a concept in its own right. New systems: systems/vitess-table-acl (the static-config mechanism on vttablet) and systems/planetscale-user-query-frontend (PlanetScale's ingress service responsible "for all user-facing functionality for PlanetScale databases" — authenticates the password, looks up the role, rewrites the MySQL-wire principal to one of three synthetic Vitess usernames planetscale-reader / planetscale-writer / planetscale-admin). New pattern patterns/external-credential-store-with-principal-rewrite — the reusable shape for managed-multi-tenant deployments on top of any system with static-config authorization: keep the downstream authz file invariant across pods, store per-tenant credential + role state in an external DB, terminate client connections at an ingress that rewrites the authenticated principal into a small fixed synthetic set before forwarding to the data plane. Each of the four failure modes maps 1:1 to an architectural win: instant effect, pod-independent state, restart-safe, universal config. Wiki pages created (5): source + 2 systems + 3 concepts + 1 pattern. Wiki pages extended (3): systems/vitess (new Seen-in entry for the authz axis; frontmatter + Related extended with the 5 new-page links); systems/planetscale (new Seen-in entry; frontmatter + Related extended); companies/planetscale (this entry). Introduces Phani Raju as a new named PlanetScale voice on the wiki (joining Dicken / Lambert / Van Wiggeren / Martí / Lord / Gangal / Sigireddi / Hazen / Noach / Barnett / Guevara / Gupta). Scope disposition: on-scope Tier-3 → clears. 2022 product-launch-voice but architectural content ~90% of body — four-failure-modes enumeration is itself an architectural primitive on the static-vs-dynamic authz tradeoff, the credential-DB schema is a concrete artifact, and the frontend-rewrite mechanism is the core disclosure. No 2022-era precedent for this kind of canonical composition on the wiki. No production numbers (credential-DB QPS, frontend rewrite p99 latency, password count per database — all absent). URL rule compliance: source url: + body ## Source link use verbatim raw-frontmatter URL (https://planetscale.com/blog/behind-the-scenes-how-we-built-password-roles); raw-markdown link uses actual filename (2026-04-21-behind-the-scenes-how-we-built-password-roles-9cb6252d.md). Wikilink syntax compliance: body wikilinks use clean [[target]] / [[target|display]]; frontmatter uses bare array form.

  • 2026-04-21 — Backward compatible database changes (Taylor Barnett) — sixteenth PlanetScale first-party ingest and first canonical wiki walkthrough of the six-step expand-migrate-contract pattern (also known as "parallel change" or simply "backward compatible changes"). The post is deliberately pattern-first, PlanetScale-second — the six-step sequence (Expand schema → Dual-write → Backfill → Read-new → Stop-old- writes → Delete) applies to any relational database, and PlanetScale-specific machinery is presented as convenience. Substance across five orthogonal axes: (1) Five-reason structural argument against coupled schema-and-app deploys — doubled risk, no atomic cross-system deploy, migration time scaling from seconds to days, pipeline-blocking on schema failure, and forced backward-compat discipline — canonicalised as concepts/coupled-vs-decoupled-database-schema-app-deploy. (2) Additive vs mutating distinction — adding a column/table is just "deploy schema before app"; the full six-step dance is mandatory only when the change touches existing schema the production app is using. (3) Canonical six-step sequence canonicalised as patterns/expand-migrate-contract — at each step through Step 4, the application-code deploy is rollback- able because the old schema is still read-complete and write-complete (the property that makes the pattern low-risk). (4) Backfill-size threshold (qualitative) — one-shot script for small data, background jobs for large data to avoid IOPS contention / replication lag / lock contention; cutover-verification via Insights or equivalent APM. (5) MySQL-specific Step-6 primitive — invisible columns (MySQL 8.0) canonicalised as the deprecation-discovery tool that inverts the "who is still reading this column?" flow; columns that remain named-accessible but are silently hidden from SELECT * reveal un- migrated callers via runtime errors before the final drop. PlanetScale platform integration: branching + deploy requests + Insights + refusal of direct RENAME COLUMN DDL (implementing the pattern at the platform level) frame PlanetScale as a convenience substrate for the pattern rather than a unique enabler. Worked example (GitHub-stars tracker): consolidating a star table into repo by adding repo.star_count, dual-writing, backfilling, cutting reads, contracting, dropping star. Wiki pages created (4): sources/2026-04-21-planetscale-backward-compatible-database-changes, 2026-04-21-planetscale-behind-the-scenes-how-schema-reverts-work; 1 pattern (patterns/expand-migrate-contract); 2 concepts (concepts/coupled-vs-decoupled-database-schema-app-deploy, concepts/mysql-invisible-column). Wiki pages extended (6): systems/mysql (new Seen-in entry + frontmatter; MySQL as canonical substrate for expand-migrate-contract; MySQL 8.0 invisible-column first citation); systems/planetscale (new Seen-in entry with pattern-first, PlanetScale-second framing); concepts/backward-compatibility (new Seen-in entry canonicalising the schema-DML deploy-discipline variant of backward-compat, different from the GitHub-Issues / Cloudflare-DNS / Lyft-protobuf variants — not a long-lived contract with remote clients but a short-window coexistence contract between two systems within one team's deploy pipeline); concepts/schema-evolution (new third schema-evolution axis on the wiki alongside Datadog async-CDC + Lyft compile-time schema- design — PlanetScale deploy-time within-single-service coexistence of old + new schema through a sequenced migration); patterns/dual-write-migration (extends with the within-a-single-database specialisation — Step 2 of expand-migrate-contract is the classical dual- write pattern applied at the DDL altitude, sibling of Airbnb's cross-system protocol migration). Scope disposition: on-scope Tier-3 — pedagogical-voice post, borderline per the "pedagogy listicle" skip signal, but clears the Tier-3 bar decisively on five orthogonal axes. Per companies/planetscale.md skip-rules: "Dicken database-internals / Burlacu architecture / Lambert ex-GitHub war-story posts — default include" — Taylor Barnett's developer-education voice is the same genre at a different altitude (teaching pattern rather than product). Architectural density is ~95% of the body; every section advances the pattern walkthrough or five-reason argument. Cross-source continuity: first canonical expand-migrate-contract pattern on the wiki; extends the wiki's migration corpus across altitudes. Under concepts/backward-compatibility: third canonical PlanetScale-voice instance (alongside GitHub-Issues bookmarked-URLs, Cloudflare CNAME-A-record DNS wire-format, Lyft protobuf schema-design). Complements sources/2025-11-04-datadog-replication-redefined-multi-tenant-cdc-platform on the schema-evolution axis — Datadog framed async-CDC- pipeline schema evolution via offline validation + runtime registry; this post frames sync-app-plus-DB schema evolution via sequenced deploys + dual-state coexistence. Together they canonicalise two distinct altitudes of schema evolution on the wiki: the cross-service async-CDC altitude (Datadog) and the within-service sync-DDL altitude (PlanetScale). Complements sources/2024-09-16-lyft-protocol-buffer-design-principles-and-practices on a third dimension — Lyft operationalises schema- evolution at schema-design time (keep future changes additive by construction); PlanetScale operationalises schema-evolution at deploy time (sequence the rollout through dual-state coexistence). No existing-claim contradictions — strictly additive. Caveats: no production numbers (teaching post); backfill-size threshold is qualitative; concurrent-write semantics during dual-write are implicit-same-transaction; rollback window between Step 5 and Step 6 qualitatively described ("a few days"); pattern covers single-database / single-application scope only; mid-step failure recovery is implicit-idempotency-assumption; PlanetScale-specific machinery (branching, deploy requests, Insights, rename-warning) is presented as "convenience" not as load-bearing. URL rule compliance: raw file's url: field is https://planetscale.com/blog/backward-compatible-databases-changes (note the plural databases-changes, which is PlanetScale's chosen canonical URL and authoritative per AGENTS.md URL rule); source page url: + body ## Source section use verbatim. Raw-markdown link uses actual raw filename 2026-04-21-backward-compatible-database-changes-45e7d8aa.md (note raw filename uses singular database-changes — slug-normalisation divergence between feed and URL). Return contract: ingested: wiki/sources/2026-04-21-planetscale-backward-compatible-database-changes.md — 13 wiki pages touched.

  • 2026-04-21 — Automating cherry-picks between OSS and private forks (Manan Gupta) — fifteenth PlanetScale first-party ingest. Retrospective on how PlanetScale keeps their private Vitess fork aligned with the open-source Vitess project. Three-stage evolution driven by scale: (1) weekly GitHub Action cherry-picking the whole private diff onto OSS main; (2) git-replay, an internal batch tool that memoises how merge conflicts were resolved and replays them across release branches (concepts/conflict-resolution-memoization); (3) the Vitess cherry-pick bot — a stateful GitHub Actions cron + PlanetScale-DB bot that continuously mirrors OSS PRs into private upstream on an hourly schedule. Formalises the branch-pair mirror topology OSS main ↔ private upstream, OSS release-x.0 ↔ private latest-x.0 (concepts/fork-upstream-sync). Four canonical new wiki patterns: (1) automated upstream cherry-pick bot — cron + DB state + incremental pull with monotonic timestamp stopping criterion ("stops once it encounters a PR that predates any PR already present in the database"); (2) draft PR on conflict — bot "creates a PR even if conflicts arise during cherry-picking, ensuring no PRs are missed"; conflict PRs are draft + do not merge + Conflict labels + git status comment tagging the original author, turning pipeline-blocker into async task queue; (3) label-triggered backport — Backport to: latest-x.0 label on a private PR is the signal to backport to a specific release branch; author owns the decision, silent-omission failure mode; (4) weekly reconciliation check — two independent audits run weekly and post to a dedicated GitHub issue: upstream-in-sync-with-OSS (flags open cherry-pick PRs, OSS main PRs not cherry-picked, direct-merges to latest-x.0) and latest-branches-consistent (flags open backport PRs, direct merges into latest, PRs backported to latest-x.0 but not higher-numbered latest branches). Closing (concepts/weekly-integrity-reconciliation) shape: fast path does the work, slow path audits the invariants, DB as state makes both possible. Outcome: "Over a year and six months later, the results have been remarkable." Scope disposition: on-scope Tier-3 borderline — not a Vitess- internals post, but a concrete architectural retrospective on how a team runs scheduled-bot automation in production with external state, with crisp design decisions (stateful vs stateless, auto-cherry-pick vs label-triggered backport, memoised-resolution vs draft-PR escalation) and 18-month track record. Clears Tier-3 bar via architectural density + production retrospective framing.

  • 2026-04-21 — Announcing Vitess 21 (Vitess Engineering Team) — fifteenth PlanetScale first-party ingest and fifth canonical Vitess- internals disclosure on the wiki after evalengine (2025-04-05), VReplication/VDiff/MoveTables (2026-02-16), Consistent Lookup Vindex (2026-04-21), and the three-part Throttler series (2026-04-21). Release announcement for Vitess 21 and Vitess Operator 2.14.0. Five architectural shipping-points canonicalised: (1) Atomic distributed transactions reintroduced with deeper integration into VReplication + Online DDL + MoveTables

  • Reshard (the strong-atomicity endpoint of the cross- shard-write design space, complementing the same-day- shipping ordered-commit Consistent Lookup Vindex); (2) Experimental recursive CTEs in the evalengine SQL surface; (3) Reference-table materialization as a first-class Materialize primitive via VReplication — new canonical patterns/reference-table-materialization-via-vreplication pattern, push small lookup tables to every shard for shard-local joins; (4) Dynamic VReplication workflow config — runtime knobs moved off VTTablet process-flags onto the workflow control plane; (5) Multi-metric throttler shipping with v20 backward compat (removed in v22) + "throttler overrides the configuration when it requires heartbeat information" self-protective invariant. New Vitess subsystems canonicalised: mysqlshell logical-backup engine (contributed by Slack Engineering — first external-company contribution into Vitess's backup subsystem, materialised via new patterns/logical-backup-engine-plug-in pattern + new concepts/logical-vs-physical-backup concept); vexplain observability surface with vexplain trace (JSON execution trace) + vexplain keys (column-usage analysis, usable against standalone MySQL). Vitess Operator 2.14 adds VTGate HPA autoscaling, Kubernetes 1.31 support, per-keyspace Docker image selection. VTOrc gains an errant-GTID count metric as precursor-of-reparent visibility. VTAdmin dashboard for distributed transactions + VReplication workflow creation / monitoring. Online DDL improvements: ALTER VITESS_MIGRATION CLEANUP ALL, more INSTANT DDL scenario analysis, programmatic charset-conversion replacing CONVERT(... USING utf8mb4), schemadiff- library delegation of schema + diff analysis. Wiki pages created: 1 source + 3 systems (vitess-mysqlshell-backup, vitess-vexplain) + 5 concepts (atomic-distributed-transaction, recursive-cte, reference-table-materialization, dynamic-workflow- configuration, errant-gtid-tracking, logical-vs- physical-backup, online-ddl) + 2 patterns (reference- table-materialization-via-vreplication, logical-backup- engine-plug-in). Wiki pages extended: systems/vitess (new Seen-in framing v21 as fifth canonical Vitess- internals disclosure spanning multiple subsystems); systems/vitess-vreplication (new Seen-in on reference- table materialization + dynamic config); systems/vitess- throttler (new Seen-in on multi-metric shipping + heartbeat-override invariant); patterns/multi-metric- throttling (new Seen-in on production shipping). Scope disposition: on-scope Tier-3 — release- notes voice but architecturally dense (~60-70% of body advances specific Vitess architectural primitives with motivation + subsystem-interaction + compat-envelope framing). Per the wiki's PlanetScale skip rules, this release-notes post is the companion shipping datum to the four same-day architecturally-deep posts (Consistent Lookup Vindex + three-part Throttler series) — it's the menu, they're the ingredients.

  • 2026-04-21 — Anatomy of a Throttler, part 3 (Shlomi Noach) — fourteenth PlanetScale first-party ingest and closing instalment of the three-part Vitess-throttler series. Part 3 moves from the deployment-topology + self-cost axis of part 2 to the client-side axis: who is asking the throttler, why it matters, and how to differentiate between them. Six canonical new wiki framings: (1) client identity — Vitess's canonical 4-level hierarchical identifier <uuid>:<flow>:<subsystem>:<job-category> (e.g. d666bbfc_169e_11ef_b0b3_0a43f95f28a3:vcopier:vreplication:online-ddl) supporting both specific (this one job) and categorical (all online-DDL jobs) rules, plus observability slicing; (2) client starvation — failure mode where one unthrottled or low-rejection client pins the metric above threshold, continuously rejecting every other client for minutes to hours; three structural causes (rogue clients, exemption, differential metrics); (3) exemption — the risky-in-general prioritisation lever and its three justified cases (short transient starvation, incident- fix tasks, essential system components); (4) probabilistic rejection prioritisation — per-client rejection ratio applied as a dice roll independent of the metric check; prioritises without exemption because "both clients still play by the rules: none is given permission to act if the database has unhealthy metrics. It's just that one sometimes doesn't even get the chance to check those metrics"; (5) de-prioritise all except target — the operationally simpler dual framing (high global rejection ratio + zero on favoured identity) equivalent to per-client ratios but reducing rule-count; (6) time-bounded throttler rule — every exemption / prioritisation / de-prioritisation carries a TTL so incident-response and rush-hour adjustments can't become permanent stale policy. Rogue-client structural hole of the cooperative model — "A rogue client might neglect to connect to the throttler and just go ahead and send some massive workload" — closed only by a barrier / proxy-shaped throttler, instantiated by the new Vitess transaction throttler that sits in VTTablet's query-execution path and actively delays database queries under degradation; "Clients cannot bypass the throttler, and may not even be aware of its existence." Enforcement cost: client identification becomes an inference problem on SQL comments / connection attributes / session variables / auth scope rather than self-reported identity. The cooperative tablet throttler (parts 1-2) and the transaction throttler (part 3) coexist in production — cooperative for internal batch subsystems where identity is free, enforcement for OLTP application traffic where cooperation can't be assumed. Explicit differential-metrics-as-exemption insight: "While the second client throttles based on load average, the first client is effectively exempted from checking load average" — per-client metric sets are exemption in disguise and have the same starvation risk. Closing prescriptive claim: "Dynamic control of the throttler is absolutely critical, and the ability to prioritize or push back specific requests or jobs is essential in production systems." New pages (8): source

  • 3 concepts (concepts/throttler-client-identity, concepts/throttler-client-starvation, concepts/throttler-exemption) + 4 patterns (patterns/probabilistic-rejection-prioritization, patterns/deprioritize-all-except-target, patterns/time-bounded-throttler-rule, patterns/enforcement-throttler-proxy) + 1 system (systems/vitess-transaction-throttler). Extends systems/vitess-throttler (new client-identity + prioritisation-levers + relationship-to-transaction- throttler sections; Seen-in + Related + frontmatter extended) and concepts/database-throttler (new part-3 Seen-in entry + frontmatter extended with enforcement / client-identity / exemption axes). Tier-3 scope: on-scope uncontested — Vitess-internals content by a Vitess core maintainer; architectural density ~100% of body; no "Introducing" / "Announcing" / pricing content. Caveats: no production tuning numbers (specific rejection ratios, time-window defaults, rule-TTL policies not disclosed); no starvation-detection mechanism named; transaction- throttler internals (control loop, threshold logic, queue management, VTTablet connection-pool integration) described abstractly only; client-identification inference mechanisms under enforcement named as a category but not enumerated (SQLCommenter / session variables / auth scope / query shape plausible but unspecified); probabilistic-rejection pattern conflates short-circuit-metric vs additional-reject-on-green implementations. Series closes without a production- retrospective instalment — numbers from a running PlanetScale fleet are a future-ingest surface.

  • 2026-04-21 — Anatomy of a Throttler, part 2 (Shlomi Noach) — thirteenth PlanetScale first-party ingest and fourth canonical Vitess-internals axis on the wiki after expression evaluation (2025-04-05 Vicent Martí), data motion (2026-02-16 Matt Lord), and query routing + transactional writes (2026-04-21 Harshit Gangal + Deepthi Sigireddi). Shlomi Noach — Vitess maintainer, now at PlanetScale — opens the load- admission-control axis via a three-part series on the throttler. Part 2 covers deployment topology (singular vs distributed, per-AZ, active-passive, singular-plus-host-agent, fully-distributed per-vttablet) and self-cost (fail-open vs fail-closed HA semantics, busy-loop avoidance, free-pass windows, throttler hibernation). The Vitess tablet throttler is the canonical working example throughout: one throttler per vttablet mapping 1:1 to a MySQL server; shard-primary's throttler aggregates every replica throttler's metrics for shard-scope queries (max replication lag); no cross-shard throttler communication so fan-out stays bounded. Canonical new wiki framings for: (1) fail-open vs fail-closed — the client-side decision when the throttler is unreachable; the pragmatic bounded-wait-then-proceed compromise; (2) host-scope vs shard-scope metrics — "This introduces the concept of a metric's scope, which can be an entire shard or a specific host"; different workloads consult different scopes against the same throttler hierarchy; (3) throttler hibernation — slow or stop metric collection + heartbeat injection during idle periods; re-ignite on first client request; first few checks reject on stale data; client retry is the load-bearing compensation; (4) replication-lag heartbeats — canonical wiki source for the pt-heartbeat-style timestamp-injection technique that dominates MySQL lag measurement, and its binlog-volume cost ("It is not uncommon to see MySQL deployments where the total size of binary logs is larger than the actual data set"); (5) layered polling-interval staleness — agent-mediated throttler sees metrics up to 2 s stale (1 Hz agent

  • 1 Hz throttler) vs 1 s for direct access. New canonical patterns: patterns/singular-vs-distributed-throttler (the full topology design space); patterns/host-agent-metrics-api (per-host daemon exposes metrics over HTTP; consumer polls one API per host); patterns/throttler-per-shard-hierarchy (per-host + shard-primary rollup; Vitess tablet throttler as canonical instance); patterns/idle-state-throttler-hibernation (hibernate metric collection + heartbeat generation during idle; coordinated re-ignition across distributed peers on first-touch). Extends systems/vitess with a fourth non-storage subsystem disclosure; extends systems/mysql with canonical heartbeat + binlog-cost framing. Tier-3 scope: on-scope uncontested — Vitess-internals content by a Vitess core maintainer, architectural density ~100% of body; no "Introducing" / "Announcing" / pricing content. Caveats: no latency/throughput numbers for throttler itself; no comparative topology benchmarks; no PlanetScale-specific production numbers (vendor-educational tone); active-passive metric-collection-while-passive described but not evaluated; part 1 referenced but not ingested at this time; part 3 forward-referenced but not yet published. Client-retry is load-bearing throughout — free-pass windows after successful checks, cold-start windows after hibernation, bounded wait under throttler unavailable — canonical wiki framing of the throttler- client contract as an asymmetric cost share (throttler fails safe, client retries to compensate). Opens four new wiki pages (5 concepts + 4 patterns + 1 source) and extends 4 existing system/company pages.

  • 2026-04-21 — AI-Powered Postgres index suggestions (Rafer Hazen) — twelfth PlanetScale first-party ingest and first canonical wiki treatment of LLM-generated database changes + hypothetical-index evaluation. PlanetScale Insights now ships an AI-powered index suggestion surface on Postgres databases: the feature "monitors your database workload and periodically suggests new indexes to improve query execution speed and overall database performance." Two architectural pillars: (1) workload-aware LLM prompting — Insights' per-query-pattern telemetry narrows the candidate query set via three index-candidate filter gates (rows-read:rows-returned ratio, ≥ 0.1% of aggregated workload runtime, minimum execution count) before the LLM sees anything, so the LLM is never asked the untrustworthy "does anything need to change?" question; schema is narrowed to just the tables referenced by surviving queries; the LLM is asked to propose CREATE INDEX statements and cite which queries each index is designed to help; (2) LLM-plus-planner validation — every candidate passes (a) a syntactic parse of the CREATE INDEX statement and (b) a HypoPG + EXPLAIN comparison: register the candidate as a hypothetical index, re-cost each cited query, discard any candidate that doesn't produce a "substantial improvement" on at least one. Insights surfaces only the survivors, with the estimated query-cost reduction attached. Canonical wiki framing of two LLM failure modes: (i) LLMs almost always find an answer even when the right answer is nothing (mitigated by workload-telemetry pre-filter); (ii) LLMs produce plausibly-correct-but-wrong output (mitigated by planner-based validation). Establishes concepts/llm-generated-database-changes as the umbrella class of workflows this post is the canonical instance of. Strictly a Tier-3-clearing launch post per PlanetScale skip-rules — architectural content ~85% of body (validation pipeline + workload pre-filter + HypoPG integration + two-phase validation + prompt structure), not a pure pricing / feature-list announcement. No quantitative efficacy disclosed (no hit rate, acceptance rate, or workload-level before/after numbers) + no model/provider named + no adoption figures — but these are caveats on the measurement, not on the architectural substance. Adds one new system (systems/hypopg), three new concepts (concepts/hypothetical-index-evaluation, concepts/llm-generated-database-changes, concepts/index-candidate-filtering), and two new patterns (patterns/llm-plus-planner-validation, patterns/workload-aware-llm-prompting). Extends systems/planetscale-insights with AI index suggestions as a new Insights capability alongside query-performance telemetry and Traffic Control. Extends systems/postgresql with EXPLAIN + HypoPG as validation-oracle framing. Extends concepts/secondary-index with canonical production instance of the "write-amplification must be earned back by read-side wins" discipline.

  • 2026-04-21 — Achieving data consistency with the consistent lookup Vindex (Harshit Gangal + Deepthi Sigireddi) — eleventh PlanetScale first-party ingest and third canonical Vitess-internals disclosure on the wiki after the 2025-04-05 evalengine post (Vicent Martí) and the 2026-02-16 VReplication / VDiff / MoveTables post (Matt Lord). Canonicalises the Vitess query-routing + transactional-write layer: Vindexes map rows to keyspace_id shard addresses; the Consistent Lookup Vindex mechanism keeps user-facing query results consistent with the authoritative user-data table across a cross-shard schema without running 2PC on every DML. The mechanism: three MySQL connections per DML (Pre, Main, Post) committed in that order, rolled back in the same order on failure. Pre = lookup-table inserts

  • reclamation updates; Main = user-table DML; Post = lookup-table deletes. Canonical new patterns/ordered-commit-without-2pc pattern. The architectural invariant: "The lookup Vindex table may be inconsistent with the user table but the results returned for the query remained consistent with the user table." Weaker than 2PC (orphan lookup rows can exist) but sufficient because the data table is authoritative and user queries always filter on it. Canonical new concepts/orphan-lookup-row concept: lookup rows whose referenced user row no longer exists are lazily reclaimed on the next colliding unique insert via a SELECT ... FOR UPDATE on the orphan + SELECT ... FOR UPDATE on the (absent) user row + repoint UPDATE — all three under row-locks that prevent races. Identity-update no-op optimisation documented: without it, a Vindex-column update to its current value would have its Post-delete and Pre-insert deadlock against each other on the same lookup-table row. The two-separate-statements-in-one-transaction variant is a known limitation not fixed by the no-op. Tier-3 scope — tutorial-voice, no production numbers (no QPS, no latency, no orphan-rate), no formal specification of the isolation level user queries see, no 2PC-cost baseline. Extends the PlanetScale voice on the wiki from customer-facing product (Metal, Postgres launch, Database Imports) into the sharded-database internals axis: paired with the 2025-04-05 evalengine post and 2026-02-16 migration post, these three canonicalise query-routing + transactional writes (this post), data motion (2026-02-16), and expression evaluation (2025-04-05) as Vitess's three non-storage subsystems.

  • 2026-04-11 — Keeping a Postgres queue healthy (Simeon Griggs) — tenth PlanetScale first-party ingest and first canonical wiki deep-dive on MVCC horizon / vacuum / queue-table dynamics. Griggs reproduces Brandur Leach's 2015 que-degradation-test on Postgres 18 (PS-5 cluster, $5/mo) and shows: the failure mode is unchanged. FOR UPDATE SKIP LOCKED + batch-claims + Postgres 14+ B-tree bottom-up deletion lift the floor (modern Postgres survives much longer than the 2015 15-min death spiral) but do not remove the ceiling — dead tuples still accumulate linearly whenever long or continuously-overlapping transactions pin the MVCC horizon faster than autovacuum can reclaim. Lock time rose from 2–3 ms baseline to 84.5 ms spikes at 33k dead tuples (CTE) and 180 ms at 24k (SKIP LOCKED) on the paired benchmarks — curves "almost identical" because both approaches scan the same B-tree and encounter the same dead tuples. Headline new wiki vocabulary: concepts/mvcc-horizon (oldest-active-transaction cutoff below which vacuum can reclaim; continuously-overlapping short queries can pin it indefinitely with no upstream timeout catching it), concepts/dead-tuples-vacuum (heap + index-entry accumulation + B-tree walk-over cost on index scans), concepts/postgres-autovacuum (necessary not sufficient; can have nothing to reclaim when horizon is pinned), concepts/select-for-update-skip-locked (modern claim-API; doesn't fix horizon pinning), concepts/postgres-queue-table (canonical workload shape: transient rows, constant size, enormous cumulative throughput), concepts/write-churn (high-mutation at steady-state size = worst case for MVCC). Canonical structural fix: patterns/workload-class-resource-budget — tag queries by class (SQLCommenter action=analytics) and cap concurrent workers for low-priority classes so vacuum windows open up. PlanetScale's Traffic Control (feature of Insights extension; 3 dials — server-share+burst, per-query limit, max concurrent workers) is the canonical wiki instance. Measured efficacy: 800 jobs/sec + 3 overlapping 120-sec analytics queries, 15-min run. Traffic Control disabled = 155,000-job backlog, 300+ ms lock time, 383,000 dead tuples, VACUUM blocked, death spiral. Traffic Control enabled (analytics cap = 1 worker, 25% of max_worker_processes) = 0 backlog, 2 ms lock time, dead tuples cycling 0–23k, VACUUM running normally, 15 analytics queries completed, completely stable. Key structural claim the post establishes: upstream Postgres timeouts (statement_timeout PG 7.3+, idle_in_transaction_session_timeout PG 9.6+, transaction_timeout PG 17+) target single-query duration and cannot limit workload-class concurrency — a class-of-mechanism Postgres doesn't have. Also establishes the "quiet failure mode" framing: the dangerous version isn't a dramatic crash but a quietly degraded equilibrium where lock times creep up, jobs slow down, and no alert fires; in a "just use Postgres" world with mixed workloads this is the normal operating condition. Extends concepts/postgres-mvcc-hot-updates with a third orthogonal axis (horizon management, beyond HOT write-amplification and CSN/ProcArray visibility). Opens systems/planetscale-insights + systems/planetscale-traffic-control as the first canonical Insights / Traffic Control pages on the wiki.

  • 2026-02-16 — Zero downtime migrations at petabyte scale (Matt Lord) — ninth PlanetScale first-party ingest and first canonical wiki deep-dive on Vitess's data-motion subsystems. Matt Lord (Vitess core maintainer) walks the entire mechanism underlying PlanetScale's Database Imports feature — the thing PlanetScale customers use to move terabyte-to-petabyte databases from external MySQL into PlanetScale without downtime. Substance across six orthogonal axes: (1) canonical introduction of VReplication — per-stream per-target-shard data motion with consistent non-locking snapshots (LOCK TABLES READ → START TRANSACTION WITH CONSISTENT SNAPSHOT → @@global.GTID_EXECUTED → unlock), PK-ordered row streaming exploiting the clustered index to avoid filesort, interleaved copy-plus- binlog-replication catch-up cycles to stay inside the source's binlog retention horizon, continuous replication via COM_BINLOG_DUMP_GTID with per-stream GTID progress persisted in sidecar tables for restart- from-exact-checkpoint semantics; (2) canonical introduction of VDiff — zero-downtime consistency verification via source + per- target-shard consistent snapshots at matching GTID positions + concurrent full-table scans on REPLICA tablets; resumable across weeks of cutover-prep horizons; (3) canonical introduction of MoveTables — the user- facing workflow wrapper around VReplication + VDiff + SwitchTraffic cutover + ReverseTraffic rollback + Complete lifecycle; (4) canonical patterns/routing-rule-swap-cutover instance — the SwitchTraffic sequence: pre-check → lock keyspaces + workflow → stop source writes → buffer queries at VTGate → wait for full catch-up → create reverse workflow → initialise Vitess Sequences for sharded targets → allow target writes → atomically update schema routing rules → release buffered queries → start reverse workflow → freeze original → release locks. Typical cutover "less than 1 second." (5) canonical patterns/reverse-replication-for-rollback instance — reverse VReplication workflow created at cutover + kept running until Complete so cutover is reversible without data loss; "can be done back and forth as many times as necessary." (6) canonical concepts/fault-tolerant-long-running-workflow statement — fault tolerance is a correctness requirement (not a nice-to-have) for workflows whose duration approaches fleet MTBF: "at a certain scale where things can take many hours, days, or even weeks to complete and the likelihood of encountering some type of error … becomes increasingly likely." Also canonicalises patterns/read-replica-as-migration-source (run the migration against a standard MySQL replica so the production primary sees zero load) and concepts/query-buffering-cutover (VTGate query buffering at cutover — "the last part that allows the entire migration to be done without any downtime"). Extends concepts/online-database-import — prior canonical instance (2025-07-01 PlanetScale for Postgres) announced the capability for Postgres (v13+ imports, mechanism undisclosed); this post reveals the mechanism in full on the MySQL side. New wiki pages: sources/2026-02-16-planetscale-zero-downtime-migrations-at-petabyte-scale; 3 systems (systems/vitess-vreplication, systems/vitess-vdiff, systems/vitess-movetables); 6 concepts (concepts/consistent-non-locking-snapshot, concepts/gtid-position, concepts/binlog-replication, concepts/query-buffering-cutover, concepts/reverse-replication-workflow, concepts/schema-routing-rules, concepts/fault-tolerant-long-running-workflow); 5 patterns (patterns/snapshot-plus-catchup-replication, patterns/vdiff-verify-before-cutover, patterns/reverse-replication-for-rollback, patterns/routing-rule-swap-cutover, patterns/read-replica-as-migration-source).

  • 2025-10-14 — Benchmarking Postgres 17 vs 18 (Ben Dicken) — eighth PlanetScale first-party ingest and first canonical Postgres 18 / io_method benchmark post on the wiki. Ben Dicken's third wiki ingest after 2024-09-09 B-trees and 2025-03-13 IO-devices; methodology post implicitly referenced by the 2025-07-01 PlanetScale-for-Postgres launch's benchmark claim. Substance across six orthogonal axes: (1) canonicalises Postgres 18's io_method feature with all three modes (sync / worker / io_uring) and their trade-offs explained; (2) counter-intuitive headline — io_uring does not dominate: on network-attached storage at low concurrency, Postgres 18 sync and worker beat Postgres 17 and Postgres 18 io_uring, and io_uring only slightly wins at 50 connections + large range scans on local NVMe; (3) canonicalises concepts/async-io-concurrency-threshold as the phenomenon behind the headline — async I/O only pays off above a concurrency threshold that varies by storage latency floor; (4) canonicalises patterns/background-worker-pool-for-async-io as the deliberate design choice behind Postgres 18's io_method=worker default — process-level parallelism distributes both I/O dispatch and CPU-bound post-I/O work (checksums, memcpy) across worker processes where io_uring keeps it all in the caller; (5) empirical backing for Metal — the i7i.2xlarge with local NVMe wins every benchmark scenario tested, and the $551.15/mo price beats io2-16k's $1,513.82/mo; (6) benchmark methodology — 96 runs (4 Postgres configs × 4 instance types × 2 range-sizes × 3 concurrency levels × 5 min each) with 10-min pre-warm, ~300 GB database vs 64 GB RAM — canonical wiki instance of systematic vendor benchmarking. Cites Tomas Vondra's tuning blog for the three architectural reasons worker beats io_uring (index scans don't use AIO; checksums/memcpy remain CPU-bound; workers gives process-level parallelism). New wiki pages: sources/2025-10-14-planetscale-benchmarking-postgres-17-vs-18; 2 systems (systems/linux-io-uring, systems/sysbench); 2 concepts (concepts/postgres-async-io, concepts/async-io-concurrency-threshold); 1 pattern (patterns/background-worker-pool-for-async-io).

  • 2025-07-08 — Caching (Ben Dicken) — seventh PlanetScale first-party ingest and first canonical caching primer on the wiki covering the full stack from CPU cache to CDN to database buffer pool. Pedagogical deep-dive from Ben Dicken's database-internals voice on caching as "the most elegant, powerful, and pervasive innovation in computing" — the one idea appearing at every tier: pair a small amount of expensive fast storage with a large amount of cheap slow storage, keeping frequently-accessed data in the fast tier. Canonical wiki introduction of the foundational caching pattern across its full hierarchy of applications. Substance across six orthogonal axes: (1) hit- rate as load-bearing metric — formula + cache-size effect + economic intuition; (2) [[concepts/cpu-cache- hierarchy|CPU cache hierarchy]] canonicalised (L1 → L2 → L3 → RAM, size-vs-speed trade-off, "faster lookup means more cost or size limitations due to how physically close the data needs to be to the requester"); (3) temporal locality / recency bias canonicalised via X.com timeline worked example with the Karpathy tweet trendline ("after a few days, are rarely viewed") — first canonical wiki statement that "these websites store much of their content in 'slow' storage (like Amazon S3 or similar), but cache recent content in faster, in-memory stores (like CloudFront, Redis or Memcached)"; (4) spatial locality + spatial-prefetch-on-access canonicalised via the photo-album worked example ("when one photo is loaded, we can predict which ones we think they will want to see next, and prefetch those into the cache as well"); (5) CDN edge cache over central origin canonicalised with concrete latency stratification (east-coast ~10-20 ms, west-coast ~50-100 ms, across-the-world 250+ ms) — "we live on a big spinning rock 25,000 miles in circumference, and we are limited by 'physics' for how fast data can move from point A to B"; (6) four cache- replacement-policy pages introduced — FIFO, LRU, time-aware LRU, LFRU (dual-queue) — with Dicken's explicit ranking: FIFO is simplest but "isn't optimal for most caching scenarios because it doesn't consider usage patterns," LRU is "the industry standard for many caching systems … aligns well with temporal locality in real-world data access patterns." Closing database-caching section canonicalises Postgres's two-layer stack (shared_buffers on top of OS page cache, canonical 25%-of-RAM rule) and positions the MySQL InnoDB buffer pool as the single-layer counterpart. Explicit acknowledgement of out-of-scope axes: "We completely avoided the subject of handling writes and updates in caching systems… We didn't address consistency issues, sharded caches" — wiki already has those on other source pages (concepts/write-through-cache, concepts/invalidation-based-cache, concepts/cache-ttl-staleness-dilemma). Borderline- case ingest that passes the Tier-3 bar decisively per skip rules (Dicken database-internals post = default include) despite its announcement-voice framing — architectural density ~90% of body, every section advances a reusable cache primitive with worked example + trade-off + generalisation.

  • 2025-07-01 — Announcing PlanetScale for Postgres (Sam Lambert, CEO). Fifth PlanetScale first-party ingest and first canonical PlanetScale-for-Postgres post on the wiki. Private-preview launch of PlanetScale for Postgres — real Postgres v17 under a proprietary operator (not a fork), with a proprietary proxy layer incorporating PgBouncer for connection pooling + query buffering for automatic failover. Metal direct- attached-NVMe architecture extended from MySQL to Postgres: "PlanetScale Metal's locally-attached NVMe SSD drives fundamentally change the performance/cost ratio for hosting relational databases in the cloud. We're excited to bring this performance to Postgres." Online imports from any Postgres v13+ + automatic Postgres version updates without downtime. Benchmark claim: "consistently outperform every Postgres product on the market, even when giving the competition 2x the resources" (numbers deferred to separate methodology post). 50-customer pain survey frames the market — "regular outages, poor performance, and high cost" identical across existing-Postgres-vendor set (same three axes Metal-for-MySQL targeted). Vitess explicitly rejected for Postgres — "Vitess' achievements are enabled by leveraging MySQL's strengths and engineering around its weaknesses. To achieve Vitess' power for Postgres we are architecting from first principles" — canonical wiki introduction of patterns/architect-sharding-from-first-principles-per-engine via PlanetScale's new Neki system (neki.dev, waitlist-only). Convex named as launch-window production customer migrating its reactive database infrastructure. Positions PlanetScale for Postgres structurally against RDS / Aurora Limitless / Aurora DSQL / Lakebase as the real-Postgres-on-direct-attached-NVMe shape — distinct from extend-via-extensions (DSQL) and compute-storage-separation (Lakebase / Neon).

  • 2025-04-05 — Faster interpreters in Go: Catching up with C++ (Vicent Martí) — fifth PlanetScale first-party ingest and first canonical Go-interpreter / bytecode-VM / static-type-specialization / VM-deoptimization post on the wiki. Retrospective on rewriting Vitess's SQL expression evaluation engine (evalengine) from an AST interpreter into a bytecode-less virtual machine — compile each SQL sub-expression's AST into a []func(*VirtualMachine) int slice of Go closures (concepts/callback-slice-interpreter, patterns/callback-slice-vm-go), driven by planner-derived static type specialization from MySQL's information schema. Rejected alternatives with explicit reasoning: the big-switch VM loop (Go's compiler often lowers switches to binary- search dispatch rather than jump tables); the tail-call continuation loop (Go's compiler doesn't reliably emit tail calls); Brunthaler quickening (Vitess's semantic analyzer makes runtime type observation unnecessary); JIT ([[concepts/instruction- dispatch-cost|dispatch overhead]] is <20% of SQL-expression runtime, so JIT is "needlessly complex dead optimization" for this workload). Retains the AST interpreter permanently as deoptimization fallback (for value-dependent type promotions like -BIGINT_MIN → DECIMAL) + one-shot evaluator (constant folding in the planner) + fuzz-oracle sibling — Vitess fuzzes its AST interpreter, its new VM, and MySQL's own C++ engine against each other and has upstreamed multiple bugs to MySQL itself (collation PR 602, insert function PR 517, substring PR 515). Benchmark results: geomean −48.60% sec/op vs the original Vitess AST interpreter (MySQL C++ −43.58% on the same baseline), VM faster than MySQL on 4/5 benchmarks, zero memory allocations on 4/5 benchmarks. First canonical wiki statement of the dispatch-overhead-share threshold (JIT when >30%, stay in VM when <20%) as the decision rule for interpreter architecture investment. Complements the existing PlanetScale first-party corpus with the compiler/interpreter voice — distinct from the storage- engine-internals voice (Dicken B-trees, Lambert slotted counter), vector-architecture voice (Vitess + SPFresh + InnoDB), and hardware-platform voice (Van Wiggeren's Metal

  • EBS reliability posts).

  • 2025-03-18 — The Real Failure Rate of EBS (Nick Van Wiggeren) — fleet-scale operational retrospective on EBS as the reliability companion to the 2025-03-13 latency post (both published against the Metal launch window). The gp3 SLO guarantees only "at least 90% of provisioned IOPS 99% of the time" — 14 min/day or 86 h/year of potential degraded operation; "slow is often as bad as failed" for OLTP; PlanetScale's back-of-envelope arithmetic on a 256-shard × 3-replica fleet (768 gp3 volumes) → 99.65% chance of at least one active impacting event at any moment; io2 not immune ("roughly one third of the time in any given year" on the same fleet); correlated- AZ-failure observed on io2 too, contradicting the naive "replicas on different volumes in the same AZ → fate independence" assumption. PlanetScale's customer-side mitigation stack on EBS: [[patterns/automated-volume- health-monitoring|automated volume-health monitoring]] (read/write latency + idle % + synthetic write-file smoke test) + zero-downtime reparent in seconds to a healthy cluster node + automated replacement-volume provisioning — clamps the impact window but cannot eliminate events. Structural fix: Metal — shared-nothing on direct-attached NVMe + cluster replication for durability. Canonical wiki instance of concepts/slow-is-failure + concepts/performance-variance-degradation + concepts/blast-radius-multiplier-at-fleet-scale + concepts/correlated-ebs-failure + concepts/partial-failure. Fourth PlanetScale first- party ingest.

  • 2025-03-13 — IO devices and latency (Ben Dicken) — pedagogical history of storage media (tape → HDD → SSD → NVMe → cloud-era network-attached) framed as a sequence of latency step-changes, published to announce PlanetScale Metal. Core numbers: RAM ~100 ns; local NVMe ~50 μs; network-attached SSD (EBS) ~250 μs — a 5× regression the cloud-database industry accepted for elasticity + durability. Metal inverts the trade: direct-attached NVMe + primary-and-two-replicas replication + frequent backups + automated node replacement restores local-NVMe latency while closing the durability gap that drove the industry to network-attached storage in the first place. Canonical wiki introduction of concepts/storage-latency-hierarchy, concepts/nand-flash-page-block-erasure (targets / blocks / pages with per-page read-write + per-block erase), concepts/ssd-parallelism-via-targets (dedicated lines per target, host data layout determines parallelism extraction), concepts/ssd-garbage-collection (dirty-page reclamation copies live pages + erases block — hidden tail-latency tax), concepts/network-attached-storage-latency-penalty (5× hop the cloud default pays), concepts/iops-throttle-network-storage (GP3 default 3,000 IOPS cap, GP2 burst bucket, direct NVMe uncapped), concepts/storage-replication-for-durability (1% × 1% × 1% = 0.0001% 3-replica math), and concepts/tape-storage-sequential-access (tape as archive tier — CERN's 400 PB + AWS Storage Gateway VTL). New system: systems/nvme-ssd (PCIe-native SSD interface) + systems/planetscale-metal (the product embodiment). New pattern: patterns/direct-attached-nvme-with-replication.

  • 2024-10-22 — Announcing the PlanetScale vectors public beta — PlanetScale announces vector search + storage inside its MySQL-fork product. Vector columns + indexes live inside InnoDB as first-class durable structures, via a transactional extension of SPFresh (continuously-updatable descendant of SPANN). Full SQL (JOIN, WHERE, subqueries), ACID, pre-/post-filtering, Vitess- sharded terabyte scale. Rejects HNSW (RAM-bound, rebuild-required) and DiskANN (worse latency, incremental updates hard to map to transactional SQL) on structural grounds. Canonical wiki instance of patterns/vector-index-inside-storage-engine

  • patterns/hybrid-tree-graph-ann-index.

  • 2025-02-04 — The Slotted Counter Pattern (Sam Lambert, originally July 2022) — canonicalises the slotted counter pattern: replace one hot counter row with N slot rows keyed by (record_type, record_id, slot), pick RAND() * 100 per write, SUM(count) on read. Introduces row-level lock contention (InnoDB lock_mode X locks rec but not gap with SHOW ENGINE INNODB STATUS\G diagnostic snippet) and the hot-row problem as canonical wiki vocabulary. Canonical deployment: GitHub's github.downloads counter — the diagnostic's index PRIMARY of table github.downloads is the telltale. Complements the B-trees post on the same axis (InnoDB storage-engine internals from PlanetScale's MySQL-expert voice).

  • 2024-09-09 — B-trees and database indexes (Ben Dicken) — pedagogical deep-dive on B-trees / B+trees as the foundation of InnoDB tables + indexes + why sequential primary keys beat random UUIDs on three axes (insert-path locality, leaf write amplification, range-scan I/O shape).

  • systems/planetscale — PlanetScale as a managed relational DB product (MySQL + Postgres).
  • systems/planetscale-for-postgres — the Postgres product (private preview 2025-07-01, later GA); real Postgres v17 under a proprietary operator with a proprietary proxy layer + PgBouncer + query buffering + online imports from v13+.
  • systems/planetscale-metal — direct-attached NVMe + replication product tier (March 2025); canonical wiki instance of patterns/direct-attached-nvme-with-replication; engine-agnostic at the cluster-shape layer (MySQL in March 2025 + Postgres from July 2025).
  • systems/neki — PlanetScale's upcoming horizontal- sharding system for Postgres (neki.dev, waitlist-only); canonical wiki instance of patterns/architect-sharding-from-first-principles-per-engine — architected from first principles rather than porting Vitess, because "Vitess' achievements are enabled by leveraging MySQL's strengths and engineering around its weaknesses."
  • systems/pgbouncer — open-source Postgres connection pooler integrated inside PlanetScale's proprietary proxy layer for PlanetScale for Postgres.
  • systems/convex — reactive-database / BaaS named as launch-window production customer migrating to PlanetScale for Postgres.
  • systems/nvme-ssd — storage medium underneath Metal; canonical NVMe page added by 2025-03-13 ingest.
  • systems/vitess — sharding substrate under PlanetScale MySQL; composes with transactional SPFresh for sharded vector indexes and with Metal's direct-NVMe tablets. Explicitly NOT the Postgres sharding solution (see Neki).
  • systems/vitess-evalengine — Vitess's SQL expression evaluation engine in vtgate; canonical wiki instance of a Go VM catching up with C++ via bytecode-less callback-slice design + static type specialization (2025-04-05 Martí post).
  • systems/innodb — MySQL storage engine anchored by Dicken's 2024 B-tree post + Lambert's 2022 slotted-counter post (InnoDB lock-contention diagnosis); also hosts PlanetScale's transactional SPFresh vector index (2024-10-22).
  • systems/mysql / systems/postgresql — the two engines PlanetScale hosts.
  • systems/aws-ebs — named counterweight to Metal; Dicken's 2025-03-13 post frames EBS's 250 μs round-trip + 3,000 IOPS default cap as the architectural problem Metal solves.
  • systems/spann — hybrid tree + graph, SSD-resident ANN algorithm PlanetScale builds on.
  • systems/spfresh — continuously-updatable extension of SPANN; PlanetScale extends it further with transactional semantics.
  • systems/hnsw / systems/diskann — rejected structural alternatives to SPFresh, with PlanetScale's explicit reasoning on each.
  • systems/github — deployment context for the slotted counter pattern (several PlanetScale founders, including Sam Lambert, previously worked at GitHub).

Key concepts introduced

Key patterns introduced

  • patterns/sequential-primary-key — canonical B+tree locality pattern.
  • patterns/slotted-counter-pattern — split one hot counter row into N slots; GitHub's github.downloads workload is the canonical deployment.
  • patterns/vector-index-inside-storage-engine — put the ANN index inside the durable engine; PlanetScale transactional SPFresh inside InnoDB is the canonical wiki instance.
  • patterns/hybrid-tree-graph-ann-index — the algorithmic family (SPANN + SPFresh) that makes the storage-engine- hosted shape viable.
  • patterns/direct-attached-nvme-with-replication — primary
  • 2 replicas on local NVMe + auto-failover + frequent backups; the Metal architecture in pattern form. Inverts the cloud-database default of network-attached storage.
  • patterns/shared-nothing-storage-topology — each node owns its own storage; no shared fabric. The structural answer (2025-03-18 post) to EBS fleet-scale correlated-AZ-failure.
  • patterns/automated-volume-health-monitoring — in-app heuristic monitor (read/write latency + idle % + synthetic write-file smoke test) that classifies volume degradation in seconds (2025-03-18 post). Customer-side mitigation while running on EBS.
  • patterns/zero-downtime-reparent-on-degradation — trigger downstream of the health monitor: promote replica + fence primary + auto-provision replacement volume in seconds (2025-03-18 post). Clamps the impact window.
  • patterns/callback-slice-vm-go — Vitess evalengine's Go-specific bytecode-less VM design: compile each instruction into a closure pushed onto a []func(*VM) int slice; the VM loop is a single indirect call per opcode (2025-04-05 post). Canonical wiki instance of a high-performance Go interpreter.
  • patterns/static-type-specialized-bytecode — the compile-time specialization that makes the Vitess VM allocate zero memory on most opcodes; static types derived from MySQL's information schema by the Vitess semantic analyzer (2025-04-05 post).
  • patterns/vm-ast-dual-interpreter-fallback — retain the AST interpreter permanently alongside the VM as deoptimization fallback for value-dependent type promotions (canonical case: -BIGINT_MIN → DECIMAL) + one-shot evaluator for constant folding (2025-04-05 post).
  • patterns/fuzz-ast-vs-vm-oracle — differential-fuzz both interpreters against each other + against MySQL's C++ reference; surfaces bugs in MySQL itself that Vitess upstreams (2025-04-05 post).
  • patterns/architect-sharding-from-first-principles-per-engine — canonical new wiki pattern introduced by the 2025-07-01 Postgres launch. PlanetScale's stance that sharding layers are engine-specific by construction and not portable across engines: "Vitess' achievements are enabled by leveraging MySQL's strengths and engineering around its weaknesses. To achieve Vitess' power for Postgres we are architecting from first principles." Canonical wiki instance: Neki.

Key concepts introduced (continued — 2025-03-18 EBS reliability post)

Key concepts introduced (continued — 2025-04-05 interpreter post)

Key concepts introduced (continued — 2025-07-01 Postgres post)

  • concepts/proprietary-database-operator — architectural category for "real open-source engine + proprietary control plane + proprietary proxy layer" — distinct from forking the engine ( Aurora DSQL extension-based) and from compute-storage separation (Lakebase / Neon). PlanetScale for Postgres is the canonical wiki instance.
  • concepts/online-database-import — zero-downtime vendor-boundary database migration; PlanetScale for Postgres imports from any Postgres v13+ + supports automatic zero-downtime Postgres version upgrades. Canonical wiki instance of vendor-boundary migration as an advertised capability.

Key concepts introduced (continued — 2025-07-08 Caching post)

  • concepts/cache-hit-rate — canonical definition + hit- rate-vs-size framing + production-metric pointers (Postgres pg_stat_database, MySQL SHOW ENGINE INNODB STATUS, Redis INFO stats, CloudFront CacheHitRate).
  • concepts/cpu-cache-hierarchy — L1/L2/L3/RAM canonical vocabulary + physics of wire-delay / SRAM-vs-DRAM density / power + cache-line framing + coherence / false- sharing pointers.
  • concepts/temporal-locality-recency-bias — X.com worked example + the "slow storage + in-memory recency cache" architectural consequence + when it breaks down (scans, uniform-random, adversarial).
  • concepts/spatial-locality-prefetching — photo-album worked example + cross-tier applicability table (cache line / disk block / database range scan / OS readahead / application prefetch / HDD sequential).
  • concepts/fifo-cache-eviction — simplest policy; the baseline LRU improves on; second-chance variants (SIEVE / Clock) as production-reality form.
  • concepts/lru-cache-eviction — industry-default; hash-map + doubly-linked-list canonical shape; midpoint- insertion refinement (InnoDB); known failure modes (scan pollution, adversarial workload, concurrency tax).
  • concepts/time-aware-lru-cache-eviction — LRU + per- entry TTL with the three Dicken workload examples (48-hour social posts, daily weather, weekly email); TTL jitter pointer.
  • concepts/lfru-cache-eviction — dual-queue (high-priority LRU + low-priority aggressive) with promotion/demotion; sibling of InnoDB midpoint-insertion, ARC, W-TinyLFU, 2Q in the recency+frequency family.
  • concepts/postgres-shared-buffers-double-buffering — Postgres's two-layer caching stack (shared_buffers over OS page cache) + canonical 25%-of-RAM sizing rule + the philosophical split from MySQL InnoDB's single-layer design + the "more complex because of ACID" framing.

Key patterns introduced (continued — 2025-07-08 Caching post)

  • patterns/pair-fast-small-cache-with-slow-large-storage — the foundational caching pattern at every tier (CPU cache / RAM / database buffer pool / application in-memory cache / CDN edge / warehouse query cache). Canonical wiki anchor for the "single idea applied at every layer" framing.
  • patterns/spatial-prefetch-on-access — speculatively load neighbours on every access, exploits spatial locality. Photo-album is the canonical application-tier instance; cache-line + OS readahead + database range-scan are cross-tier relatives.
  • patterns/cdn-edge-cache-over-central-origin — classic CDN architecture: edges absorb reads near users, origin remains the single source of truth + write target. CloudFront + S3 is the canonical deployment; Cloudflare and the Tigris counterexample both live on this axis.

Key systems introduced (continued — 2025-10-14 Postgres 17 vs 18 post)

  • systems/linux-io-uring — Linux's async-I/O kernel interface; one of Postgres 18's three io_method options. Kernel 5.1+ (May 2019); shared-memory ring buffers for submission + completion. Loses on Postgres's workload shape for structural reasons (index scans don't use AIO; checksum/memcpy remain CPU-serial).
  • systems/sysbench — open-source Lua-scriptable OLTP benchmark tool (akopytov/sysbench) used by Dicken for the 96-run sweep. oltp_read_only workload is the read-path proxy.

Key concepts introduced (continued — 2025-10-14 Postgres 17 vs 18 post)

  • concepts/postgres-async-io — Postgres 18's new io_method configuration with three modes (sync, worker, io_uring). Scope limited to reads in 18.x; index scans don't use AIO; post-I/O work remains synchronous.
  • concepts/async-io-concurrency-threshold — async I/O only pays off above a concurrency / I/O-rate threshold; below it, sync wins on simpler code path. Measured explicitly in Dicken's data: io_uring loses at 1 connection, narrows at 10, wins only at 50 on local NVMe with large range scans.

Key patterns introduced (continued — 2025-10-14 Postgres 17 vs 18 post)

  • patterns/background-worker-pool-for-async-io — dedicated background worker processes handle I/O on behalf of calling backends; distributes post-I/O CPU work across workers. Postgres 18's io_method=worker default. Deliberately chosen over io_uring for CPU-distribution + interface- portability + tuneability.

Key concepts introduced (continued — 2026-04-21 Consistent Lookup Vindex post)

  • concepts/vindex — Vitess's row-to-keyspace_id mapping primitive. Primary Vindex drives sharding; Secondary Vindex is a cross-shard index materialising column → keyspace_id for non-sharding-key lookups.
  • concepts/consistent-lookup-vindex — a Secondary Vindex variant that avoids 2PC on every DML via three-connection ordered commit, tolerating orphan lookup rows because user queries always filter on the authoritative user table.
  • concepts/keyspace-id — binary(8) shard-address identifier; the output of any Vindex lookup.
  • concepts/orphan-lookup-row — a lookup-Vindex row whose referenced user row no longer exists; safe (user queries still return correct results) and lazily reclaimed on next colliding unique insert.

Key patterns introduced (continued — 2026-04-21 Consistent Lookup Vindex post)

  • patterns/ordered-commit-without-2pc — three independent MySQL connections (Pre / Main / Post) committed in a fixed order (rollback in same order on failure) as an alternative to 2PC for the cross-shard- write case where one participant is authoritative and user-facing queries always route through it. Weaker guarantee than 2PC but dramatically cheaper on the happy path and self-healing on the error path (via lazy orphan reclamation on duplicate-key errors).

Queued

  • PlanetScale Blog (tier 3) — ~295 raw articles downloaded / 283 pending ingestion. Dicken + Lambert + team's database-internals content makes up a meaningful minority; the bulk is product marketing and general how-to content that needs tier-3 scope filtering on ingest.
Last updated · 319 distilled / 1,201 read