PlanetScale — Announcing Vitess 21¶

Summary¶

PlanetScale's Vitess Engineering Team announces Vitess 21 and Vitess Operator 2.14.0. The release is organised around four substantive architectural axes — query compatibility (reintroduced atomic distributed transactions with deeper integration into Online DDL + VReplication; experimental recursive CTEs), VReplication (reference-table materialization via Materialize; dynamic workflow configuration replacing VTTablet-flag-driven config), throttler (multi-metric support with v20 backward compatibility removed in v22), backup & restore (experimental mysqlshell logical-backup engine contributed by Slack Engineering) — plus new VTGate HPA support, Kubernetes 1.31 compatibility, and new vexplain trace + vexplain keys subcommands on the [vexplain](<../systems/vitess-vexplain.md>) surface. Fifth canonical Vitess-internals disclosure on the wiki (after evalengine, VReplication/VDiff/MoveTables, Consistent Lookup Vindex, and the Throttler three-part series).

Key takeaways¶

Atomic distributed transactions reintroduced with deeper Vitess integration. "We're reintroducing atomic distributed transactions with a revamped, more resilient design. This feature now offers deeper integration with core Vitess components and workflows, such as Online DDL and VReplication (including operations like MoveTables and Reshard)." Configuration simplified; still experimental. Positions against the lower-overhead Consistent Lookup Vindex ordered-commit protocol — atomic distributed transactions (2PC-like) are the strong-atomicity option for cross-shard writes; the Consistent Lookup Vindex is the weaker-than-2PC option for routing metadata. Two distinct points in Vitess's cross-shard-transaction design space now shipping in the same release train (Source: sources/2026-04-21-planetscale-announcing-vitess-21).
Experimental recursive CTEs for hierarchical queries. "Vitess 21 introduces experimental support for recursive CTEs, allowing more complex hierarchical queries and graph traversals. This feature enhances query flexibility, particularly for managing parent-child relationships like organizational structures or tree-like data." First wiki disclosure of recursive CTE support landing in the vtgate evalengine SQL surface. Experimental; feedback-requested (Source: sources/2026-04-21-planetscale-announcing-vitess-21).
Reference-table materialization canonicalised as a first-class VReplication primitive. "Vitess provides Reference Tables as a mechanism to replicate commonly used lookup tables from an unsharded keyspace into all shards in a sharded keyspace. Such tables might be used to hold lists of countries, states, zip codes, etc., which are commonly used in joins with other tables in the sharded keyspace. Using reference tables allows Vitess to execute joins in parallel on each shard, thus avoiding cross-shard joins." Prior workflow required hand-authored Materialize workflows; v21 adds explicit Materialize-command support. New pattern: patterns/reference-table-materialization-via-vreplication — push the lookup data to every shard so the join stays shard-local (Source: sources/2026-04-21-planetscale-announcing-vitess-21).
Dynamic VReplication workflow configuration. "Previously, many configuration options for VReplication workflows were controlled by VTTablet flags. This meant that any change required restarting all VTTablets. We now allow these to be overridden while creating a workflow or updated dynamically once the workflow is in progress." Canonical dynamic-workflow-config shift: move runtime knobs off the process-flag surface (which couples them to VTTablet restart) into the workflow control plane (which allows in-flight update). Aligns with the broader wiki theme of moving config out of deploys and into runtime control planes (Source: sources/2026-04-21-planetscale-announcing-vitess-21).
Multi-metric throttler with v20-compatible wire format. "The tablet throttler has been redesigned with new multi-metric support. With this, the throttler now handles more than just replication lag or custom queries, but instead can work with multiple metrics at the same time, and check for different metrics for different clients or for different workflows." Pairs structurally with the 2026-04-21 three-part Anatomy of a Throttler series (same publication day) — that series canonicalises the design space of multi-metric throttling; this release canonicalises the shipping primitive. Backward compatible with v20 throttler (v20 primary + v21 replica or vice versa); compatibility removed in v22. Deprecations: --check-as-check-self / --check-as-check-shard sub-flags, SHOW VITESS_THROTTLER STATUS, SHOW VITESS_THROTTLED_APPS, and all /throttler/ API access points except /throttler/check. Response now includes a human-readable rejection summary. "When enabled, the throttler ensures it leases heartbeat updates, even if heartbeat configuration is otherwise unset. In other words, the throttler overrides the configuration when it requires heartbeat information." — first canonical wiki datum on the throttler forcing heartbeat injection when its own correctness requires it, independent of operator-specified heartbeat config (Source: sources/2026-04-21-planetscale-announcing-vitess-21).
mysqlshell experimental logical-backup engine, contributed by Slack Engineering. "Introducing an experimental mysqlshell engine. With this engine, it is possible to run logical backups and restores. The mysqlshell engine can be used to create full backups, incremental backups, and point-in-time recoveries. It is also available to use with the Vitess Kubernetes Operator." First canonical wiki instance of logical-backup coexistence with physical backup inside a Vitess cluster — complementary design points with different size / restore-speed / schema-version-independence trade-offs. Canonical wiki datum that backup engines are pluggable in Vitess; Slack's mysqlshell engine joins the prior-default physical-backup engine. External-company-contribution datum: Slack directly authors production substrates for Vitess (Source: sources/2026-04-21-planetscale-announcing-vitess-21).
vexplain trace + vexplain keys — two new sub-commands on the vexplain surface. "The new vexplain trace command provides deeper insights into query execution paths by capturing detailed execution traces. This helps developers and DBAs analyze performance bottlenecks, review query plans, and gain visibility into how Vitess processes queries across distributed nodes. The trace output is delivered as a JSON object, making it easy to integrate with external analysis tools." "The new vexplain keys feature helps you analyze how your queries interact with your schema, showing which columns are used in filters, groupings, and joins across tables. This tool is especially useful for identifying candidate columns for indexing, sharding, or optimization, whether you're using Vitess or a standalone MySQL setup." First canonical wiki datum on vexplain as an explicit observability surface; vexplain keys usable against standalone MySQL too, extending Vitess's tooling reach beyond its own clusters (Source: sources/2026-04-21-planetscale-announcing-vitess-21).
Vitess Operator 2.14: VTGate HPA + Kubernetes 1.31 + per-keyspace Docker image selection. "In v2.14, we have added the ability to horizontally scale the VTGate deployment using an HPA. We have upgraded the supported version of Kubernetes to the latest version (v1.31). We have added a feature that allows users to select Docker images on a per-keyspace basis instead of a single setting for the entire cluster." First canonical wiki datum on VTGate horizontal autoscaling under a Kubernetes HPA — canonicalises autoscaling at the query-routing-proxy tier (as opposed to the data-tier vttablet layer, which is driven by topology + throttler decisions rather than HPA). Per-keyspace image selection supports staged rollouts (Source: sources/2026-04-21-planetscale-announcing-vitess-21).
VTOrc errant-GTID visibility metric. "We have added a new metric in VTOrc that shows the count of errant GTIDs in all the tablets for better visibility and alerting. This will help operators track and manage errant GTIDs across the cluster." Canonical errant-GTID operational-visibility datum — errant GTIDs indicate replication divergence (transactions committed on a replica that aren't on the primary) and are a precondition for split-brain consequences. VTOrc's fleet-wide count is the precursor metric that fires before a reparent decision has to be made (Source: sources/2026-04-21-planetscale-announcing-vitess-21).
Online DDL improvements. ALTER VITESS_MIGRATION CLEANUP ALL command support; more INSTANT DDL scenario analysis; charset-change handling in Online DDL now does programmatic text conversion rather than MySQL's CONVERT(... USING utf8mb4) (improves performance when such columns are in the primary key or iteration key); more schema + diff analysis delegated to the schemadiff library (programmatic power + better testability); self-referencing foreign key bug fixes (PlanetScale MySQL build only) (Source: sources/2026-04-21-planetscale-announcing-vitess-21).
VTAdmin UI for VReplication workflows + distributed transactions. "New VTAdmin pages have been added for creating, monitoring, and managing VReplication Workflows. We have also added a dashboard to view and conclude distributed transactions." First canonical wiki datum on VTAdmin as the UI-level control plane for atomic distributed transactions — pairs with the feature's reintroduction (Source: sources/2026-04-21-planetscale-announcing-vitess-21).

Architectural framings¶

Release notes as public architectural disclosure¶

This post is a release-notes announcement, not a deep-dive architectural retrospective. The architectural substance is layered on top of the feature list via explicit framing: each feature is described with its motivation (what operator pain it addresses), its interaction with existing Vitess subsystems (Online DDL + VReplication + VTTablet + VTOrc), and its compatibility envelope (experimental vs production-ready; backward-compat vs deprecation).

Canonical wiki datum that high-quality release notes can legitimately be a Tier-3-clearing architectural disclosure — the density here is ~60-70% architectural content once you unpack the sub-features. PlanetScale's release-notes voice (as opposed to, say, the Rafer Hazen LLM-index-suggestions voice or the Shlomi Noach throttler voice) is enumerative but still architecturally dense via the per-feature framing discipline.

Four design-space points for cross-shard writes in one release¶

The 2026-04-21 PlanetScale publication day canonicalises an unusual richness of Vitess cross-shard-write primitives, all published on the same day:

Consistent Lookup Vindex (ordered-commit without 2PC) — authoritative-table-plus-derivatives asymmetric protocol for sharding metadata (Achieving data consistency with the consistent lookup Vindex, same day).
Atomic distributed transactions (revamped, deeper integration) — strong 2PC-like atomicity for cross-shard application writes (this post).
Multi-metric tablet throttler — load-admission gate on primary ingest (this post + the sibling throttler series).
VReplication reference-table materialization + dynamic config — cross-shard data motion for join locality (this post).

Together they span the full correctness vs performance vs flexibility trade space for multi-shard writes. Canonical wiki framing: Vitess's sharding layer offers a menu, not a point solution — applications pick the primitive that matches their shape of cross-shard work.

Backup engine as a pluggable surface¶

The mysqlshell engine lands alongside the existing physical-backup engine, authored by Slack Engineering rather than PlanetScale. This is the first canonical wiki instance of Vitess's backup subsystem as a pluggable surface accepting external-company contributions. Canonical new pattern: patterns/logical-backup-engine-plug-in — treat the backup engine as an interface rather than a single implementation; logical and physical can coexist for different restore-time / schema-version / incremental-delta requirements.

Heartbeat-injection-override invariant¶

The throttler now "ensures it leases heartbeat updates, even if heartbeat configuration is otherwise unset." Canonical wiki datum that subsystems with correctness-critical dependencies on observability primitives should override operator configuration when their own guarantees require it. Sibling to Pinterest's @authorize_tool decorator shape (the application owns the fine-grained policy; the platform owns the coarse gate) — the throttler owns the heartbeat-injection gate regardless of operator-supplied heartbeat config, because without heartbeats the throttler's replication-lag metric has no source of ground truth.

Caveats¶

Release-notes voice, not architectural retrospective: every feature named but few internal mechanisms explained. Atomic distributed transactions "deeper integration" not elaborated (no 2PC protocol details, no Online-DDL hand-off mechanism, no VReplication-coordination semantics). Recursive CTE implementation not described. mysqlshell engine internals not disclosed.
Experimental status on multiple headline features — atomic distributed transactions, recursive CTEs, mysqlshell backup engine all flagged experimental. No production-scale usage or feedback-loop data.
Throttler multi-metric design is disclosed at the "what changed" altitude; the architectural rationale is published separately in the three-part Anatomy of a Throttler series on the same day — this post treats the throttler as one bullet in a feature list.
No production numbers: no QPS / latency / correctness deltas, no A/B on any of the new features, no rollout cadence, no adoption data.
VTGate HPA disclosed but the specific signals driving the HPA (CPU? request rate? VTGate-internal metric?) not named.
Per-keyspace Docker image feature use case stated abstractly; no canonical operator workflow for staged version upgrades shown.
VTOrc errant-GTID metric shipped but no threshold guidance; reader has to know errant-GTID operational lore to interpret the count.
Backward-compat sunset timer: v21 throttler is backward-compatible with v20; v22 removes this compatibility. Operators on older versions need a migration window before v22.
Tier-3 source; PlanetScale has direct commercial interest in Vitess uptake.

Source¶

Original: https://planetscale.com/blog/announcing-vitess-21
Raw markdown: raw/planetscale/2026-04-21-announcing-vitess-21-d1ecbe7b.md