Skip to content

SYSTEM Cited by 3 sources

VReplication

What it is

VReplication (vitess.io docs) is Vitess's native data-replication framework — the substrate Vitess uses for all of its data-motion workflows: database imports (migrate an external MySQL into Vitess), resharding (split or merge shards), table moves (relocate tables between keyspaces), materialised views, and online schema changes. Every one of these user-facing operations is implemented on top of the same VReplication primitive: a per-source-to-per-target stream that consists of an initial row-copy phase (read a consistent non-locking snapshot of the table ordered by primary key) followed by a continuous replication phase (tail MySQL binlog events from the GTID position captured at the end of the copy phase).

Architectural shape

Each VReplication workflow consists of N streams, one per (source, target-shard) pair. Each stream has its own independent state and advances independently.

Copy phase (per table, serial across tables within a workflow):

  1. Source tablet issues LOCK TABLES <tbl> READ on the source MySQL (read-only for milliseconds).
  2. Opens START TRANSACTION WITH CONSISTENT SNAPSHOT.
  3. Reads @@global.GTID_EXECUTED to get the GTID set corresponding to this snapshot.
  4. Releases the LOCK TABLES.
  5. Reads rows from the snapshot ordered by PRIMARY KEY (or a best-PK-equivalent non-null unique key) so the source MySQL reads directly off the clustered index without a filesort.
  6. Each row is filtered (per the workflow's sharding scheme) and routed to the appropriate target shard's PRIMARY tablet, where it is inserted.
  7. Per-stream progress is persisted in the target's sidecar copy_state table so the copy can resume from the exact last-copied key on restart.

Copy phases are interruptible and resumable — the vreplication_copy_phase_duration flag bounds how long a single copy cycle runs before the stream pauses, catches up on binlog events for the rows copied so far, and re-enters the copy phase. This regular catch-up loop prevents the binlog retention horizon from lapping the stream during long copies. (Source: sources/2026-02-16-planetscale-zero-downtime-migrations-at-petabyte-scale.)

Replication phase (after all rows in the workflow are copied):

  1. Per-stream COM_BINLOG_DUMP_GTID command to the source MySQL, providing the stream's recorded GTID position.
  2. Source MySQL streams binlog events from that position.
  3. Events are filtered (per the sharding scheme) and applied on the target PRIMARY tablet.
  4. Advancing GTID persisted in the sidecar vreplication table on every event-batch commit so the stream can always restart from exactly-where-it-left-off.

Use cases

Via the MoveTables and Reshard workflow wrappers:

  • External-MySQL → Vitess imports (PlanetScale Database Imports is built on this shape): import an unmanaged external MySQL into a Vitess cluster. "An unsharded MySQL database … split into N shards as part of the data migration into PlanetScale." (Source: sources/2026-02-16-planetscale-zero-downtime-migrations-at-petabyte-scale.)
  • Resharding — split or merge existing Vitess shards.
  • Table move between keyspaces — e.g. promote a materialised view to its own keyspace.
  • Online schema change — Vitess's native DDL workflow uses VReplication to shadow-migrate rows into a schema-evolved copy and then cutover.

Why it shows up on this wiki

VReplication is the load-bearing data-motion primitive of Vitess, and therefore of PlanetScale MySQL. Every zero-downtime migration PlanetScale performs — including the petabyte-scale customer migrations documented by Matt Lord — runs on VReplication. The 2026-02-16 post is the canonical public walkthrough of the mechanism: the exact MySQL-primitive choices (LOCK TABLES READ → START TRANSACTION WITH CONSISTENT SNAPSHOT → @@global.GTID_EXECUTED → release → PRIMARY KEY-ordered stream → COM_BINLOG_DUMP_GTID), the copy / catch-up interleaving, the per-target-shard fan-out, and the sidecar persistence for restart semantics.

Fault tolerance

Every state-bearing decision point in a VReplication stream persists state in sidecar tables so restart is the recovery path for any failure. "Anything can fail throughout this process and the system will be able to recover and continue where it left off." (Source: sources/2026-02-16-planetscale-zero-downtime-migrations-at-petabyte-scale.) See concepts/fault-tolerant-long-running-workflow for why this is a correctness requirement, not a nice-to-have, at petabyte scale.

Seen in

  • sources/2026-02-16-planetscale-zero-downtime-migrations-at-petabyte-scale — canonical wiki description of VReplication's copy-plus- replication architecture, the exact MySQL primitives used, per-stream GTID tracking, sidecar persistence for fault tolerance, and the copy/catch-up interleaving that prevents binlog retention from lapping long copies. VReplication is named as the substrate under PlanetScale's Database Imports feature + the MoveTables workflow.

  • sources/2026-04-21-planetscale-announcing-vitess-21 — Vitess 21 ships two VReplication enhancements: (1) Reference-table materialization as a first-class Materialize primitive — prior workflow required hand-authored workflows; v21 adds explicit Materialize-command support for replicating small read-mostly lookup tables from an unsharded keyspace into every shard, enabling shard-local joins. New canonical patterns/reference-table-materialization-via-vreplication pattern. (2) Dynamic workflow configuration — runtime knobs previously bound to VTTablet command-line flags (requiring process restart to change) are moved onto the workflow control plane: "We now allow these to be overridden while creating a workflow or updated dynamically once the workflow is in progress." Canonical wiki instance of moving config off process-flags into the workflow control plane. Also canonicalises VReplication's deeper integration with the reintroduced atomic distributed transactions, including MoveTables and Reshard operations.

  • sources/2026-04-21-planetscale-behind-the-scenes-how-schema-reverts-work — canonical wiki description of VReplication as the substrate for online schema changes + instant schema reverts. Guevara + Noach walk the full online-DDL lifecycle: (1) schema-change execution uses a concepts/shadow-table-based shadow- table online schema change, with VReplication handling the backfill + catch-up + cut-over phases exactly as in the 2026-02-16 data-motion post — just at table scope inside a single keyspace rather than across keyspaces. (2) Five VReplication design properties named as uniqueness factors: copy-and-changelog progress both tracked (not just backfill); per-transaction GTID mapping; GTID-set-driven interleaving between copy and change-log phases; transactional coupling of sidecar state with destination write; and crucially "Unlike any other schema change solution, Vitess does not terminate upon migration completion." (3) Schema revert as the load-bearing payoff: after cut-over, VReplication is re-primed in the inverse direction (new → old) so the old-schema table stays current with every post-cut-over write — canonical concepts/pre-staged-inverse-replication concept and patterns/instant-schema-revert-via-inverse-replication pattern. Revert is then a second freeze-point swap of two already-in-sync tables, no data copy, invoked by the "Revert changes" button in the PlanetScale deploy- requests UI. The architectural insight: VReplication's non-termination after cut-over turns a one-way door into a revolving door for the schema-change case, mirroring the already-canonical patterns/reverse-replication-for-rollback at the data-motion scale.

Last updated · 319 distilled / 1,201 read