Skip to content

CONCEPT Cited by 2 sources

Shadow table

Definition

A shadow table is a second, hidden table created in the same database as the production table it mirrors, carrying either the same schema (for zero-downtime moves / reshards) or the schema-after-DDL (for online schema changes). The table is initially empty; the migration tool fills it by copying rows from the production table under a consistent snapshot while simultaneously applying every concurrent write that lands on the production table so the shadow catches up in bounded time. At a cut-over freeze point the two tables are declared equivalent and the shadow is atomically swapped into the production table's name.

The term comes from the online-DDL-tool family — pt-online-schema-change, gh-ost, Vitess VReplication-driven Online DDL — which all build this same kind of hidden copy. The alternatives in the literature are "ghost table" (gh-ost's naming) and "swap table."

Shape

For online schema changes:

  1. Build: CREATE TABLE users_shadow LIKE users; (same schema), then apply the DDL to the shadow: ALTER TABLE users_shadow DROP COLUMN title;.
  2. Backfill: copy rows from users to users_shadow, ordered by primary key, in batches inside consistent snapshots so the read is non-locking.
  3. Track: tail the source's change log (binlog / WAL) and apply concurrent writes to the shadow, filtered through the DDL's column-projection.
  4. Cut over: brief write lock on the source, final catch-up pass, atomic table rename: RENAME TABLE users TO users_old, users_shadow TO users;.

For data-motion migrations (imports, reshards, table moves) the shadow table lives on the destination keyspace / shard and the shape is analogous — the "swap" is a routing-rule flip at the proxy layer rather than a RENAME TABLE.

Why this is the dominant online-DDL primitive

MySQL's native ALTER TABLE against a large production table blocks writes for the duration of the rewrite (modulo ALGORITHM=INSTANT / INPLACE for specific DDL shapes). Even INPLACE rebuilds can take hours on a TB- scale table, and during that time the table is either write-locked (worst case) or incurs heavy IOPS contention (best case). A shadow table moves the rewrite off the hot path: production writes continue to hit the original table; the cost of the DDL is absorbed by the backfill on the shadow; the only production-visible cost is a brief table-level lock at the cut-over swap.

Seen in

  • sources/2026-04-21-planetscale-behind-the-scenes-how-schema-reverts-work — canonical wiki framing. Guevara + Noach: "Instead of altering your production table, they create an empty shadow table with the same schema as the production table but no data. They implement the schema change on the shadow table. This is a cheap operation, given the table has no data. They copy data from the original table, track incoming changes, and apply them to the shadow table. Finally, when the tables are in sync, the tools cut over. They move aside your original table and replace it with the shadow table." The article makes the shadow table explicitly first-class and walks through ten architectural diagrams of its lifecycle. Critically, the article introduces a second life for the shadow table: Vitess keeps it alive after cut-over and uses it as a pre-staged inverse shadow so a schema revert is a second swap of the same already-in-sync table rather than a second data copy — the load-bearing novelty of PlanetScale's schema-revert feature.
  • sources/2026-02-16-planetscale-zero-downtime-migrations-at-petabyte-scale — the shadow table is the destination-side primitive filled by VReplication during zero-downtime migrations. Matt Lord's walk of copy-phase / replication-phase / VDiff / cut-over is the canonical mechanics for how the shadow is populated and kept in sync. Same primitive, larger scale (data motion across keyspaces / shards rather than DDL within a table).
Last updated · 319 distilled / 1,201 read