Skip to content

PATTERN Cited by 1 source

Local MySQL CI for fast tests

Problem

CI pipelines for applications that run on a managed sharded MySQL platform (PlanetScale, Vitess-backed deployments) face a fidelity-vs-speed trade-off:

  • High fidelity — run CI against the actual production-like platform (PlanetScale branch, Vitess cluster). Catches routing-layer / sharding / read-replica-split bugs. But slow: branch provisioning latency, network round-trips per query, billing per CI run.
  • High speed — run CI against a local database that speaks the same wire protocol. Fast because there's no network + no branch provisioning. But lower fidelity: no Vitess routing, no shard-aware behaviour, no read-replica topology.

For a test suite (as opposed to an integration-test suite), the choice is almost always speed.

Shape

Run the CI test suite against local MySQL (or equivalent) in the same container as the CI job, not against a remote managed instance. Close the fidelity gap separately via the deploy-request lifecycle: the PR-bot creates a database-branch on the production platform and executes the migration there for production-topology verification, while the test suite itself uses the local MySQL for speed.

Canonical instance

Mike Coutermarsh, PlanetScale's internal Rails-CI:

"Our CI runs against local MySQL for lowest latency."

(Source: sources/2026-04-21-planetscale-how-planetscale-makes-schema-changes)

Coutermarsh's post leaves the fidelity-gap-closing implicit, but the architecture is complete: local MySQL for test-suite speed + PlanetScale deploy-request for production-topology verification + Vitess online migration for production-safety execution.

Why the trade-off almost always favours speed

The test feedback loop is a first-order DevEx primitive — wall-clock from push to pass/fail determines how many PRs an engineer can validate per day. The latency tax of running a full test suite against a remote branch is often the dominant term:

  • Per-query RTT: local MySQL is ~0.1ms; remote branch is ~10-100ms (network + routing overhead). At thousands of queries per test, this compounds to minutes per run.
  • Branch-provisioning time: creating a fresh branch per PR has a provisioning latency on top of every run.
  • Concurrency limit: the managed platform may have a branch-count limit that bottlenecks parallel CI.

Local MySQL avoids all three.

The fidelity gap is small for most test classes

The test classes that actually depend on production-topology behaviour are narrow:

  • Vitess routing-rule tests.
  • Sharded-query correctness tests.
  • Read-replica consistency-window tests.
  • Vindex-specific tests.

The majority of application tests (model logic, business rules, validations, controller behaviour, happy-path request/response) don't depend on the storage topology and run identically on local MySQL and PlanetScale.

Composition

  • With patterns/pr-bot-auto-deploy-request — the PR-bot runs the migration twice: once in CI against local MySQL (for fast test feedback), once against the PlanetScale branch (for production-topology verification). Both gates must pass before merge.
  • With CI-parallel- over-local-serial — Coutermarsh's earlier 2022 post describes running the Rails test suite in parallel on 64-core Buildkite agents; the local-MySQL choice compounds the speedup because each worker has its own low-latency database.
  • With local emulation first — same principle applied to other managed services (emulators for cloud queues, KV stores, etc.).

Trade-offs

  • Topology-specific bugs escape to deploy-request gate — any bug that requires Vitess routing to reproduce won't surface in CI; it shows up on the PlanetScale branch deploy step or in production.
  • Migration-file compatibility gap — migrations that use MySQL-specific features supported by PlanetScale's Vitess layer may behave differently; local MySQL catches syntactic validity but not Vitess-version-compatibility.
  • Data-distribution behaviour invisible — tests that would expose shard-key distribution issues can't see them on unsharded local MySQL.
  • Schema-lint is the backstop — if local MySQL accepts a DDL that PlanetScale rejects (e.g. a Vitess-specific migration-safety rule), the PR-bot's lint + deploy-request lint catches it before it ships.

When the trade-off doesn't favour speed

  • Integration tests for sharding/routing — if the test class's whole purpose is to verify Vitess behaviour, local MySQL defeats the test.
  • Platform-feature tests — tests for planetscale_rails gem itself, or for PlanetScale- API integrations, need the managed platform.
  • High-fidelity staging environment — some teams invest in a separate staging pipeline that runs against a PlanetScale branch, used as a pre- production gate rather than a per-PR gate.

Seen in

  • sources/2026-04-21-planetscale-how-planetscale-makes-schema-changes — Mike Coutermarsh names the choice verbatim: "Our CI runs against local MySQL for lowest latency." PlanetScale's internal Rails CI uses local MySQL for test-suite speed; production-topology fidelity is closed downstream by the PR-bot creating a PlanetScale branch for every PR with schema changes.
Last updated · 378 distilled / 1,213 read