Skip to content

PATTERN Cited by 2 sources

Query routing proxy with health-aware pool

The pattern

Decompose a database-protocol proxy tier into two processes plus a shared state store:

  1. A stateless query-routing layer in front (accepts client connections, parses queries, makes routing decisions by reading the shared state store).
  2. A stateful per-backend middleware in back (owns the connection pool to a single database instance, runs health checks, publishes health + role into the shared state store).
  3. A shared state store (a small strongly-consistent KV store like etcd / Consul / ZooKeeper) that the middleware writes into and the router reads from.

The canonical instance on the wiki is VTGate + VTTablet + topo-server in Vitess.

The data path

client → [stateless router] → [per-backend middleware] → database
             ↓                          ↑
             └──── reads ────┐         │ writes
                             ▼         │
                        [shared state store]

Why split router from pool owner

The naive shape is a single proxy that holds client connections, holds back-end connections, and does routing all in one process. That shape has three failure modes at scale:

  1. Connection-fan-in is bounded by one process's max-open-files / max-threads. 1M client connections cannot be held by a single process.
  2. Health checking is duplicated: every proxy instance has to probe every database. N × M cost at fleet scale.
  3. Stateful routing state (which tablet is primary, etc.) has to be synchronized across proxy replicas, or every proxy has to re-probe.

The decomposed shape fixes all three:

  • Router is stateless and scale-out. Many router replicas behind a load balancer; no coordination between them. Each one re-reads the shared state store per routing decision (usually cached with watch-based invalidation).
  • Health check is owned by the middleware co-located with the database. Local loopback probe, no network round-trip, no duplication across proxy replicas.
  • The shared state store is the coordination point, not the routers among themselves.

When to use it

  • Database proxy tiers that must scale to very high client-connection counts (the two-tier connection pooling discipline exploits this split to queue back-end requests).
  • Topologies with role-based routing (primary / replica, live / draining, serving / not-serving) that change over time — the shared state store is the source of truth for "which backend gets this query".
  • Topologies that need automatic failover without client reconnect — clients connect to the router; the middleware + state store handle the role transition; the router starts sending traffic to the new primary without the client noticing.

When NOT to use it

  • Very small deployments where a single-process proxy is fine (pgBouncer / ProxySQL in their default configurations are the single-process shape).
  • Environments where the coordination overhead of a state store isn't worth it (e.g., a pair of MySQL instances with a VIP).
  • Synchronous replication topologies that need cross-replica consensus on every write (the shared state store isn't a consensus log for data; it's a small metadata store for topology).

Canonical instance: VTGate + VTTablet + topo-server

Verbatim from PlanetScale's architectural description:

"VTGate is an application-level query routing layer while VTTablet behaves as a middleware between VTGate and MySQL. … The VTTablet will manage connection pooling and perform health checks for MySQL instances, updating its status in a topo-server. Meanwhile, VTGate determines available tablets and their roles via the topo server and reroutes traffic as needed."

(Source: sources/2026-04-21-planetscale-planetscale-vs-amazon-rds)

This is the full pattern: stateless router (VTGate), stateful per-backend middleware (VTTablet, one per MySQL instance), shared state store (topo-server, typically etcd). Every PlanetScale MySQL database runs on this topology.

Variants

  • Same-host middleware (Vitess): VTTablet runs on the same host as MySQL, probes are local.
  • Sidecar middleware: same shape but the middleware is a Kubernetes sidecar container in the same pod.
  • Agent-on-node: the middleware is a per-node daemon, not per-instance.

The load-bearing property is co-location with the backend — it makes health checks cheap and gives the middleware privileged access to probe + reconfigure the backend.

Trade-offs

  • Latency: adds two hops (client → router → middleware → backend) vs one for direct-connect. Worth it when client connections are plentiful and backend connections are precious.
  • Operational complexity: you are now running three tiers (router, middleware, state store) instead of one. The state store is the hard part — usually it's etcd and usually it's fine, but it is a coordination point that can fail.
  • State-store bottleneck: if the router re-reads the state store on every query without caching, the state store is the scaling bottleneck. In Vitess, VTGate caches topo-server reads with watch-based invalidation — state-store load scales with topology-change rate, not query rate.

Seen in

Last updated · 470 distilled / 1,213 read