Skip to content

CONCEPT Cited by 1 source

Connection multiplexing

Connection multiplexing is the architectural pattern of decoupling the number of upstream connections a backend holds open from the number of client-side connections by interposing a proxy tier that accepts many client connections and multiplexes their commands onto a small, stable pool of outbound connections to the backend.

Why it matters

Most database / cache backends have a hard or practical ceiling on concurrent connections (memory per-connection, file-descriptor limits, kernel socket state, per-connection locking, etc.). In an elastic client fleet, the natural scaling mode is more client pods open more connections — so the backend's connection pressure scales with client-fleet size rather than client-request rate.

Two specific failure modes arise without multiplexing:

  • Asymptotic connection ceiling. Client fleet grows until the backend hits its connection limit and refuses new connections or degrades.
  • Thundering-herd new-connection storms on rapid client-fleet scale-up. Every new pod opens a fresh pool of connections to every backend; the TCP / TLS / auth handshake burst bottlenecks backend I/O and degrades availability for existing clients.

A multiplexer tier absorbs both: client-pod count and elasticity decouple from the backend's connection load.

Canonical shape

┌──────────┐  many short-lived / elastic
│ clients  ├─ conns ──┐
└──────────┘          │
┌──────────┐          ▼        few, stable
│ clients  ├──► [ multiplexer ] ──────────► [ backend ]
└──────────┘          ▲         conns
┌──────────┐          │
│ clients  ├─ conns ──┘
└──────────┘

Client connections are cheap (designed for it). Outbound connections are pooled + long-lived. The multiplexer coalesces in-flight commands onto the outbound pool under whatever pipelining / protocol rules the backend supports.

Load-bearing design decisions

  • Cheap client connections. The multiplexer's frontend must make accepting a new client connection as lightweight as possible; the whole point is to absorb fleet churn on this side.
  • Stable outbound pool sizing. Outbound pool size is a slow- moving operational parameter tuned to the backend's capacity, not to the client fleet's size.
  • Protocol awareness. To multiplex safely, the proxy must understand enough of the protocol to avoid interleaving illegal state — e.g. transactions / multi-command sequences / subscriptions pin to a single outbound connection while they're in progress. Opaque byte-shoveling isn't enough.
  • Stateless proxy. The multiplexer itself should hold no authoritative state so the fleet can scale horizontally without coordinating state migration.

Bonus properties it unlocks

  • Topology-change absorption. Node failovers / cluster scaling / transient backend errors can be handled inside the multiplexer (retry on a different outbound connection, replay-safe commands); clients don't see them.
  • Centralized observability. Every command crosses the multiplexer → uniform metrics/logs/traces independent of client language.
  • Centralized guardrails. Command-level policy enforcement (key-prefix routing, command-type allowlists, QoS backpressure, inline encryption) implements once in the multiplexer rather than in every client library.

Caveats

  • Adds a critical-path network hop → some latency penalty (mitigable via zonal colocation + careful I/O).
  • Must be available. If the multiplexer is a single point of failure, it's just moved the reliability problem. Horizontal scaling + statelessness + no-authoritative-data are the countermeasures.
  • Pipelining correctness depends on the proxy correctly modeling which commands can share an outbound connection and which cannot. Transactions + pub/sub + blocking commands are the usual pin-the- connection cases.

Seen in

  • sources/2026-04-21-figma-figcache-next-generation-data-caching-platform — Figma FigCache is the canonical instance given in the ingest. Rolling FigCache out in front of ElastiCache Redis dropped cluster connection counts by an order of magnitude and eliminated the thundering-herd class of reliability incidents ("Connection pooling in FigCache eliminated an entire class of reliability risks around thundering herds of new connections from clients"). Specifically named as a design objective: "the connection volume served by Redis should be decoupled from the size and elasticity of client applications, and Redis should be isolated from thundering herds of new connections when client capacity scales up rapidly."
Last updated · 200 distilled / 1,178 read