Skip to content

PATTERN Cited by 1 source

Big pooler with affinity plus small pooler for HA

Statement

When latency variability from the default AZ-spread pooler deployment is intolerable, replace it with:

  • One "big" pooler instance configured with Kubernetes node-affinity to run on the same node as the database, configured with the CPU Manager static policy for exclusive physical cores.
  • One "small" pooler on a different node as a warm HA backup, taking over when the primary pooler or its node fails.

The first pooler is the primary path — it gets the lowest possible latency because it's same-node as Postgres (no cross-AZ network hop, likely no cross-node at all). The second is the availability insurance.

When to use it

  • Latency-critical Postgres workloads — trading infrastructure operator nicety for p99 predictability.
  • Workloads where operators can accept higher complexity and less automatic self-healing in exchange for consistency.
  • Single-node-capacity workloads — the "big" pooler must fit the aggregate connection demand on one node; if it can't, this pattern doesn't scale and you're back to horizontal spread.

Why it works

  • Co-location with database eliminates the inter-pod, and potentially inter-AZ, network RTT on every query — the single largest source of Kubernetes-induced latency variance.
  • Exclusive CPU pinning eliminates the HT-softirq and noisy-neighbour dimensions.
  • Small secondary pooler satisfies availability SLOs — when the big pooler fails, clients reconnect through the secondary while the primary is rescheduled.

Trade-offs

  • Manual operator effort: the pattern is explicitly outside Zalando's "default happy path" — operators assemble node-affinity, CPU manager, and Service routing rules by hand.
  • Reduced availability posture — the primary pooler has concentrated fate-sharing with its host; the secondary must be right-sized for the full load during failover.
  • Wasted headroom on the secondary in steady state.
  • Harder to upgrade — rolling the pooler fleet now involves coordinating primary vs. secondary rather than a simple Deployment rollout.

Contrast with the default pattern

Dimension Default (Deployment, AZ-spread) This pattern
Latency p99 Higher, variable Lower, stable
Availability High (N pods, N AZs) Two pods, primary is SPOF-like
Complexity Low — one Deployment + Service Higher — two Deployments + affinity
Failover behaviour Automatic Manual or backup-activation-based
Throughput scaling Horizontal add replicas Vertical scale the big pooler only

Seen in

  • sources/2020-06-23-zalando-pgbouncer-on-kubernetes-minimal-latency — Zalando explicitly documents this as the escape-hatch pattern beyond the default: "For those cases when this variability could not be tolerated, we would consider creating manually a single 'big' pooler instance with the affinity to put it on the same node as the database and adjust CPU manager to squeeze everything we can from this setup. This new instance would be a primary one for connecting with another one providing HA."
Last updated · 476 distilled / 1,218 read