Skip to content

SYSTEM Cited by 8 sources

PostgreSQL

What it is

PostgreSQL ("Postgres") is an open-source object-relational database system, first released in 1986 and continuously developed since. As of 2025 the codebase is >1M lines of C, with thousands of contributors.

Why it shows up on this wiki

Postgres is the query-processing substrate for systems/aurora-dsql: DSQL reuses Postgres's parser and planner and replaces replication, concurrency control, durability, storage, and session management via Postgres's public extension API. This is a canonical example of the patterns/postgres-extension-over-fork pattern — building a radically different system on top of Postgres without forking its codebase.

Extension points matter because they:

  • Let the extension code run in the Postgres process, but live in separate files/packages.
  • Allow the extension to change behavior without touching core Postgres code.
  • Preserve the ability to pull upstream improvements and new features.

(Source: sources/2025-05-27-allthingsdistributed-aurora-dsql-rust-journey.)

Seen in

  • sources/2025-05-27-allthingsdistributed-aurora-dsql-rust-journey — Aurora DSQL uses Postgres via extensions rather than forking it; extensions written in Rust for concepts/memory-safety.
  • sources/2026-04-20-databricks-take-control-customer-managed-keys-for-lakebase-postgres — systems/lakebase is Databricks' Postgres-compatible serverless OLTP service descended from Neon; externalises page and WAL storage into systems/pageserver-safekeeper for compute/storage separation. Different structural answer to DSQL's "extend, don't fork" idiom for the same "make Postgres scale-to-zero" problem.

  • sources/2026-03-23-datadog-debugging-postgres-upsert-wal — Datadog diagnoses a 2× IOPS / 4× WAL-sync regression on a host_last_ingested upsert table that was deliberately engineered for HOT updates (dedicated narrow table, last_ingested unindexed, fillfactor=80). Using the Postgres 15+ pg_walinspect extension plus an lldb breakpoint on WALInsertLockAcquire, they prove that INSERT ... ON CONFLICT DO UPDATE locks the conflicting row before evaluating the WHERE clause — the lock mutates tuple metadata with a transaction ID, the xid forces a Transaction COMMIT WAL record, and the COMMIT forces an fsync. At the planned 25,000 upserts/sec scale this would exceed the cluster's single-writer fsync budget (~1,000 8-KiB fsyncs/sec on gp3 EBS per pg_test_fsync). Fix: rewrite as a patterns/cte-emulated-upsert (WITH insert_attempt AS (INSERT ... ON CONFLICT DO NOTHING RETURNING ...) UPDATE ... WHERE ... AND NOT EXISTS (SELECT FROM insert_attempt)) — no implicit lock, the common "row exists and is fresh" path emits zero WAL records. Trade-off: a small concurrent-delete race between the insert attempt and the update, accepted because host-liveness tracking is inherently imprecise. Canonical worked example of concepts/wal-write-ahead-logging as both the cost ceiling and the observability surface.

  • sources/2025-05-03-aws-postgresql-transaction-visibility-read-replicas — AWS's response to Jepsen's 2025-04-29 RDS-Postgres Multi-AZ analysis disclosing that community Postgres's visibility order can diverge from its WAL commit order (the commit path writes the WAL record then asynchronously removes the xid from the in-memory ProcArray), admitting the Long Fork anomaly — a violation of Snapshot Isolation's atomic-visibility property in which two readers on different nodes can observe concurrent non-conflicting transactions in different orders. Affects all Postgres isolation levels (Read Committed / Repeatable Read / Serializable) because all three take snapshots against ProcArray. Absent in Single-AZ Postgres, systems/aurora-limitless, and systems/aurora-dsql (both of which replace ProcArray-based visibility with time-based MVCC via Postgres-extension surgery — see patterns/postgres-extension-over-fork). Known on pgsql-hackers since 2013. Proposed upstream fix is Commit Sequence Numbers — stamp each commit with a monotonic CSN so snapshot-by-CSN-watermark makes visibility order = commit order; multi-patch effort, PGConf.EU 2024 talk, AWS PostgreSQL Contributors Team participating. See concepts/visibility-order-vs-commit-order for the generalized framing. Also surfaces the ProcArray-scan CPU cost — measurable fraction of CPU at thousands of connections on large Postgres servers in read-heavy workloads, so the fix is both a correctness and a perf win. Load-bearing against five classes of enterprise capability: distributed-SQL consistent snapshots, read-write splitting, snapshot-then-replay data sync, PITR-to-LSN, and tuple-xid replacement by logical/clock-based commit time.

  • sources/2026-04-21-figma-how-figmas-databases-team-lived-to-tell-the-scale — Figma's in-house concepts/horizontal-sharding effort on top of RDS Postgres (2022→). Post names three production Postgres ceilings driving the horizontal-sharding decision: (1) vacuums at several-TB / billions-of-rows tables producing user-visible reliability impact (vacuums are essential for avoiding transaction-ID exhaustion — the fundamental cost of MVCC at scale); (2) per-table write rates approaching RDS's per-instance IOPS ceiling; (3) CPU on the hottest vertical partition. Demonstrates that Postgres views are a viable logical-shard representation against a still-unsharded physical instance (patterns/sharded-views-over-unsharded-db) with validated <10% worst-case overhead vs direct-table access — the primitive that lets horizontal-shard semantics be feature-flag rolled out + rolled back in seconds before the riskier physical 1→N failover. Shipped first horizontally-sharded table September 2023; 10 seconds partial primary availability, no replica impact. DBProxy is the Go router + query engine that implements the sharded-query subset. An alternative to systems/aurora-limitless and systems/aurora-dsql's "extend Postgres via extensions to make it scale" approach — Figma instead extends Postgres via views + an external router, keeping community Postgres unmodified. Canonical companion to patterns/postgres-extension-over-fork on the same underlying substrate question.

  • sources/2026-04-21-figma-keeping-it-100x-with-real-time-data-at-scale — Figma's LiveGraph real-time data-fetching service taps the Postgres WAL logical replication stream as its CDC source — the same stream that keeps replicas up-to-date. LiveGraph's old one-server-in-memory-mutation-based-cache design required a globally-ordered replication stream (reasonable for a single primary Postgres); under vertical partitioning the order broke and a stopgap artificially combined per-shard streams, and the design couldn't survive horizontal sharding via DBProxy. LiveGraph 100x decouples by sharding the stateless invalidator the same way as the physical DBs and letting each shard's invalidator tail its own WAL stream independently — a canonical example of how moving from a single-primary to a sharded-Postgres world forces downstream services to drop global-order assumptions baked into the WAL-consumer contract.

  • sources/2025-10-12-mongodb-cars24-improves-search-for-300-million-users-with-atlas — named explicitly as Cars24's legacy RDBMS "for managing and searching data" before the migration to MongoDB Atlas. Cars24's shape was the canonical Postgres + separate bolt-on search engine + sync pipeline topology the synchronization-tax concept names; patterns/consolidate-database-and-search captures the remediation Cars24 chose. Not a knock on Postgres itself — Postgres has full-text search ([tsvector] / GIN) and pgvector that implement the same consolidation in-place; Cars24 chose vendor-swap rather than in-place consolidation.

  • sources/2025-11-04-datadog-replication-redefined-multi-tenant-cdc-platform — Postgres as the CDC source of a managed multi-tenant replication platform. The source-side 7-step runbook (wal_level=logical, replication-permissioned users, publications + slots, Debezium instances, Kafka topics, heartbeat tables for slot LSN advancement, sink connectors) is the canonical wiki operational reference for Postgres logical replication as a CDC source. Motivating incident: a Metrics Summary page joining 82K × 817K rows on a shared Postgres hit p90 ~7 s, resolved by rerouting search + faceted filtering to a dedicated search engine populated by Postgres → Debezium → Kafka → sink-connector replication (lag ~500 ms, page-load ~30 s → ~1 s). Canonical instance of the split answer to the database-and-search problem (patterns/debezium-kafka-connect-cdc-pipeline) vs Cars24's consolidate answer via Atlas Search.

  • sources/2026-04-16-cloudflare-deploy-postgres-and-mysql-databases-with-planetscale-workers — Postgres as the target wire protocol for the Cloudflare × PlanetScale integration. Workers open a standard pg client against env.DATABASE.connectionString (the binding exposed by systems/hyperdrive), which in turn pools + caches against a PlanetScale Postgres origin. No Postgres-specific adaptation in Workers — the binding preserves the Postgres wire protocol end-to-end, so any Postgres client / ORM works unchanged. Motivates the edge-compute-against-central-SQL latency hazard captured in concepts/edge-to-origin-database-latency and resolved via patterns/explicit-placement-hint. Ancillary: pgvector called out explicitly as the AI-workload reason Postgres specifically matters for Workers apps.

Last updated · 200 distilled / 1,178 read