Skip to content

SYSTEM Cited by 1 source

Google Cloud Spanner

Google Cloud Spanner is Google's managed, globally-distributed, strongly-consistent relational database — the commercial product derived from Google's internal Spanner research (Corbett et al., OSDI 2012). It combines SQL semantics with automatic horizontal sharding, synchronous cross-region replication via Paxos, and external consistency via TrueTime.

Stub page — expand on future Spanner-internals sources. Canonical wiki reference so far is the 2025-03 Redpanda CDC-connectors post which introduces Spanner's change streams feature as one of four engine-native CDC mechanisms surfaced via Redpanda Connect's gcp_spanner_cdc input connector.

Change streams (CDC)

Spanner change streams are Spanner's native CDC mechanism: a named change stream is defined on a database, table, or column set, and emits change records (insert / update / delete with before/after column values) ordered within a partition. Redpanda Connect's gcp_spanner_cdc connector canonicalises three distinguishing operational properties:

  • Transactional progress storage. "Stores progress transactionally in a configurable spanner table for at least once delivery" — the CDC consumer's offset is stored in Spanner itself as a transactional row, not in an external cache. This is structurally distinct from the MySQL / MongoDB external-offset-store shape and from Postgres's server-owned replication slot.
  • Dynamic partition split/merge handling. "Automatically processes partitions that are merged and split, avoiding hotspots." Spanner's change stream partition topology can change over time as the underlying table's sharding evolves; the connector must reconcile partition identity across splits/merges without duplicating reads or missing records. This is the structural distinguishing feature from Postgres / MySQL / MongoDB CDC where the change-log topology is static.
  • Configurable retention + filters. "Custom change data retention, table modification filters and value types."

(Source: sources/2025-03-18-redpanda-3-powerful-connectors-for-real-time-change-data-capture)

Seen in

Last updated · 470 distilled / 1,213 read