CONCEPT Cited by 1 source

Lucene changes snapshot

The Lucene changes snapshot is a shard-local API in OpenSearch that provides an operation-level changelog — a sequence of index, delete, and no-op operations since a given sequence number. It is the OpenSearch equivalent of a database's CDC stream or WAL replay.

Mechanics

• Each operation gets a monotonically increasing sequence number.
• The snapshot returns operations between two sequence numbers (from retained history).
• Combined with retention leases, it guarantees no gaps in the stream.
• Operations include the full document source for inserts/updates, but deletes carry only the _id (not routing keys), which creates challenges for custom-routed indices.

Role in AOSC

AOSC uses the changes snapshot as its "CDC stream" for incremental replay during online index migration. The shard migration worker repeatedly reads the snapshot, replays operations to the target, and advances the retention lease until the gap falls below the convergence threshold (default: 500 ops).

Limitation

Translog.Delete entries do not include the original routing key. For indices using custom routing, AOSC must synthesize routing keys or fan deletes across target shards.

Seen In

• systems/opensearch-aosc — primary consumer of the changes snapshot for shard-level replay
• sources/2026-07-24-atlassian-online-index-migration-and-shard-scaling-in-opensearch-with-aosc

(Source: sources/2026-07-24-atlassian-online-index-migration-and-shard-scaling-in-opensearch-with-aosc)