SYSTEM Cited by 1 source

Rocksplicator

Rocksplicator is Pinterest's open-source real-time data replication layer for RocksDB. It is the replication substrate underneath KVStore, Pinterest's in-house KV store that replaced HBase for KV workloads.

Definition

Rocksplicator handles:

• Real-time replication between RocksDB instances — near-online consistency, not batch snapshot.
• Automatic leader election and failover among replicas.
• Cluster management (shard assignment, rebalancing).

It is open-sourced by Pinterest ("Open-Sourcing Rocksplicator: A Real-Time RocksDB Data Replicator", 2019 Pinterest Engineering post; the post is linked from the HBase-deprecation retrospective but is not itself ingested on the wiki).

Role at Pinterest

Rocksplicator is the "R" in Pinterest's KV stack:

• RocksDB provides the single-node LSM-tree KV engine.
• Rocksplicator provides the cross-node replication.
• KVStore composes these into a production KV service with its own API and orchestration.

This layering was cited in the HBase-deprecation retrospective as "more performant and infra efficient" than HBase for KV workloads (Source: sources/2024-05-14-pinterest-hbase-deprecation-at-pinterest).

Seen in

• sources/2024-05-14-pinterest-hbase-deprecation-at-pinterest — named as the replication substrate underneath Pinterest's KVStore, which replaced HBase for KV workloads.

Related

• systems/rocksdb — the local KV engine Rocksplicator replicates.
• systems/pinterest-kvstore — the KV service Rocksplicator is part of.
• concepts/wal-replication — the general replication-via-log primitive Rocksplicator is one instance of.
• companies/pinterest.