Skip to content

SYSTEM Cited by 1 source

Pinterest KVStore

KVStore is Pinterest's in-house key-value store built on RocksDB for local storage and Rocksplicator for real-time data replication. It is the successor to HBase for Pinterest's KV workloads.

Definition

KVStore architecture (per the public references):

  • RocksDB — LSM-tree KV engine for on-disk state on each node.
  • Rocksplicator — Pinterest's open-source real-time data replication layer for RocksDB; handles log-shipping / mirror- maintenance across nodes.
  • Purpose-built for KV workloads — unlike HBase, which was a general NoSQL substrate, KVStore is scoped narrowly to the key-value access pattern and tuned accordingly.

A separate Pinterest Engineering post — "3 Innovations while Unifying Pinterest's Key-Value Storage" — describes KVStore's architecture in depth; that post is linked from the HBase deprecation retrospective but not ingested on the wiki yet.

Why it replaces HBase for KV

From the HBase deprecation post:

"It was also not as performant or infra efficient as compared to KVStore, an in-house key-value store built on top of RocksDB and Rocksplicator." (Source: sources/2024-05-14-pinterest-hbase-deprecation-at-pinterest)

This is a canonical instance of workload-specific datastore migration: instead of keeping all KV workloads on HBase, Pinterest rehomed them to a purpose-built KV store. The gains:

  • Performance — LSM-tree KV engine directly tuned for KV access, rather than a wide-column abstraction on top of a KV layer.
  • Infra efficiency — fewer JVM layers, less operational overhead, smaller replica-count multiplier relative to HBase's primary- standby shape.

Seen in

Last updated · 550 distilled / 1,221 read