Skip to content

SYSTEM Cited by 1 source

Pinterest Zen (graph service)

Zen is Pinterest's in-house graph service — a higher-level abstraction over the underlying storage substrate that exposes a graph-shaped API (vertices, edges, neighbours, traversals) to application services.

Definition

Zen provides:

  • Graph entities (vertices and edges, typed).
  • Indexed neighbour lookups, reverse-edge queries, and traversals.
  • A graph-native query surface so application code doesn't have to deal with raw row/column storage.

Originally built on top of HBase, which is the configuration named in the HBase deprecation retrospective (Source: sources/2024-05-14-pinterest-hbase-deprecation-at-pinterest).

Why it is load-bearing in HBase-deprecation discussion

Graph updates typically touch multiple rows — writing an edge A → B requires touching at least the edge row and often both vertex rows (degree counts, adjacency lists). Without distributed transactions in the substrate, partially-failed updates can leave the graph in an inconsistent state.

Pinterest cites Zen as the canonical example of this gap:

"The lack of distributed transactions in HBase led to a number of bugs and incidents of Zen, our in-house graph service, because partially failed updates could leave a graph in an inconsistent state. Debugging such problems was usually difficult and time- consuming, causing frustration for service owners and their customers." (Source: sources/2024-05-14-pinterest-hbase-deprecation-at-pinterest)

This is axis 2 of the five-reason NoSQL-to-NewSQL deprecation framework — missing functionality — and one of the concrete production pains that justified migrating off HBase.

Successor

With HBase deprecated, Zen is expected to migrate to a substrate with native distributed transactions (the Pinterest retrospective names TiDB as the general-NoSQL successor). Post- migration architecture is not covered in Part 1 of the series; Part 3 is announced to cover the unified storage service.

Seen in

Last updated · 550 distilled / 1,221 read