Skip to content

SYSTEM Cited by 1 source

Pinterest Ixia (indexed datastore)

Ixia is Pinterest's in-house indexed datastore — a service layer built on top of HBase and Manas realtime to support global secondary indexing that HBase does not provide natively.

Definition

Ixia gives HBase-backed tables:

  • Secondary indexes over columns other than the row key.
  • Real-time indexing — writes update indexes with minimal latency.
  • Manas-backed inverted indexes under the hood for the indexing substrate.

The canonical description on the wiki (per the HBase deprecation retrospective): "we built Ixia on top of HBase and Manas realtime to support global secondary indexing in HBase" (Source: sources/2024-05-14-pinterest-hbase-deprecation-at-pinterest).

Separate Pinterest Engineering write-ups on Ixia ("Building Scalable Near-Real-Time Indexing on HBase") and Manas realtime ("Enabling changes to be searchable in a blink of an eye") exist but are not ingested on this wiki yet.

Why it matters in the HBase-deprecation case

Ixia is the canonical bolt-on example Pinterest cites for the "system complexity" axis of NoSQL-to-NewSQL deprecation:

"To provide these advanced features for customers, we built several new services on top of HBase over the past few years. For example, we built Ixia on top of HBase and Manas realtime to support global secondary indexing in HBase. [...] While we had no better alternatives to satisfy the business requirements back then, these systems incurred significant development costs and increased the maintenance load."

Each bolt-on service is its own codebase, on-call rotation, and upgrade cadence — layered over the same underlying HBase that itself needs maintenance. When the substrate gains the feature natively (secondary indexes in TiDB), the bolt-on can be retired and the complexity tax reclaimed.

Seen in

Last updated · 550 distilled / 1,221 read