PATTERN Cited by 1 source

NoSQL-to-NewSQL deprecation

Shape

A structured five-reason framework for retiring a load-bearing NoSQL store in favour of a NewSQL successor, rather than a one-off "we switched databases" migration. Each reason is an independent axis; a single axis usually isn't enough to justify deprecation, but enough of them firing together builds the org-wide case.

The five axes (Pinterest 2024):

1. Maintenance cost — version lag behind upstream, painful upgrades (Pinterest's 0.94 → 1.2 HBase upgrade took ~2 years), shrinking in-house expertise.
2. Missing functionality — distributed transactions, global secondary indexes, richer query, stronger consistency. The NoSQL substrate lacks features the business has grown to need.
3. System complexity tax — bolt-on services built above the substrate to paper over missing features (Pinterest built Ixia on top of HBase + Manas for secondary indexing; Sparrow on top of Apache Phoenix + Omid for distributed transactions). Each bolt-on becomes its own codebase, on-call rotation, and upgrade cadence.
4. Infra cost — replica-count arithmetic. HBase's primary-standby shape used 6 replicas per logical record; alternatives (TiDB, MySQL, RocksDB-based KV stores) achieve similar durability with 3 replicas. At PB-scale the cost delta is large (see concepts/replica-cost-tradeoff).
5. Community health — upstream velocity, hiring market, peer- company adoption trajectory. A store's long-term viability is not just what it does today but what the ecosystem around it looks like in five years.

Why it works

Any one reason is easy to dismiss — "we can do one more upgrade," "we can build one more bolt-on" — but a structured framework forces the organisation to evaluate all axes together and recognise compounding risk. Once version lag (axis 1) causes the bolt-ons (axis 3) to rot, and the replica count (axis 4) keeps growing, and the hiring market (axis 5) dries up, the cost of staying exceeds the cost of migrating.

The complement of this pattern is patterns/workload-specific-datastore-migration: instead of swapping the NoSQL store wholesale, decompose the workloads by access pattern (OLAP / time-series / KV / transactional) and rehome each axis first. That leaves a narrow residual of "general NoSQL with rich query + transactions + secondary index" — which NewSQL (TiDB) is well-suited to absorb.

Seen in

• sources/2024-05-14-pinterest-hbase-deprecation-at-pinterest — canonical wiki instance. Pinterest's end-of-2021 deprecation decision for HBase across ~50 clusters / ~9,000 EC2 instances / >6 PB data. The five reasons are stated explicitly and in order; the successor is TiDB for the residual general-NoSQL slot after workload-axis migrations to Druid / StarRocks (OLAP), Goku (time-series), and KVStore (key-value).

Related

• patterns/workload-specific-datastore-migration — the pre-work that makes the final consolidation feasible.
• patterns/primary-standby-wal-replication — the deployment shape Pinterest retired (axis 4: 6-replica infra cost).
• concepts/tech-debt-compounding — the dynamic behind axis 1 (version lag → painful upgrade → more lag).
• concepts/replica-cost-tradeoff — the arithmetic behind axis 4.
• concepts/distributed-transactions — the canonical missing feature cited by axis 2.
• concepts/nosql-database — the category this pattern retires from.
• systems/hbase — the canonical retired store.
• systems/tidb — the canonical NewSQL successor.