Skip to content

SYSTEM Cited by 1 source

OpenTSDB

OpenTSDB (Open Time Series Database) is an open-source time-series database built on top of HBase (historically also Bigtable). It was one of the first widely-adopted TSDBs for the Hadoop ecosystem and was Pinterest's monitoring-storage backend before migrating to the in-house Goku.

Definition

OpenTSDB stores time-series data as HBase rows, with a custom schema that compresses metric names + tags into short IDs. Features:

  • Millisecond-resolution writes.
  • Rate-based aggregations at query time.
  • Tag-based filtering via HBase row scans.
  • Downsampling and aggregation functions for long-range queries.

Scales as well as the underlying HBase cluster — which is the root of its eventual problems at very high cardinality + volume.

Pinterest's use and migration

Pinterest used OpenTSDB on HBase as its monitoring-storage backend. The HBase deprecation retrospective mentions it as one of the "in- house and open-source systems" running on HBase. For the time-series workload axis, Pinterest migrated to Goku, an in-house TSDB:

"It was not able to keep up with the ever increasing time series data volume, which led to significant challenges in scalability, performance, and maintenance load." (Source: sources/2024-05-14-pinterest-hbase-deprecation-at-pinterest)

The migration from OpenTSDB → Goku is a canonical instance of patterns/workload-specific-datastore-migration: purpose-built TSDB beats general-NoSQL-with-TSDB-schema at scale.

Seen in

Last updated · 550 distilled / 1,221 read