CONCEPT Cited by 1 source

Bandwidth per terabyte (BW/TB)

Definition

Bandwidth per terabyte (BW/TB) is the ratio of a drive's sustainable read/write throughput (in MB/s) to its usable capacity (in TB). Unlike raw throughput, BW/TB tells you how much bandwidth each byte of stored data has available — the single most predictive metric for whether a workload will fit on a given media tier.

BW/TB = drive_throughput_MB_per_s / drive_capacity_TB

Low BW/TB = lots of cold data per unit of I/O. High BW/TB = lots of I/O per unit of data.

Why this metric matters at hyperscale

At consumer scale, drive throughput (MB/s) and drive capacity (TB) are considered independently. At hyperscale, workloads have characteristic BW per TB stored profiles — e.g., a cold object store might need 5 MB/s/TB, a log tail might need 50 MB/s/TB, a hot cache 200+ MB/s/TB. You match workloads to media by aligning those numbers.

Meta's 2025 storage-media framing uses BW/TB as the organising axis:

Media tier	BW/TB range	Workload
HDD (16-20 TB class, 2020s)	~5-10 MB/s/TB (falling)	Cold bulk / archival
QLC flash	~10-20 MB/s/TB	Read-BW-intensive, batch IO, low-write
TLC flash	~50+ MB/s/TB	Mixed / write-heavy / latency-sensitive

HDD BW/TB is falling over time

Meta's 2025-03-04 framing:

"HDDs are growing in size, [but] they haven't been growing in I/O performance. In other words, the bandwidth per TB for HDDs has been dropping. This has been forcing data center engineers to meet their storage performance needs by shifting hot (frequently accessed) data to a TLC flash tier or by overprovisioning storage."

Same underlying physics as the flat-IOPS observation — head seeks don't scale with areal density — seen through a different axis. An HDD that was 6 TB / 150 MB/s had 25 MB/s/TB; a 30 TB drive at the same 150 MB/s/sec is 5 MB/s/TB. Same drive-level throughput, six times less BW per TB stored.

Stranded data

The BW/TB decline in HDD strands hot workloads on a cold tier:

"This has negatively affected a portion of hot workloads and forced bytes to get stranded on HDDs."

Two mitigations before the QLC tier:

1. Promote the hot data to TLC. Expensive if it's a lot of data.
2. Overprovision HDDs — buy more drives than capacity requires so IOPS add up. Defeats the capacity/$ purpose of HDDs.

Both are bad. A middle tier (QLC) whose BW/TB sits between HDD and TLC fixes the gap without either cost.

Reads and writes can have different BW/TB

QLC's read throughput is 4×+ its write throughput. So its BW/TB is a two-number tuple: read BW/TB is materially higher than write BW/TB on the same drive. Workload-media matching must consider both.

Architectural implications

• Media tiers can be defined by BW/TB bands. A workload with a 15 MB/s/TB steady-state BW requirement doesn't fit HDD (5 MB/s/TB) and doesn't need TLC (50+); it's a QLC workload.
• Drive density is a BW/TB-destroying force. Each density generation preserves throughput but drops BW/TB. Historical capacity-scaling has tacitly pushed workloads up the tier stack.
• New media tiers are inserted when the BW/TB gap between existing tiers exceeds what mixed-media racks can paper over. See patterns/middle-tier-storage-media.
• Tiering is not purely hot/cold; it's also workload-BW/TB-shape matching.

Seen in

• sources/2025-03-04-meta-a-case-for-qlc-ssds-in-the-data-center — Meta's canonical BW/TB framing, with explicit 10 MB/s/TB + 15-20 MB/s/TB bands and the HDD-BW/TB-falling observation.

Related

• concepts/hard-drive-physics — the underlying physics; BW/TB is one axis of the same observation, flat IOPS is another.
• concepts/qlc-read-write-asymmetry — two-number BW/TB for QLC (read vs write).
• concepts/storage-media-tiering — the tier structure BW/TB organises.
• concepts/hdd-sequential-io-optimization — sequential IO preserves more throughput but not BW/TB as capacity scales.
• systems/qlc-flash / systems/tlc-flash — the tiers on either side of QLC's band.
• patterns/middle-tier-storage-media.