CONCEPT Cited by 2 sources
Storage latency hierarchy¶
Definition¶
The storage latency hierarchy is the gap — measured in round-trip time — between each tier of storage a program can reach. Each step down the hierarchy gives up ~1–4 orders of magnitude of speed in exchange for cheaper, more durable, or more scalable storage.
Canonical numbers (Dicken 2025)¶
From Ben Dicken's IO devices and latency:
| Tier | Round-trip | Why |
|---|---|---|
| CPU ↔ RAM | ~100 ns | Volatile DRAM on the memory bus. |
| CPU ↔ local NVMe SSD | ~50 μs (50,000 ns) | NAND-flash read + PCIe lane. No mechanical motion. |
| CPU ↔ network-attached SSD (EBS) | ~250 μs (250,000 ns) | Same underlying SSD; 5× penalty is the intra-DC network round-trip. |
| CPU ↔ random HDD read | ~1,000–3,000 μs (1–3 ms) | Head seek + platter rotation. |
| CPU ↔ random tape read | ~1,000,000–10,000,000 μs (1–10 s) | Cartridge must be unwound to correct position. |
Compression: 1,000× from RAM to local NVMe, 5× from local NVMe to EBS, 20–60× from EBS to HDD, 1000× from HDD to tape. The gaps compound.
(Source: sources/2025-03-13-planetscale-io-devices-and-latency.)
Why every tier survives¶
- RAM — required for the working set. Cost/GB keeps it from being the full dataset.
- Local NVMe SSD — permanent, fast, and bounded in size. The OLTP-database default when the server owns its storage.
- Network-attached SSD (EBS-class) — detachable. Survives the compute instance. Price of elasticity: a 5× latency hop and (typically) a throttled IOPS budget.
- HDD — capacity-dominant workloads that tolerate seek latency. Log-structured engines and object stores align reads to sequential sweeps (concepts/hdd-sequential-io-optimization).
- Tape — cold archive. 400 PB at CERN, AWS Storage Gateway VTL. Shelf-life beats every other medium.
Why the cloud-era jump is controversial¶
The Dicken post highlights that moving from local-NVMe to network-attached-SSD is the first time the hierarchy went backwards from a performance standpoint. Every prior step — tape → HDD → SSD → NVMe — bought latency wins. EBS-class storage trades latency for durability-across-VM- lifetime + elastic resizing. Whether that trade is worth it is the argument behind systems/planetscale-metal.
See concepts/network-attached-storage-latency-penalty for the detailed framing of the 5× hop.
Compositional implications¶
- Keep working set in RAM. At 1000× the gap to NVMe, any fetchable-from-memory operation that ends up on disk is 100× over budget. Hence buffer pools, page caches, and "add RAM" as the first line of database tuning.
- Choose the lowest tier your SLO tolerates. Ledger services live on NVMe; analytics warehouses live on HDD + object store; compliance archives live on tape.
- Replication can substitute for tier durability. The reason PlanetScale can pull databases back onto local NVMe (a medium that dies with the instance) is that 3× replication + auto-failover replaces the durability guarantee EBS used to provide. See concepts/storage-replication-for-durability + patterns/direct-attached-nvme-with-replication.
Seen in¶
- sources/2025-03-13-planetscale-io-devices-and-latency — canonical table-of-tiers with round-trip numbers + Dicken's framing of the cloud-era backward step.
- sources/2025-07-08-planetscale-caching — Ben Dicken's caching primer positions the storage-latency hierarchy "all the way up" into the CPU cache tier — same principle, same trade-offs, same engineering response. "Faster data lookup means more cost or more size limitations due to how physically close the data needs to be to the requester." Canonical wiki link between the storage-side hierarchy and the new CPU cache hierarchy — together they form the full latency stack from ~1 ns L1 to ~1 s tape.