PATTERN Cited by 2 sources

Data-placement spreading

Pattern

For each new object, choose a different subset of drives to place it on — even for objects owned by the same tenant, in the same bucket. The result: a single tenant's data is scattered across the majority of the fleet, rather than concentrated on a small set of drives.

Two benefits (Warfield, 2025)

1. A customer's data only occupies a very small amount of any given disk, which helps achieve workload isolation, because individual workloads can't generate a hotspot on any one disk.
2. Individual workloads can burst up to a scale of disks that would be really difficult and really expensive to build as a stand-alone system.

(1) Per-disk workload isolation

If a tenant's data spans millions of drives, no drive holds enough of the tenant's objects for that tenant's burst to create a single-drive hotspot. The per-drive load from any given customer is tiny; only aggregate load matters.

(2) Customer burst access to the whole fleet

The post's headline number:

Today, we have tens of thousands of customers with S3 buckets that are spread across millions of drives. […] That burst of requests can be served by over a million individual disks. That's not an exaggeration.

A stand-alone system trying to match that burst profile would have to provision a million-disk fleet for one customer. In S3, it falls out of the placement policy — the customer gets the whole fleet on their burst, everyone else is unaffected because the burst is a vanishing fraction of per-drive load.

Interactions

• With concepts/aggregate-demand-smoothing: spreading is what translates smooth aggregate demand into smooth per-drive demand. Without spread, aggregate smoothing wouldn't help individual drives.
• With concepts/erasure-coding: individual objects are split into (k+m) shards that might span tens of drives. The different objects on different drive sets rule applies at the object level, not the shard level.
• With patterns/redundancy-for-heat: once objects are spread, redundancy provides the steering degree-of-freedom within the spread.
• With concepts/noisy-neighbor: the EBS lesson is that "spread a hot tenant widens blast radius." Spread works in S3 precisely because concepts/aggregate-demand-smoothing makes any one customer's impact-per-drive negligible. The patterns differ because the scale and workload mix differ.

Preconditions

• Large enough fleet that any one customer is a small fraction of total drives.
• Large enough tenant population that aggregate demand smooths.
• Metadata layer that can resolve many-drive placements efficiently (object → drive-set lookup is on the hot path).

Failure modes

• Correlated workloads across tenants (e.g., everyone hits the top-of-hour batch) can still cause aggregate peaks.
• Very large single tenants can erode the aggregation assumption. S3's customer mix has historically stayed broad enough that this doesn't dominate.
• Small buckets' tax: a bucket with few objects can't fully benefit — the spread is bounded by object count. The post doesn't address this explicitly but implies it's why S3's placement is "across lots of disks" rather than "all disks."

Seen in

• sources/2025-02-25-allthingsdistributed-building-and-operating-s3 — spread-placement framed as the mechanism that lets a bursting Lambda-parallel workload use over a million disks.
• sources/2024-03-06-highscalability-behind-aws-s3s-massive-scale — Kozlovski enumerates four benefits of spread placement in a single list — (1) hot-spot aversion, (2) burst demand parallelized across many drives, (3) greater durability via more machine-failure tolerance, (4) no read amplification on any single drive. Cleaner decomposition than the Warfield narrative; useful as a teaching-level summary of why spread placement is multi-objective (not just heat management).