SYSTEM Cited by 1 source

Amazon S3 Intelligent Tiering¶

S3 Intelligent Tiering (IT) is an Amazon S3 storage class that automatically moves objects between access tiers based on observed access age — Frequent Access (recent) → Infrequent Access (≥30 days no access) → Archive Instant Access (≥90 days no access) — and automatically promotes back when access resumes, with no per-retrieval fee.

The defining property: cost moves with actual access, not with predicted access. The data owner is not required to forecast future access correctly to capture savings.

Why it exists¶

Cold storage classes (S3 Glacier family) extract large savings if the access prediction is correct, but carry the minimum-storage- duration tax (early deletion fees) and per-retrieval fees that negate savings if access turns out to be more frequent than expected. For data with unpredictable access patterns, the cold- storage bet is asymmetric — uncertain access loses money.

IT removes the asymmetry: storage cost decreases automatically when access decreases, and comes back up when access resumes — without penalty.

(Source: sources/2026-05-21-yelp-how-partition-access-visualizations-reduced-our-data-lake-s3-cost-by-33)

Savings curve (Yelp 2026-05-21 disclosure)¶

"Objects not accessed for 30 days decrease in cost by 40%; objects not accessed for 90 days decrease in cost by 81%. The latter approaches the cost of S3 Glacier!"

30 days no-access → −40% cost (Infrequent Access tier).
90 days no-access → −81% cost (Archive Instant Access tier).

The 81% number is the load-bearing argument for IT-by-default: it captures most of the cold-storage savings without taking the minimum-duration / retrieval-fee tax.

Tier structure (named in source)¶

Frequent Access — initial tier; same as S3 Standard.
Infrequent Access — auto-tier after 30 days no access.
Archive Instant Access — auto-tier after 90 days no access. Instant retrieval is the property that distinguishes this tier from Glacier classes — reads still complete in milliseconds, just at higher per-GB read cost than IA.

Operational properties¶

No retrieval fee — distinguishes IT from Glacier classes.
No minimum-storage-duration penalty for the tier transitions themselves (objects can move freely between tiers; minimum-storage rules apply only to the IT class as a whole).
Automatic re-promotion — accessing an object in IA / Archive Instant Access tier promotes it back to Frequent Access; no operator intervention.

The reset-the-tiering-clock failure mode¶

A subtle property: any access of an object in a deeper tier re-promotes it. "Unexpected or accidental query patterns" — debugging SQL ranging over a cold partition, an over-broad SELECT * for inspection — reset the tiering clock, undoing the savings that took 90 days to land. Yelp's 2026-05-21 disclosure documents Default Access Retention as the IAM-policy-based guard against this: data outside the access window is gated behind a restrictive bucket policy, so accidental reads can't reach it and the tiering progression is guaranteed to complete.

"Storage cost is guaranteed to decrease after the initial 30 day period of Intelligent Tiering."

— but only if accidental access is prevented at the IAM layer. See patterns/iam-policy-gated-cold-tier-access.

The PB-scale full-table-scan cost bomb¶

"For our largest tables, full table scans could add significant S3 costs by accessing PBs of data from cheap Intelligent Tiers like Archive Instant Access. This is not obvious to users who are writing SQL to inspect data!"

A SQL author querying a hot table doesn't think about per-GB read cost at the Archive Instant Access tier (~10× the per-GB read cost of Frequent Access for a comparable scan workload). For a petabyte- scale historical table, an unconstrained full-table-scan can charge millions in S3 read costs without the operator realising. Default Access Retention's IAM gate is Yelp's structural defence: queries that would otherwise just work fail with an Access Denied exception, forcing the consumer through a Terraform-PR cost-acknowledgement workflow before reading deep-tier data.

Yelp's role-of-IT in the platform strategy¶

Yelp's data-platform team uses IT as the default storage class for analytics datasets with unpredictable access patterns:

"We default to S3 Intelligent Tiering when datasets have unpredictable access patterns. Its key advantage is that cost scales down automatically with reduced access, and there is no penalty if access patterns change."

The disclosed strategy is three-tier:

Predictable, frequent access → S3 Standard (or whichever class the team has historically used).
Unpredictable access → Intelligent Tiering by default.
Beyond the expected access window → Default Access Retention (data still on IT; access requires a Terraform PR + cost acknowledgement gated by an S3 Inventory cost dashboard).