Skip to content

CONCEPT Cited by 1 source

Cost-aware caching

Definition

Cost-aware caching is a cache management strategy where eviction or retention decisions account for the heterogeneous cost of cache misses across different items. Unlike uniform-cost policies (LRU, LFU) which treat all misses as equally expensive, cost-aware policies weight decisions by how much a miss actually costs — considering factors like page size, backend fetch latency, I/O expense, and operation type.

Key insight

Not all cache misses are equal. A miss on a 4 KB page that can be fetched from a fast SSD in 100 µs costs far less than a miss on a 64 KB page requiring a cross-region network round-trip. Cost-aware caching concentrates misses on cheap-to-fetch items, preserving expensive items in cache even if they're accessed less frequently.

Production evidence

In Spanner's elastic caching deployment, cost-awareness yielded: - +5.5% total cache misses (more misses overall) - +0.5% I/O cost impact (nearly all additional misses are cheap)

The 10:1 ratio between miss-rate increase and I/O-cost increase demonstrates that cost-awareness successfully shields expensive operations. (Source: sources/2026-06-25-google-optimizing-cloud-economics-with-linear-elastic-caching)

Relationship to GDSF

The Greedy Dual Size Frequency (GDSF) eviction algorithm is the classic cost-aware policy for fixed-size caches — it generalizes LRU by incorporating page size and fetch cost into eviction priority. Elastic caching achieves cost-awareness differently: through per-page TTL prediction that factors in miss cost.

Seen in

Last updated · 559 distilled / 1,651 read