Skip to content

CONCEPT Cited by 1 source

Elastic caching

Definition

Elastic caching reframes the cache management problem from "what to evict when the cache is full?" to "how long should each item stay in the cache?" Instead of maintaining a fixed-size cache with an eviction policy (LRU, LFU, GDSF), elastic caching assigns a per-item time-to-live (TTL) that balances the ongoing memory cost of caching against the one-time cost of a cache miss. The cache size becomes a dynamic outcome of individual TTL decisions rather than a fixed constraint.

This formulation maps the per-item caching decision onto the ski-rental problem: keep paying rent (memory cost per time unit) or buy (accept a miss and re-fetch). The ski-rental problem has a known 2-competitive optimal strategy, providing theoretical guarantees on the cost efficiency of the approach.

Key properties

  • Variable cache size: the cache grows and shrinks based on workload characteristics rather than being constrained to a fixed allocation.
  • Cost-aware: miss costs, page sizes, and access patterns all factor into the TTL decision.
  • Per-item granularity: different items get different retention times based on their individual cost profiles.
  • Linear cost model: "linear" elastic caching assumes memory cost is proportional to (size × time), making the ski-rental mapping exact.

Production results (Spanner)

Deployed in Google's Spanner at billions of requests per second: - Memory: −15.5% - Cache misses: +5.5% (concentrated on cheap-to-fetch pages) - TCO: −~5% - I/O cost impact: +0.5%

(Source: sources/2026-06-25-google-optimizing-cloud-economics-with-linear-elastic-caching)

Seen in

Last updated · 559 distilled / 1,651 read