Skip to content

CONCEPT Cited by 1 source

TTL-based cache sizing

Definition

TTL-based cache sizing is the technique of controlling cache memory consumption by assigning per-item time-to-live values rather than setting a fixed maximum cache size. Items expire after their TTL and are removed, so the cache's aggregate size is an emergent property of the TTL distribution across all cached items.

This inverts the traditional caching model: - Traditional: fix the cache size → choose an eviction policy to decide what to remove when full. - TTL-based sizing: assign per-item TTLs → the cache size fluctuates as items naturally expire.

Advantages

  • No eviction storms: items expire individually on their own schedule rather than being batch-evicted under memory pressure.
  • Cost-optimal sizing: TTLs can be set to the theoretically optimal retention period (e.g., ski-rental breakeven point) without being constrained by a size budget.
  • Per-item granularity: high-value items can have long TTLs while low-value items expire quickly, impossible with a single eviction threshold.

Production evidence

In Spanner's elastic caching, TTLs are predicted by a lightweight decision tree that considers page size, miss cost, and operation type. The resulting variable cache size uses 15.5% less memory than the fixed-size baseline while maintaining comparable I/O performance. (Source: sources/2026-06-25-google-optimizing-cloud-economics-with-linear-elastic-caching)

Seen in

Last updated · 559 distilled / 1,651 read