Skip to content

CONCEPT Cited by 1 source

Temporal locality (recency bias)

Definition

Temporal locality (a.k.a. recency bias in the application-caching framing) is the property that recently accessed data is more likely to be accessed again soon than older data. At application-request scale it's the observation that traffic follows a heavy recency-biased distribution: a small slice of recent items serves most requests.

Why it makes caches work

The whole point of a cache is that past access predicts future access. Temporal locality is the substrate assumption โ€” without it, caching either degenerates to random sampling (miss-heavy) or must be driven by other signals (content-based prefetch, sticky-key routing, predictive models).

When the assumption holds, simple eviction policies like LRU exploit it directly: "LRU always evicts the item that has least-recently been requested, a sensible choice to maximize cache hits. This aligns well with temporal locality in real-world data access patterns." (Source: sources/2025-07-08-planetscale-caching.)

Ben Dicken's worked example (X.com)

From the 2025-07-08 post:

The number of tweets in the entire history of X.com is easily in the trillions. However, ๐• timelines are almost exclusively filled with posts from the past ~48 hours. After a few days, [posts] are rarely viewed unless someone re-posts or scrolls far back into their history. This produces a recency bias.

Concrete datum the post uses: a Karpathy tweet that received 43,000 likes + 7M impressions over two years โ€” but the view distribution is heavily front-loaded at the time it was posted.

Architectural consequence:

These websites store much of their content (email, images, documents, videos, etc) in "slow" storage (like Amazon S3 or similar), but cache recent content in faster, in-memory stores (like CloudFront, Redis or Memcached).

This is why viewing older posts on someone's timeline often take longer to load than recent ones! It does not make much sense to cache values that are rarely needed.

User-visible signature: scrolling deep into old posts gets slower โ€” cache misses are visible.

Generalisation beyond social feeds

The same pattern underlies:

  • Session data (active sessions reused across requests).
  • Configuration / feature flags (last-fetched value reused repeatedly).
  • Trending content (viral / news / sports pattern โ€” heavily front-loaded).
  • Database row access (recently modified rows often read back; MVCC and the InnoDB buffer pool bank on this).
  • Code execution (just-accessed code is likely accessed again โ€” the CPU cache hierarchy banks on this at the hardware level).

When temporal locality breaks down

  • Long-tail / random-access workloads โ€” random reads over a large uniformly-accessed dataset have no temporal locality; cache size โ‰ˆ dataset size is the only way to hit high rates.
  • Analytic scan workloads โ€” full table scans touch every row exactly once; no recency signal. OLTP-style caching positively hurts here (pollutes hot cache with scan data). InnoDB uses midpoint-insertion LRU specifically to avoid this.
  • Random-key workloads like crypto / hashes โ€” by design, lookups distribute uniformly; no recency.
  • Adversarial workloads โ€” a crafted request stream can evict every useful entry before it's read back.
  • Spatial locality โ€” near recent, not the same as recent; enables prefetching.
  • Pareto / Zipfian distributions โ€” the quantitative shape temporal locality often takes (top ~20% of items account for ~80% of accesses).

Seen in

  • sources/2025-07-08-planetscale-caching โ€” canonical worked example using X.com tweets + Karpathy-tweet trendline + explicit "slow S3 + fast CloudFront/Redis" + user-visible deep-scroll slowness.
Last updated ยท 319 distilled / 1,201 read