Skip to content

CONCEPT Cited by 1 source

Pagecache for messaging

Definition

Pagecache is the OS kernel's in-memory cache of disk pages — any file read (first time) pulls the page into pagecache, so subsequent reads of the same page are served from RAM without touching the drive. Write-back caching is the symmetric flip side (writes land in pagecache, the kernel flushes asynchronously).

In messaging systems whose data structure is a distributed log — canonical example Apache Kafka — the tail of the log (the most recent records) is exactly where most consumers read from. Because the tail is also the most-recently-written, it is still hot in pagecache, so tail-reads serve from RAM without application involvement. This is the load-bearing reason Kafka can write all records to disk and still be fast.

Kafka's framing

Kozlovski's Kafka-101 explainer:

"Modern OSes cache the disk in free RAM. This is called pagecache. Since Kafka stores messages in a standardized binary format unmodified throughout the whole flow (producer ➡ broker ➡ consumer), it can make use of the zero-copy optimization. Zero-copy, somewhat misleadingly named, is when the OS copies data from the pagecache directly to a socket, effectively bypassing Kafka's JVM entirely." (Source: sources/2024-05-09-highscalability-kafka-101)

Kozlovski also underwrites how write-side optimisations land in pagecache rather than hitting disk synchronously:

"Write-behind optimizations group small logical writes into big physical writes — Kafka does not use fsync, its writes get written to disk asynchronously."

So both sides of the log access pattern compose with pagecache:

  • Produce: record lands in pagecache, returned to the caller quickly; kernel async-flushes.
  • Consume (tail): record served from pagecache; no disk hit.
  • Consume (cold historical): page miss; HDD seek + read-ahead.

Consequences

  • Memory sizing: rightsizing a Kafka broker's RAM is really rightsizing pagecache. Broker heap need not be large; the JVM is not caching the log.
  • Noisy-neighbor within pagecache: historical readers that scan far behind the tail evict fresh tail pages, cooling the pagecache for other consumers — this is the Kafka-101 motivation for Tiered Storage (historical reads from the object store, not local HDD through pagecache).
  • Security trade-off: encryption/TLS ("a must for all production deployments" — Kozlovski) prohibits sendfile-style zero-copy because the OS would be copying ciphertext and the CPU has to be involved to decrypt/re-encrypt — in practice the pagecache-based zero-copy path is less load-bearing than Kafka- performance folklore suggests.

Seen in

Last updated · 319 distilled / 1,201 read