Skip to content

SYSTEM Cited by 1 source

libketama

libketama is last.fm's open-source consistent-hashing library for Memcached clients, released in 2007. It's one of the earliest productionised implementations of consistent hashing in the open-source ecosystem and is built into most Memcached client libraries across languages as the default partitioning scheme (Source: sources/2023-02-22-highscalability-consistent-hashing-algorithm).

Why it matters

Before libketama, Memcached clients typically used static hash partitioning (hash(key) mod N) to pick a server. Adding or removing a Memcached node caused mass cache invalidation — most keys re-mapped, most requests became cache misses, and the backing store got overwhelmed. libketama replaced this with a consistent-hashing ring plus virtual nodes (the library calls them "replicas", typically 40 per server). The cache-invalidation blast radius dropped from ~100% of keys to ~1/N.

Canonical reference

The 2007 announcement post "libketama — a consistent hashing algo for memcache clients" by Richard Jones is the citation for this system on the wiki.

Use today

Most major Memcached client libraries (pylibmc, php-memcached, spymemcached, mcrouter) default to ketama-compatible hashing. "Ketama compatible" means the library produces the same key → server mapping given the same server list, so multiple clients in different languages partition the same cluster consistently.

Wiki scope

Stub page anchored to named use in sources/2023-02-22-highscalability-consistent-hashing-algorithm. The library itself is small (hundreds of lines of C); its significance is historical and ecological — it defined the de-facto standard for Memcached consistent-hashing client partitioning.

Seen in

Last updated · 517 distilled / 1,221 read