Skip to content

PATTERN Cited by 1 source

Mutable hotpath, immutable elsewhere

Mutable hotpath, immutable elsewhere is the optimisation pattern of selectively abandoning immutable data structures only on the critical hot path where allocation rate causes garbage collection pressure, while preserving immutability everywhere else in the codebase for safety and maintainability.

Motivation

Functional-programming best practice (especially in Scala/JVM) favours immutable data structures — every "update" creates a new object. At modest throughput this is fine; the JVM GC handles short- lived objects efficiently.

At millions of records per second, however, immutability means millions of new aggregator objects per second, each immediately discardable. The GC runs constantly:

  • Minor GCs every few seconds.
  • Major GCs taking hundreds of milliseconds.
  • More CPU spent on GC than on business logic.
  • Memory pressure → GC thrashing → processing slowdown → more memory pressure (vicious cycle).

The pattern

  1. Identify the hotpath via profiling (heap dumps, GC logs, async-profiler).
  2. Switch hotpath data structures to mutable — update aggregator fields in-place instead of creating new objects.
  3. Keep everything else immutable — non-hotpath code retains safety properties.
  4. Require measurement to justify — never switch without profiling data proving the need.
  5. Accept increased review burden — mutable code requires more careful review for concurrency bugs.

Netflix's application (2026-07-13)

Netflix Service Topology switched from immutable to mutable aggregators on the flow-log hotpath:

  • Heap allocation reduced >50%.
  • GC pause times dropped from hundreds of ms to tens of ms.
  • CPU freed up for actual business logic.
  • Instance stability improved — no more OOM-triggered cascading failures.

"This was controversial — Scala best practices emphasize immutability. But at our scale, immutability created too many objects. We pragmatically chose mutable aggregators on the hotpath (immutability elsewhere), prioritizing performance over convention. Done deliberately, with measurement justifying the decision, and with awareness of the trade-offs." (Source: sources/2026-07-13-netflix-building-service-topology-at-scale-architecture-challenges)

Decision framework

Immutable Mutable
Safety Trivially thread-safe Requires careful synchronisation
GC pressure at high rate High (new object per update) Low (in-place update)
Code clarity Easier to reason about More review needed
When to use Default everywhere Only profiling-justified hotpaths

Key principle

"Best practices are starting points, not absolute rules. At unique scale, you may need to diverge from conventions. But do it deliberately, with measurement, and with awareness of the trade- offs."

Seen in

Last updated · 585 distilled / 1,765 read