PATTERN Cited by 1 source
Soft CPU partitioning by thread priority¶
Pattern¶
Divide available CPUs into two (or more) logical pools — one for latency-critical request-path threads, one for background/housekeeping threads — and dynamically adjust pool sizes based on load. "Soft" because the partition is heuristic and adaptive, not a hard cpuset constraint.
Mechanism¶
The scheduler policy knows which threads are latency-critical (via cgroup, thread naming, or annotation). On wake-up, it dispatches critical threads to the "fast pool" and background threads to the "slow pool." Load-based heuristics resize pools dynamically (e.g., under high request load, grow the fast pool at the expense of the slow pool).
Benefits¶
- L3 cache locality: Latency-critical threads stay on the same set of CPUs over time, reducing cold-cache DRAM accesses.
- Reduced interference: Background work (logging, GC, metrics collection) doesn't preempt request-path threads.
- Better power efficiency: Fewer context switches and cache misses → fewer wasted cycles → power savings at fleet scale.
Production evidence¶
Meta's ads serving fleet: soft CPU partitioning via systems/sched-ext → 28% p99 latency reduction + 3.28 MW power savings on the initial launch. The L3 locality improvement is explicitly named as a primary mechanism. (Source: sources/2026-07-13-meta-modernizing-ads-service-open-source-kernel-scheduler)
Contrast with hard partitioning¶
| Approach | Pros | Cons |
|---|---|---|
| Hard cpuset | Simple, no scheduler changes | Static, wastes CPUs under low load |
| Soft partition (sched_ext) | Adaptive, cache-friendly, no wasted CPUs | Requires scheduler policy, more complex |
| No partitioning (CFS/EEVDF) | Fair, simple | Critical threads preempted by background work |
See also¶
- concepts/workload-specific-kernel-scheduling — the broader concept
- patterns/bpf-scheduler-for-workload-specific-optimization — the deployment pattern
- concepts/tail-latency-at-scale — the problem this pattern addresses