SYSTEM Cited by 1 source

Hyperfine

Hyperfine (sharkdp/hyperfine) is David Peter's Rust-written command-line benchmarking tool. It runs a command many times (configurable warmup runs + timed runs), measures wall-clock latency, reports mean / min / max / standard deviation and confidence intervals, and supports A/B comparison of multiple commands in the same invocation.

Canonical invocation shape

hyperfine --warmup 2 --runs 15 \
  'turbo-main run build --dry' \
  'turbo-branch run build --dry'

Runs two commands, 2 warmup + 15 timed iterations each, reports relative performance between them with confidence intervals.

Use in the Turborepo performance campaign

Anthony Shew's 2026-04-21 Turborepo-agents-sandboxes-humans post uses hyperfine as the end-to-end validation gate in the supervised Plan-Mode-then-implement agent loop:

1. Agent (in Plan Mode) identifies hotspot from Markdown profile output.
2. Agent implements the proposed change.
3. Hyperfine A/B compares the branch binary vs main binary end-to-end on a representative workload.
4. If the change is a real wall-clock win, PR merged.

This positioning — hyperfine as the A/B gate — is what catches the microbenchmark- vs-end-to-end gap: an agent can produce a 97 % win in a narrow microbenchmark that amounts to 0.02 % end-to-end. Hyperfine measures end-to-end wall-clock, so the agent's proposed change only merges if it moves the real-world number.

Noise-floor limitation

The post disclosed a canonical noise-floor finding: once Turborepo's hot-path functions dropped below a few hundred microseconds, hyperfine reports on Shew's MacBook became "increasingly noisy. As the code gets faster, system noise matters more." Real-world 2 % wins became indistinguishable from run-to-run variance. The fix was sandbox benchmarking — running the same hyperfine command inside a Vercel Sandbox where no Slack / Spotlight / cron / other background activity competed for CPU / disk / memory, restoring distinguishability at the couple-of-percent level. See patterns/ephemeral-sandbox-benchmark-pair.

Seen in

• sources/2026-04-21-vercel-making-turborepo-96-faster-with-agents-sandboxes-and-humans — canonical Hyperfine+Sandbox composition; hyperfine as the A/B validation gate in supervised agent performance-engineering loop.

Related

• concepts/run-to-run-variance — the measurement phenomenon hyperfine's warmup + many-runs structurally mitigates.
• concepts/sandbox-benchmarking-for-signal-isolation — the complement that keeps hyperfine reports distinguishable once code is fast enough.
• patterns/ephemeral-sandbox-benchmark-pair — the composition pattern.
• patterns/measurement-driven-micro-optimization — parent pattern.
• systems/turborepo — canonical benchmark subject.
• systems/vercel-sandbox — clean-signal host.