Skip to content

GITHUB 2026-10-06

Read original ↗

Building Git infrastructure for agent-scale development

Summary

GitHub is rebuilding its Git serving infrastructure to handle agent-scale development — repositories where large engineering teams and fleets of coding agents commit, merge, and fetch concurrently at a sustained rate few repos reach today. The current architecture (Spokes) stores each repository as full copies on the local disks of ~5 fileservers, using the same replicas for both durability and read scale via a quorum three-phase commit on ref updates. That coupling is the ceiling: adding read replicas slows writes (every replica participates in every push), losing a replica cuts read capacity, and losing quorum halts writes entirely. The new design separates durability from scale — authoritative data moves to Azure Blob Storage, reads come from lightweight stateless cache workers, and the push critical path shrinks to just the ref update that genuinely needs coordination. Internal benchmarks show up to 35× higher write throughput with independently scaling reads. (Source: sources/2026-10-06-github-building-git-infrastructure-for-agent-scale-development)

Why now: agent-scale workload numbers

The post opens with hard growth numbers that define the target workload (August/September 2026 vs a year earlier):

  • Total Git activity rose to >2× its previous level — from 218.2 B events/month (Sep 2025) to 473.3 B events/month (Aug 2026).
  • Commits: 7.38 B commits in September alone — >5× a year earlier.
  • Pushes: grew 4.9× YoY, from 0.69 B to 3.35 B per month.
  • PR merges: grew to nearly 4× a year ago.
  • GitHub Actions runs: 3.26 B in September — >4× a year earlier.
  • The busiest single repository saw ~1 billion requests in August.

The distribution is extremely long-tailed: the gap between a typical repo and the busiest ones is "wider than most people expect," and the architecture is being built for that leading edge.

Key takeaways

  1. Agent loops make commit turnaround a per-agent bottleneck. An agent in a tight loop commits/checkpoints after nearly every action, so its speed is bounded by how fast a single push completes. Latency a human would never notice becomes the limiting factor — a critical-path argument applied to the push itself. (Source: sources/2026-10-06-github-building-git-infrastructure-for-agent-scale-development)

  2. Thousands of agents converge write pressure onto one point. Agents on their own branches in one repo produce a sustained write rate that "converges on a single point in our architecture." Trunk-based development, release trains, and merge queues funnel every merge onto a single ref — a structural hot key that must absorb all contention.

  3. Each push multiplies into thousands of reads. CI and code scanning clone or fetch the same branch tip thousands of times per minute; that read fan-out "has to be cheap." Reads are the easy half (add caches/replicas); writes are the hard half — every push must be stored durably and made visible consistently before the next agent or CI job builds on it.

  4. The old architecture coupled durability and scale, and that is the ceiling. Spokes keeps a full repo copy on ~5 fileservers' local disks; the on-disk copies ARE the source of truth, so adding read capacity means adding another durable replica, and every replica participates in every write → "a push is only as fast as the slowest replica in its set." Adding read replicas makes writes slower; losing one cuts reads; losing quorum stops writes. (Source: sources/2026-10-06-github-building-git-infrastructure-for-agent-scale-development)

  5. Design tenet 1 — minimize coordination. Only the reference update itself truly needs agreement. Storing objects, validating object connectivity, and secret scanning are much more work but can run in parallel to other writes. Shrinking the critical path of a push to the small coordinated step means the rest no longer delays the acknowledgment. This is coordination-avoidance applied to Git semantics.

  6. Design tenet 1 (cont.) — move maintenance off the serving path. Compaction and garbage collection are the heaviest work a repo does and today run on the same hosts that answer live Git requests. The new design runs separate maintenance workers directly against durable storage, so a busy repo can be optimized continuously in the background without slowing pushes/fetches.

  7. Design tenet 2 — decouple storage from compute. Four stated benefits: (a) scale reads without adding durable copies — read capacity comes from lightweight caching workers over one durable store; (b) let each layer do one job — authoritative data in Azure Blob Storage (durability + replication at Azure scale), compute optimized for throughput at lowest latency; (c) recover faster from failures — losing a compute worker is "closer to a cache miss" (a replacement serves immediately and fills its cache from durable storage as traffic arrives) rather than rebuilding a full repo copy; (d) match capacity to demand — compute workers scale with traffic (elasticity), so a release burst or new agent fleet gets extra capacity for the burst only.

  8. Preserve the controls teams rely on while raising the floor for everyone. Branch protections, required reviews, audit logs, repository visibility, dependable automation, and observability must survive the migration. The rebuild happens live — "no maintenance window where the world's code stops moving."

  9. Result: up to 35× write throughput, reads scale on their own. Internal benchmarks report up to 35× higher write throughput with read capacity that scales independently and graceful failure recovery. A follow-up post will detail the future architecture.

Systems / concepts / patterns extracted

Operational numbers

Metric Value Window
Git events/month 218.2 B → 473.3 B (>2×) Sep 2025 → Aug 2026
Commits 7.38 B (>5× YoY) Sep 2026
Pushes/month 0.69 B → 3.35 B (4.9×) YoY
PR merges ~4× YoY
GitHub Actions runs 3.26 B (>4× YoY) Sep 2026
Busiest single repo requests ~1 B Aug 2026
Spokes default replica count 5 current arch
Write-throughput improvement up to 35× internal benchmark

Caveats

  • The 35× figure is from internal benchmarks, not disclosed production numbers, and the post does not detail the benchmark workload shape.
  • The post is a design-principles / why piece; the concrete future architecture (data path, consistency protocol, migration mechanics) is deferred to the "next post in this series."
  • No detail on how strong-consistency visibility ("made visible consistently before the next agent or CI job can build on it") is preserved once the authoritative copy lives in object storage rather than quorum fileservers — this is the hardest open question the post leaves.
  • Azure Blob Storage is named as the durable layer, consistent with GitHub's Azure footprint, but internal replication/durability guarantees are only described as "durability and replication at Azure scale."

Source

Last updated · 854 distilled / 2,448 read