Skip to content

SYSTEM Cited by 5 sources

Pingora

Pingora is Cloudflare's in-house Rust-based HTTP proxy framework and the foundation of its Rust proxy services. Open-sourced in early 2024 at github.com/cloudflare/pingora.

Pingora is not a complete proxy on its own — it's a framework / library that individual services (like systems/pingora-origin) build on to get: async I/O, TLS, HTTP/1/2/3 wire handling, upstream connection pooling, and request-lifecycle hooks. Services implement per-request logic (routing, header manipulation, filtering, metrics) in Rust on top of Pingora's core.

Role at Cloudflare scale

  • ≈60 M HTTP req/sec total Cloudflare network (2024 disclosure).
  • Pingora services run across the edge fleet; pingora-origin alone handles the ~35 M req/sec subset of non-cached outbound-to-origin requests.
  • Framework is explicitly built for CDN-scale hot-path constraints — per-request code sits in the tightest µs-budget regime (concepts/hot-path).

Why Rust

  • Memory safety without GC — no runtime pauses on the request hot path.
  • Predictable compile-time layout lets custom data structures (e.g., systems/trie-hard) exploit cache locality directly.
  • Plays well with kernel-bypass / high-throughput patterns the CDN fleet depends on.

Seen in

  • sources/2024-09-10-cloudflare-a-good-day-to-trie-hard — Pingora team running criterion microbenches against a custom clear_internal_headers helper in pingora-origin, backed by production stack-trace sampling, to cut the helper's CPU share from 1.71 % → 0.43 % and open-source the new trie-hard crate that enabled the final step.
  • sources/2026-04-17-cloudflare-agents-week-network-performance-update — context instance, not named directly. The post attributes ~half of the Sept → Dec 2025 ranking shift (40 % → 60 % fastest in top 1,000 networks) to "improving CPU usage and memory usage in our software that handles fundamental actions like establishing connections, SSL/TLS termination, traffic management, and the core proxy that all requests flow through" — i.e. the Pingora / FL2-proxy connection-handling hot path. Cloudflare's framework-level CPU efficiency is the enabling substrate for protocol-level wins like HTTP/3 and congestion-window tuning.
  • sources/2022-12-02-highscalability-stuff-the-internet-says-on-scalability-for-december-2nd-2022 — early Pingora disclosure (2022 Cloudflare blog, surfaced in Todd Hoff's Dec-2022 High Scalability roundup). Pre-open- source numbers: >1 trillion req/day, −70% CPU, −67% memory vs. the NGINX-based predecessor, 5 ms P50 TTFB reduction, 80 ms P95 TTFB reduction. Multithread
  • work-stealing + shared-connection-pool design cut new origin connections per second by ~3× overall; for one major customer the connection-reuse ratio went from 87.1% to 99.92% — a 160× reduction in new-connection rate. Hoff captures the design motivation: "Rust because it can do what C can do in a memory safe way without compromising performance; multithreading over multiprocessing in order to share resources, especially connection pools; work stealing; life-of-a-request event-based interface similar to NGINX/OpenResty." Quote: "Pingora crashes are so rare we usually find unrelated issues when we do encounter one."

  • sources/2026-09-01-cloudflare-how-we-could-save-petabytes-of-cache-storage-with-zstandard — Pingora is the proxy the Cache Transcoding prototype runs inside: on a cache miss the Pingora-based proxy zstd-encodes the eligible body before writing to disk, and decodes it back to identity before the response leaves the proxy. Another example of per-request work (compression eligibility check + encode/decode) added to the Pingora hot path — bounded by the encode-once, decode-many economics and an eligibility gate.

  • sources/2026-09-18-cloudflare-saving-another-100tb-of-ram-with-math-and-rust — another Pingora-based service surfaces: Pingora Backend Router (PBR), the internal cacheable-request load balancer. Its systems/pingora-ketama consistent-hashing rings were the memory hog (up to 6 GB/instance); a struct-packing fix (8→6 bytes/hash) plus a 90 %-fewer-hashes change reclaimed >100 TB of RAM globally. Shows the Pingora ecosystem is more than the edge proxy — internal routing services build on the same framework.

  • systems/pingora-origin — canonical pingora-built service.

  • systems/pingora-backend-router — internal cacheable-request load balancer built on Pingora; ketama-ring memory optimization.
  • systems/pingora-ketama — Pingora's consistent-hashing crate.
  • systems/trie-hard — performance-critical data structure written specifically for a pingora-origin hot path.
  • systems/cloudflare-fl2-proxy — Cloudflare's next-gen Rust edge proxy, sibling connection-handling hot path.
  • systems/cache-transcoding — cache-at-rest zstd compression built into the Pingora proxy.
  • systems/criterion-rust — microbenchmark crate used by Pingora maintainers.
  • connection-time
  • http-3
  • congestion-window
  • concepts/hot-path
  • companies/cloudflare — parent org; Rust proxy stack is a core platform bet.
Last updated · 766 distilled / 2,225 read