SYSTEM Cited by 1 source
Owl (Meta content distribution)¶
Owl is Meta's internal system for distributing hot content (binaries, data blobs, ML models) to millions of client processes in Meta's datacenters. Described in Meta's research paper "Owl: Scale and Flexibility in Distribution of Hot Content", surfaced in the High Scalability Dec-2022 roundup.
Scale¶
- 800 petabytes/day distributed.
- Serves millions of client processes.
- 106 use cases across Meta, using 55 different distribution policies.
Key design claim¶
Centralizing the control plane is not a barrier to scalability. This directly contradicts the intuition from peer-to-peer systems (like BitTorrent) where fully decentralized control is treated as the scaling win.
Owl splits the two planes:
- Control plane: centralized, manages who pulls from whom, applies per-use-case distribution policies.
- Data plane: peer-to-peer, so the actual bytes move directly between peer hosts without transiting a central fan-out server.
Performance¶
Owl reports 2-3× download-speed improvement vs.:
- BitTorrent
- A prior Meta-internal decentralized static distribution tree.
Why it shows up on this wiki¶
- Counterweight to the "decentralize everything for scale" dogma. At Meta scale, centralized control + peer-to-peer data turned out to be faster than fully decentralized.
- Clean separation of control plane and data plane is a recurring pattern — Owl is the hot-content-distribution instance of it, paralleling Consul's control-plane-plus-gossip and many other distributed systems.