Skip to content

META 2026-10-26

Read original ↗

Meta Networking at the 2026 OCP Global Summit: Open Innovation for Scale-Up, Scale-Out, and Scale-Across AI Networks

A roundup of the networking and hardware talks Meta presented at the 2026 Open Compute Project (OCP) Global Summit, with slide decks and recordings published for every session. It is the talk-level companion to Meta's 2026-10-12 overview post OCP Summit 2026: Powering AI with Open Innovations — the overview introduced the systems; this post is the index of the deep-dive talks (optics, FBOSS integration, scale-across operations, scale-up testing, RDMA transport) that back them. All of it supports Meta's latest AI models (Muse Wonder, Muse Spark) and the gigawatt-scale Prometheus cluster.

The organizing frame is the full-stack, open approach to AI networking across the three distance domains — scale-up / scale-out / scale-across — with the thesis that Ethernet + lossless RoCE under one NOS (FBOSS) can serve all three, built in the open ecosystem rather than behind proprietary walls (executive framing: "Betting on the Network: Meta's Case for Full-Stack, Open AI Networking," Omar Baldonado).

Key takeaways

  • Switch platforms are the 102.4T generation on liquid cooling. The IceCube 102.4T switch (128×800GE non-blocking, Broadcom Tomahawk 6C or NVIDIA Spectrum-6, on a 2OU PHY-less, hybrid liquid/air-cooled design, N-IceCube variant) drops into the liquid-cooled IcePack rack and runs FBOSS. (talk: "Meta 102.4T Ethernet Switch")

  • Integrated optics are the rack-scale AI lever — power, density, disaggregation, reliability. A dedicated talk covers lessons from validating 51.2T co-packaged optics (CPO) and what carries over to 102.4T ("Integrated Optics at Scale: From 51.2T CPO Validation to Meta's 102.4T"). At AI scale the per-module retimer DSP is a repeated power/heat/cost tax across millions of links, which motivates linear pluggable optics (LPO) — removing the retimer but pushing equalization and triage onto the host ("Operating Linear Pluggable Optics at Hyperscale"). Also covered: 800G active electrical cables (AEC) as the economics-defining short-reach link, and building the near-package optics (NPO) ecosystem beyond what pluggables can deliver.

  • Copper scale-up is a statistical problem, not a debugging problem. In a copper scale-up domain, thousands of connections per rack mean ~10 bad links even at 99.9% yield, and blind-mate cartridges leave no per-link serviceability — so Meta's test methodology treats link health as a yield / statistics problem rather than a set of isolated faults ("Test Methodology for Multi-Accelerator Copper Scale-up Interconnect").

  • NVIDIA SDK + FBOSS ships production AI fabrics via a CI/CD, test-driven workflow. NVIDIA's Spectrum-X switch + SDK + SAI implementation + simulator integrate with FBOSS; the talk's load-bearing claim is that a CI/CD and test-driven integration workflow made rapid joint bring-up possible ("NVIDIA SDK + FBOSS: Building Production-Ready AI Network Fabrics").

  • FBOSS is config-driven from board bring-up to qualification, and vendors self-onboard. A tour of the FBOSS stack (routing, forwarding, platform services, board support) shows a config-driven approach taking a platform from board bring-up to OCP-qualified status quickly ("Open, AI-Ready FBOSS: From Platform Bring-up to Solution Qualification"), and the platform-layer standard OSPA (co-led with Microsoft + Alibaba, v1.0 submitted to OCP) removes the "months of duplicated bring-up work per platform" that SAI (data-plane-portable) left on the table ("OSPA — A Unified Network Switch Platform Reference Design").

  • Scale-across makes distance a first-order design constraint. BAG (Backend Aggregation) is the petabit super-spine joining buildings, campuses, and regions in Prometheus; the operations talk covers platform fit, distance management, reliability, and capacity planning against a quickly evolving AI ecosystem ("Scaling AI Infrastructure Across DC and Beyond: Meta's Journey with BAG"). A separate SRv6 regional backbone talk details why Meta chose SRv6, how its data/control plane run in production on FBOSS + SAI using shallow buffers at WAN distances, and a concept-to-deployment timeline "in a matter of months" ("SRv6 Powered Backbone Networks"). A cross-industry Scale-Across Networking Panel (Meta, Google Cloud, Microsoft Azure, Broadcom, Arista) compares routing and optical-transport approaches across hyperscalers, silicon, and systems vendors.

  • Ethernet scale-up: Tahan for MTIA 300 today, 72 accelerators/rack next, ESUN as the trajectory. Tahan is the Ethernet scale-up switch for the rack-scale MTIA 300 AI system (Broadcom Tomahawk 5), walked through for bandwidth/latency and scale-up feature set ("Tahan: An Open, Ethernet-Based Scale-Up Switch for MTIA"); a preview of 72 accelerators per rack on 102.4T silicon points to the next generation. ESUN (Ethernet for Scale-up Networking) progresses 1.0 (released earlier in the year) → 1.1 (upcoming) ("ESUN: Open Ethernet for the AI Scale-Up Era," Meta

  • Microsoft). A dedicated validation-methodology talk situates scale-up within the broader set of AI scaling domains — scale in, up, out, across, edge, and above ("Validation Methodologies for the ESUN Ecosystem," Meta + Keysight).

  • MetaRoCE is going open source — starting with the software implementations. Beyond the MetaRoCE transport-spec talk ("MetaRoCE: Rethinking RDMA for AI," with the framing that all-reduce / all-to-all is gated by the slowest node across hundreds of thousands of accelerators), Meta is open-sourcing its MetaRoCE software implementations, starting with a user-space version on GitHub (libsoftmetaroce) with a DPDK-based version coming soon ("MetaRoCE in Software"). This is the concrete delivery of the open-source plan announced in the 2026-08-24 MetaRoCE post.

  • Gigawatt framing from the top. Keynote/executive sessions set the scale: AI has changed the shape of infrastructure (Santosh Janardhan, "Concrete to Code: Engineering AI Infrastructure at Gigawatt Scale"), and open scale-across is needed because training compute is compounding faster than infrastructure can scale (Vipul Deokar + Cisco, "Breaking the Walls: Open Scale-Across for the AI Era").

Systems / concepts extracted

Switch platforms & rack: IceCube / IcePack — 102.4T, 128×800GE non-blocking, 2OU PHY-less hybrid-cooled, Tomahawk 6C / Spectrum-6, on FBOSS/SAI. Scale-out fabric: Flattened Planar Fabric (FPF) — 8 planes per GPU, 3-tier→2-tier Clos, NIC packet spraying, combined host+switch control plane (NVIDIA Spectrum-X Multiplane). Scale-across: BAG petabit super-spine + SRv6 regional backbone on FBOSS/SAI (shallow buffers at WAN distance). Scale-up: Tahan (Tomahawk 5, MTIA 300, 72-accel/rack preview) → ESUN 1.0→1.1 data plane. Standards/NOS: FBOSS (config-driven bring-up, NVIDIA SDK CI/CD integration, vendor self-onboarding), OSPA (v1.0 to OCP), SAI. Transport: MetaRoCE spec + open-source software (libsoftmetaroce user-space, DPDK next), riding lossless RoCE and lossless Ethernet. Silicon: NVIDIA Spectrum-6 / Spectrum-X, Broadcom Tomahawk 5/6C, Cisco SiliconOne (SRv6 shallow-buffer).

Named-only, below the page-creation gate (recorded as tags + prose, no pages, no links): co-packaged optics (CPO), linear pluggable optics (LPO), near-package optics (NPO), 800G active electrical cables (AEC), integrated optics, SRv6 / segment routing, blind-mate copper cartridges, yield-as-statistics copper testing, NVIDIA Spectrum-X simulator, Keysight test tooling. These are talk-abstract mentions without a section's worth of standalone architecture; a later deep-dive post can mint pages and back-fill.

Operational numbers / specifics

  • IceCube: 102.4 Tbps = 128 × 800GE non-blocking; 2OU, PHY-less, hybrid liquid/air design (IceCube and N-IceCube variants); Tomahawk 6C / Spectrum-6.
  • Optics tax framing: the per-module retimer DSP repeats across "millions of links" in power, heat, and cost — the motivation for LPO.
  • Copper scale-up yield math: ~10 bad links per rack at 99.9% yield across thousands of connections, with no per-link serviceability on blind-mate cartridges.
  • ESUN: 1.0 specification released earlier in the year; 1.1 upcoming.
  • OSPA: v1.0 specification submitted to OCP.
  • MetaRoCE software: user-space libsoftmetaroce on GitHub now; DPDK-based version coming soon.
  • Scale-across geography: BAG spans buildings → campuses → regions in the ~1 GW Prometheus cluster (toward Hyperion); SRv6 concept-to-deployment "in a matter of months."
  • ESUN validation framing: six AI scaling domains — scale in, up, out, across, edge, and above.

Caveats

This is a talk-roundup / index post, not a single engineering deep-dive — most technical substance is in the linked slide decks and recorded talks (hosted on Google Drive / YouTube and the OCP Summit site), not transcribed in the post body. Concrete production numbers (fabric sizes, fleet counts, loss/throughput figures, latency) live in the individual talks, not the roundup. The optics items (CPO / LPO / NPO / AEC) and SRv6 are named at abstract granularity only. Several talks are forward-looking (72-accel/rack preview, ESUN 1.1, DPDK MetaRoCE "coming soon").

Source

Last updated · 854 distilled / 2,448 read