SYSTEM Cited by 1 source
TorchRec¶
Definition¶
TorchRec is PyTorch's library for sparse-table sharding — the substrate that lets large recommendation models split embedding tables (mapping items / users / features to learned vectors) across HBM, GPU host DRAM, and remote CPU-host DRAM so the table layout is decoupled from the model's compute graph (Source: sources/2026-05-26-meta-silvertorch-index-as-model-a-new-retrieval-paradigm-for-recommendation-systems).
Open-source: github.com/pytorch/torchrec.
Use in SilverTorch¶
In SilverTorch, the "very large sparse networks inside the model — embedding tables that map every item and every user feature to a learned vector" exceed single-GPU HBM. TorchRec is the named sharding substrate:
"TorchRec spreads these tables across HBM, GPU host DRAM, and even remote CPU-host DRAM, decoupling sparse data movement from computation."
This is the fourth tier of SilverTorch's scale-up-first strategy — once the dense network has been placed across the GPU memory hierarchy and document-sharded across hosts, TorchRec handles the sparse-table dimension that follows different placement rules.
Why it matters — sparse / dense placement asymmetry¶
Recsys models combine:
- Dense modules (towers, ANN, scoring) that benefit from staying on-GPU and being fused.
- Sparse modules (embedding tables) whose size is dominated by table dimensions × vocabulary, often reaching tens of billions of parameters that no single GPU can hold in HBM.
A unified placement policy fails — the dense path wants HBM-resident, the sparse path needs to span HBM / host DRAM / remote DRAM. TorchRec is the canonical answer to that asymmetry inside the PyTorch ecosystem: it presents a sharded-table API to the model code while orchestrating data movement between memory tiers under the hood.
Lineage on the wiki¶
The wiki already documents Meta's broader approach to sparse-dense co-design via:
- concepts/multi-card-embedding-sharding — splitting tables exceeding single-GPU memory across an optimised hardware cluster (Meta Adaptive Ranking Model 2026-03-31).
- concepts/unified-embeddings — multiple features sharing one table to amortise lookup cost.
- concepts/hash-collision-embedding-tradeoff — the tradeoff vocabulary-sized tables manage by hashing into a smaller codomain.
- patterns/multi-card-sharded-embedding-serving — the serving-time pattern.
TorchRec is the library substrate that mechanically realises these patterns inside PyTorch.
Caveats¶
- The 2026-05-26 SilverTorch post is the first canonical TorchRec page on the wiki; deeper internals (sharding algorithms, communication backends, batched lookups) are not disclosed in this source. See the open-source repo + Meta's prior MARM (sources/2026-03-31-meta-adaptive-ranking-model-bending-the-inference-scaling-curve) for adjacent architecture material.