Skip to content

SYSTEM Cited by 2 sources

TorchRec

Definition

TorchRec is PyTorch's library for sparse-table sharding — the substrate that lets large recommendation models split embedding tables (mapping items / users / features to learned vectors) across HBM, GPU host DRAM, and remote CPU-host DRAM so the table layout is decoupled from the model's compute graph (Source: sources/2026-05-26-meta-silvertorch-index-as-model-a-new-retrieval-paradigm-for-recommendation-systems).

Open-source: github.com/pytorch/torchrec.

Use in SilverTorch

In SilverTorch, the "very large sparse networks inside the model — embedding tables that map every item and every user feature to a learned vector" exceed single-GPU HBM. TorchRec is the named sharding substrate:

"TorchRec spreads these tables across HBM, GPU host DRAM, and even remote CPU-host DRAM, decoupling sparse data movement from computation."

This is the fourth tier of SilverTorch's scale-up-first strategy — once the dense network has been placed across the GPU memory hierarchy and document-sharded across hosts, TorchRec handles the sparse-table dimension that follows different placement rules.

Why it matters — sparse / dense placement asymmetry

Recsys models combine:

  • Dense modules (towers, ANN, scoring) that benefit from staying on-GPU and being fused.
  • Sparse modules (embedding tables) whose size is dominated by table dimensions × vocabulary, often reaching tens of billions of parameters that no single GPU can hold in HBM.

A unified placement policy fails — the dense path wants HBM-resident, the sparse path needs to span HBM / host DRAM / remote DRAM. TorchRec is the canonical answer to that asymmetry inside the PyTorch ecosystem: it presents a sharded-table API to the model code while orchestrating data movement between memory tiers under the hood.

Lineage on the wiki

The wiki already documents Meta's broader approach to sparse-dense co-design via:

  • concepts/tensor-parallelism — splitting tables exceeding single-GPU memory across an optimised hardware cluster (Meta Adaptive Ranking Model 2026-03-31).
  • unified-embeddings — multiple features sharing one table to amortise lookup cost.
  • hash-collision-embedding-tradeoff — the tradeoff vocabulary-sized tables manage by hashing into a smaller codomain.
  • multi-card-sharded-embedding-serving — the serving-time pattern.

TorchRec is the library substrate that mechanically realises these patterns inside PyTorch.

Caveats

Seen in

Last updated · 766 distilled / 2,225 read