Skip to content

SYSTEM Cited by 1 source

Haxl (Haskell concurrent data-fetch framework)

What it is

Haxl is a Haskell library, built and open-sourced by Meta, that provides implicit concurrent data fetching: a programmer writes code that looks like a series of sequential fetches against multiple data sources; the Haxl framework + GHC's Applicative do-notation together automatically batch calls to the same data source and overlap calls to distinct data sources, without the programmer writing any explicit concurrency constructs (forkIO, MVar, etc.).

Why it exists

Meta built Haxl for Sigma, its anti-abuse rule engine. Most Sigma policies fetch external data (user profile, link reputation, graph features, etc.) before deciding whether to allow or block an interaction. Policy authors are anti-abuse engineers — they write spam-detection logic, not schedulers. The requirement is therefore that concurrency be implicit: whatever fetches a policy happens to do, the runtime arranges them efficiently, and the policy code stays about the spam logic.

All of Haskell's native concurrency primitives are explicit (forkIO, MVar, STM). Haxl provides a different abstraction — the programmer expresses a pure functional computation involving multiple fetches; the Haxl monad tracks data dependencies; same-source calls are coalesced into a single batched request; independent calls on different sources are issued in parallel.

GHC co-design: Applicative do-notation

For Haxl's batching/overlapping to work on statement sequences that look imperative, the compiler has to distinguish genuinely sequential statements (a later statement uses an earlier one's result) from independent statements (parallelisable). Meta designed and implemented the Applicative do-notation feature in GHC to perform this rearrangement automatically at compile time. It is now a standard GHC feature.

Meta's integration architecture

In Sigma, the "data sources" wrapped as Haxl data sources are existing C++ service-client libraries for Facebook's internal services — wrapped via Haskell's FFI rather than rewritten. A compile-time C++ name-demangler removes the need for C-shim layers for most calls.

Seen in

Last updated · 319 distilled / 1,201 read