Skip to content

CONCEPT Cited by 1 source

Zero-copy sharing

Zero-copy sharing means two or more processes (or languages running inside one process) read the same in-memory data without serialization, deserialization, or memcpy between them. The canonical modern instantiation is systems/apache-arrow buffers shared via shared memory or filesystem-backed memory-mapped files.

Why it matters

  • Serialization is a tax on every data-plane operation. At exabyte scale, JSON/pickle/Avro/Protobuf round-trips dominate CPU budget.
  • Fine-grained task/actor frameworks (concepts/task-and-actor-model) hand inputs and outputs between many short-lived workers; serial- ization of every hand-off would saturate CPU.
  • Cross-language interop (Python ↔ C++ ↔ Rust ↔ Java) is the other big winner: Arrow's on-the-wire format is the in-memory format, so a Python-over-PyArrow pipeline that calls into a C++ library doesn't pay a crossover cost.

Mechanism

  • Shared memory object store — one-per-node store (Ray's Plasma; Arrow's IPC), objects are reference-counted and handed out by pointer.
  • Arrow IPC — a standard on-wire format that is also the in-RAM format; "sending" an Arrow batch is handing a reference.
  • Memory-mapped files — files mapped into the virtual address space; multiple processes mmap the same file, OS pages the data in on demand.

Prerequisite: locality

Zero-copy only works when the consuming task runs on the same node as the object. That is why the companion policy is concepts/locality-aware-scheduling — schedule the task on the node that already has the object, bypass the network entirely.

Scale evidence (Amazon BDT, 2024)

Ray's zero-copy intranode object exchange during locality-aware shuffles is named as one of the five mechanisms that drove the 82% cost-efficiency improvement vs Spark at exabyte scale. In-memory Arrow volume alone: ~4 EiB per quarter processed through Ray's object store across the BDT Flash Compactor.

(Source: sources/2024-07-29-aws-amazons-exabyte-scale-migration-from-apache-spark-to-ray-on-ec2)

Seen in

Last updated · 200 distilled / 1,178 read