Skip to content

CONCEPT Cited by 1 source

Data Mesh

Data mesh is an architectural approach to organisation-wide data sharing in which data products are owned and exposed by domain teams (R&D, After-Sales, Marketing, etc.), rather than pooled through a central data-platform team that owns every pipeline. A federated catalog + interoperable sharing protocol replaces the monolithic data warehouse as the point of contact.

Short definition for this wiki: a mesh is decentralised data ownership + centralised governance + an exchange protocol. The sharing protocol turns what would otherwise be point-to-point producer-consumer integrations into a mesh topology.

Shape that shows up in practice

  1. Domain-owned data products. Each producing team owns the lifecycle (schema, SLA, correctness) of a data product. Products, not tables, are the unit of exchange.
  2. A global catalog. One metadata + access-control plane over the mesh. Without this, the mesh is just "many data warehouses". On the Mercedes-Benz mesh this is systems/unity-catalog; the governance posture is concepts/hub-and-spoke-governance.
  3. An open exchange protocol. Producers publish once; consumers subscribe by name. On the Mercedes-Benz mesh this is systems/delta-sharing. The protocol decouples "who produced this" from "where does it run".
  4. Consumer-side flexibility. Because the exchange protocol is standardised, consumers can pick their read strategy (live, cached, replicated) independently of the producer. This is what makes patterns/cross-cloud-replica-cache expressible inside a mesh.
  5. Self-service control plane. Mercedes-Benz's DDX is an example of the "teams don't operate shares by hand" layer every non-trivial mesh grows; see systems/ddx-orchestrator.

What a mesh is not a synonym for

  • Not just "data lake + catalog". A data lake with a catalog is a mesh only if domain teams are the owners and there is a contract exchange protocol between them; a single central team running every pipeline against the lake is still a central platform.
  • Not just "Delta Sharing between metastores". Delta Sharing is the plumbing; a mesh also has the organisational + governance shape.
  • Not inherently multi-cloud. Cross-cloud is a forcing function that shows up on big meshes (see concepts/cross-cloud-architecture), but a single-cloud mesh is equally valid.

Seen in

Last updated · 200 distilled / 1,178 read