Skip to content

PATTERN

Recipient per partner, Share per dataset group

Intent

Deploy a cross-organisation data-sharing surface using a three-primitive model that separates what to share (a Share — logical container of tables), who to share it with (a Recipient — digital identity per partner), and how the partner authenticates (an Activation Link — secure bootstrap URL). This gives per-partner access granularity without exposing physical storage paths and without copying data.

Canonical implementation: Databricks' Delta Sharing model in systems/unity-catalog.

Problem

A data provider wants to share analytical datasets with many external parties (partners / customers / suppliers). Each party should see only their own authorised datasets, and grants should be easy to audit, change, and revoke. The naïve alternatives fail:

  • Per-partner database user — every partner needs full network access to the data plane; database-level access control is not fine-grained enough; credentials are long-lived and embedded in partner code.
  • Per-partner S3 bucket + IAM policy — duplicates data across buckets or uses presigned URLs; IAM boundaries don't align with table boundaries; revocation requires touching infrastructure.
  • Per-partner REST API key — doesn't scale past a few tables; pagination / schema evolution kills analytics workloads.

Solution

Use three separate primitives:

  1. Share — a logical container in the governance plane (Unity Catalog) that groups related tables. A Share is not a copy; it's a permission-bundle that references Delta tables in place. One Share can contain many tables. Many Recipients can be granted access to the same Share.

  2. Recipient — a digital identity representing one external consumer (one partner). Recipients are named objects in the governance plane. Each Recipient has its own credentials / tokens, its own audit trail, and its own set of granted Shares. Revoking one Recipient doesn't affect others.

  3. Activation Link — a one-time-use secure URL used to bootstrap the Recipient's credentials. Partner visits URL, downloads credential file, points their Delta Sharing client at it. Canonicalised as concepts/activation-link-credential-bootstrap.

Five-step deployment

1. Prepare datasets (Delta tables) in Unity Catalog.
2. CREATE SHARE   finance_data_share;
3. ALTER SHARE    finance_data_share ADD TABLE gold.finance.orders;
                  ALTER SHARE finance_data_share ADD TABLE gold.finance.returns;
4. CREATE RECIPIENT acme_corp_recipient;  -- returns activation URL
5. GRANT SELECT ON SHARE finance_data_share TO RECIPIENT acme_corp_recipient;

Activation URL is then delivered to Acme Corp out of band; they download their credential file and start reading.

Shape

[Unity Catalog]
      │
      ├── Share: finance_data_share
      │     ├── Table: gold.finance.orders
      │     └── Table: gold.finance.returns
      │
      ├── Share: inventory_data_share
      │     └── Table: silver.inventory.stock_levels
      │
      ├── Recipient: acme_corp                 ← granted finance_data_share
      │     └── credential: activation URL / downloaded profile file
      │
      ├── Recipient: umbrella_corp             ← granted finance_data_share + inventory_data_share
      │     └── credential: activation URL / downloaded profile file
      │
      └── Recipient: initech                   ← granted inventory_data_share
            └── credential: activation URL / downloaded profile file

The many-to-many relationship between Shares and Recipients is the load-bearing property. One dataset group (Share) can be exposed to many partners; one partner (Recipient) can be granted many dataset groups.

Consequences

Upsides

  • Fine-grained revocation. Drop a Recipient, all that partner's access disappears; other partners unaffected.
  • Dataset-group reuse. Ten partners who need the same finance datasets share one Share definition; no N-copy drift.
  • Audit alignment. Every read is attributable to a specific Recipient on a specific Share; audit trail is denormalised-free at the governance plane.
  • Zero-copy underneath. The Share doesn't materialise data; reads hit the underlying Delta tables in place (concepts/zero-copy-data-sharing-protocol).
  • Client-agnostic. Recipients authenticate with the same credential file regardless of which Delta Sharing client they use — works across the full open-client ecosystem (patterns/open-protocol-over-proprietary-exchange).

Downsides

  • Pilot-phase manual ops. Until the provisioning path is automated, each Recipient creation is a human step (create Recipient → get URL → send URL → confirm partner connected). Does not scale past ~tens of partners without automation.
  • Share-level granularity only. If a Share has table A and partner B should see A but not table C, you must split into two Shares. Many Shares multiply governance surface.
  • Credential file sits on partner disk. A long-lived token lives on partner-controlled infrastructure; revocation requires partner-cooperation or active token rotation. OIDC federation removes this; until then, activation-link bootstrap is the default.

Pilot-to-platform evolution

In Zalando Partner Tech's pilot, the five-step deployment is executed manually per partner. For the platform build-out, each step turns into a platform primitive:

  • Recipient creation → API endpoint in the internal platform.
  • Activation URL delivery → automated via partner portal / partner-provisioning workflow.
  • Grant management → self-service UI for dataset owners to manage Share composition + Recipient grants.
  • Credential lifecycle → automated rotation + revocation hooks tied to partner-lifecycle events.

See patterns/pilot-to-platform-via-internal-demand for the organisational context that drives this automation.

Generalisation beyond Delta Sharing

The three-primitive model (Share / Recipient / Activation Link) generalises to any cross-org exchange layer:

  • Content distribution with access control — one piece of content, many subscribers, per-subscriber revocation.
  • Multi-tenant API gateway — Shares = API surface bundles, Recipients = tenant apps, Activation Links = API-key-bootstrap URLs.
  • Partner-facing event streams — Shares = Kafka topic groups, Recipients = partner consumer groups, Activation Links = client-credential-bootstrap.

The naming is Delta-Sharing-specific; the architectural shape is not.

Seen in

  • — Zalando Partner Tech deploys this exact five-step primitive against Databricks' managed Delta Sharing service. Pilot-phase manual; platform-phase automated. Canonical source for the three-role model (Delta Share, Recipient, Activation Link).
Last updated · 428 distilled / 1,221 read