Skip to content

PATTERN Cited by 2 sources

Open protocol over proprietary exchange

Intent

When building a cross-organisation data or service exchange layer (partner data access, supplier API integration, customer-facing analytics feed), pick an open protocol with a broad open-client ecosystem over a proprietary alternative — even if the proprietary alternative is technically superior on some axes — because open protocols:

  1. Let external parties use their existing stacks instead of adopting a vendor client library.
  2. Avoid a client-ecosystem fork where every language / BI tool / notebook need its own vendor-specific adapter.
  3. Make the "switch vendor" migration cost finite (re-point a client at a different server) instead of unbounded (re-platform every partner's integration code).
  4. Reduce partner adoption friction, which is often the load-bearing bottleneck for B2B data programmes.

Problem

Service providers building external-facing exchange surfaces have a choice between:

  • Proprietary protocol (vendor-specific API, SDK, client libraries). Can be technically tighter-coupled, sometimes faster, and vendor-optimised.
  • Open protocol (public specification, multiple independent client implementations across languages / tools).

Proprietary often looks cheaper to build initially because you own the full stack. But:

  • External parties have to adopt your client library — bigger barrier than one-button-install-from-package-manager clients.
  • Tools those parties already use (Excel, Tableau, pandas, Spark) don't natively speak the vendor protocol.
  • Vendor lock-in from the provider's side: switching costs rise because every consumer has integrated against the proprietary shape.

Solution

Pick a protocol with these properties:

  • Public specification. Anyone can implement a client.
  • Healthy client ecosystem. Clients already exist for the dominant consumer tools in the consumer's ecosystem.
  • Vendor-neutral. No single vendor owns the protocol's evolution.
  • Composable with existing governance. Integrates with existing catalog / identity / audit primitives the provider already uses.

Deploy via a managed service that speaks the protocol, so the provider doesn't pay self-hosting operational cost (often composed with patterns/managed-services-over-custom-ml-platform).

Canonical instance — Delta Sharing

systems/delta-sharing (Databricks, 2021) ticks all four:

  • Public protocol — open source, published specification.
  • Client ecosystem — Spark / pandas / Power BI / Tableau / Excel / Python / Java clients.
  • Vendor-neutral — implementations exist outside Databricks (open-source reference server, third-party clients).
  • Composable governance — pairs with systems/unity-catalog on Databricks but is not locked to it; open-source reference server is standalone.

Zalando Partner Tech picked it explicitly against the alternative of building a proprietary API or expanding their existing self-service report + SFTP paths. Quote:

We needed something that wouldn't lock us into a corner. The solution had to be extensible and compatible with open tools, ensuring our partners could integrate it with their existing workflows rather than forcing them to adopt entirely new processes.

(Source: sources/2025-07-07-zalando-direct-data-sharing-using-delta-sharing-introduction-our-journey-to-empower-partners)

Mercedes-Benz used the same architectural axis for a cross-cloud data mesh (AWS↔Azure), explicitly framing Delta Sharing's openness as what made it "a more secure alternative to FTP drops or shared-secret exchange" for external partner integrations (sources/2026-04-20-databricks-mercedes-benz-cross-cloud-data-mesh).

Consequences

Upsides

  • Lower partner adoption friction. Partners use tools they already have. Onboarding docs become "point your existing Spark/pandas/Excel client at this URL + credential file" rather than "install vendor SDK, configure auth, write integration code".
  • Broader partner segmentation reach (concepts/segmented-partner-data-access-tiering). Large partners on Spark + medium partners on BI + small partners on Excel all work from day one.
  • Future vendor migration is tractable. If the managed service is dropped or becomes too expensive, self-hosting the open-source reference is an escape hatch. Partners don't re-integrate because they're speaking the protocol, not the vendor.
  • Community-driven client improvements. Bug fixes and new features in open clients happen without provider engineering investment.

Downsides

  • Protocol evolution is not provider-controlled. Adding a feature requires community (or specification-body) agreement.
  • Edge cases in open clients. Some clients may be less-maintained; provider may still need to debug misbehaviours.
  • Performance sometimes lags proprietary. Open protocols optimise for compatibility over peak throughput; a vendor-optimised binary protocol may be faster in narrow benchmarks.

When not to apply

  • Internal-only exchange. If the consumers are all inside your organisation with the same client stack, proprietary is cheaper.
  • Protocol does not exist yet. If no open standard covers your shape, inventing one from scratch before you have production traction is premature; use proprietary until industry convergence happens.
  • Security / compliance requires vendor-signed clients. Some regulated domains mandate specific vendor-attested client implementations; open clients may not meet audit requirements.

Composition

Seen in

Last updated · 550 distilled / 1,221 read