Skip to content

CONCEPT Cited by 2 sources

Credential vending

Credential vending is the architectural pattern in which a central authority (typically a catalog, registry, or governance plane) mints short-lived, resource-scoped credentials on demand for clients (engines, services, agents) wanting to access a downstream resource (cloud storage, database, API), instead of issuing the clients long-lived credentials directly tied to the underlying resource.

The defining properties:

  1. On-demand issuance — credentials are minted per request, not provisioned ahead of time and cached client-side.
  2. Resource-scoped — each credential carries authorisation only for the specific resource the request targeted, not broad access to the underlying credential store.
  3. Short-lived — minutes-scale TTL by default; compromise window is bounded.
  4. Catalog-mediated — the catalog is the only authoritative issuer; access policies are evaluated centrally; revocation is a catalog-side operation, not a downstream IAM-side operation.
  5. Auto-refreshable — the client SDK detects approaching expiry and re-invokes the vending API, transparent to the calling workload.

Why it exists

The architectural problem credential vending solves: how do you let external compute engines, agents, or services access a governed resource (a managed Delta table, a vector store, an unstructured- asset volume) without the structural failure modes of static credentials?

The two failure modes credential vending eliminates:

  • Static-IAM-principal sharing — granting an external party an IAM role / service-account key with broad cloud-storage permissions. Compromise is unbounded; revocation requires IAM-side action; scoping is permission-policy-only, not request-by-request.
  • Catalog-as-authoriser-but-direct-cloud-grant — the catalog decides who can access what but the cloud storage itself enforces via static credentials. Same blast-radius problem as static IAM.

Credential vending substitutes both with catalog-mints-per-request, storage-trusts-the-token: every access has a fresh, scoped, short- lived token, with the catalog as the only authoritative issuer. This is the short-lived-credential pattern applied at the governance-plane → resource-plane boundary.

Canonical instance: Unity Catalog Credential Vending

The 2026-05-14 Expanded interoperability with Unity Catalog Open APIs post canonicalises credential vending at the data-platform / lakehouse altitude:

"Unity Catalog handles this through credential vending… UC issues short-lived, scoped credentials to external engines on demand, with access policies enforced centrally." (sources/2026-05-14-databricks-expanded-interoperability-with-unity-catalog-open-apis)

Two productionisation additions disclosed at GA on 2026-05-14:

  1. M2M OAuth replaces PATs as the engine-to-catalog auth substrate. The structural failure modes of PATs the post calls out — "per-user, long-lived, and hard to rotate" — are dissolved by service-to-service OAuth (no human binding + short-lived tokens + automatic rotation). See concepts/m2m-oauth-vs-pat.

  2. Engine-side automatic refresh closes the long-running-job gap. "Credentials are refreshed automatically by engines via the UC credential vending APIs, so pipelines that run for hours complete reliably without tokens expiring mid-job." This is the property that makes minutes-scale credential lifetimes operationally tractable for hour-scale pipelines.

The architectural unification claim: the same credential-vending primitive governs both tabular data (UC managed Delta tables, GA at 2026-05-14) and unstructured data (UC Volumes for images / PDFs / videos, Public Preview at the same date). "The same access control model, audit trail, and scoped credentials apply whether you're querying a table or processing a raw video file externally."

Composes with

  • Short-lived-credential auth — credential vending is the catalog-mediated instance of the broader short-lived-credential family. The four properties (dynamic generation + short lifetime
  • self-rotating + centrally-revocable) all apply.
  • OAuth JWT short-lived credentials — the underlying token shape; UC's M2M OAuth tokens are an instance of the broader OAuth-JWT short-lived-credential family canonicalised on the 2026-04-30 Backstage-with-Lakebase ingest.
  • Catalog-managed commits — credential vending governs the data-path access; catalog commits govern the write-coordination path. Together they are the two halves of the external-engine- write-to-managed-table shape.
  • ABAC — per the 2026-05-14 forward roadmap, ABAC policies will compose onto credential vending so the fine-grained row + column-level governance disclosed for first-party reads on 2026-05-13 carries through to external reads.

When this is the right shape

  • The resource being accessed is multi-tenant: per-resource scoping is structurally necessary because tenants must not see each other's data.
  • The set of accessing principals is dynamic (external engines, agents, ephemeral compute): static IAM provisioning doesn't keep up.
  • Audit + governance is centralised — the catalog is the only place that knows the full access policy, so credentials must be minted there.
  • The downstream resource has a clean way to verify catalog-issued tokens — typically via a trust relationship with the catalog's OAuth issuer.

When this isn't the right shape

  • The accessing principal is static and long-lived with stable permissions — the operational overhead of vending may exceed the blast-radius savings (e.g., a single-tenant warehouse with one service account).
  • The downstream resource cannot verify the catalog-issued token and would require the catalog to proxy the data path — which is the proxy-path anti- pattern.
  • Latency budget for credential issuance is sub-millisecond — vending typically adds tens of milliseconds per request, which is fine for query-class operations but problematic for inner-loop micro-operations.

Seen in

  • sources/2026-05-28-databricks-advancing-apache-iceberg-on-databricks-iceberg-v3-ga-open-sharing-and-unified-governance — Credential Vending for Foreign Iceberg reaches GA, extending the credential-vending shape from UC's managed tables to foreign Iceberg tables — Iceberg tables managed in external catalogs (AWS Glue, Snowflake Horizon, Hive Metastore, Apache Polaris, Salesforce Data Cloud, Google Cloud Lakehouse, Palantir, Workday) but registered into UC for governance. UC mints short-lived scoped credentials for the federated tables exactly as it does for managed tables — unifying the auth model across managed + foreign Iceberg + foreign Volumes + managed Delta. Architectural significance: the catalog-mints-per-request, storage-trusts-the-token shape now works across catalog boundaries, not just within the catalog vendor's substrate. Tier-3 marketing-roundup framing acknowledged in the source page; mechanism details (how UC brokers credentials for storage backing tables managed by an external catalog) deferred to docs.

  • sources/2026-05-14-databricks-expanded-interoperability-with-unity-catalog-open-apis — First wiki canonicalisation of the architectural concept. Unity Catalog Credential Vending GA for tables, Public Preview for Volumes; M2M OAuth + auto-refresh as the two GA-readiness additions; shape unifies tabular + unstructured asset access under one credential model.

Last updated · 542 distilled / 1,571 read