Skip to content

CONCEPT Cited by 1 source

Foreign Iceberg table

A foreign Iceberg table is an Apache Iceberg table whose data and primary metadata catalog live in an external system — AWS Glue, Snowflake Horizon, Hive Metastore, Apache Polaris, Salesforce Data Cloud, Google Cloud Lakehouse Runtime Catalog, Palantir, Workday, etc. — but which is registered into and governed by a different catalog (Unity Catalog in the canonical instance).

The architectural property is catalog-of-catalogs governance: customers can apply central governance, fine-grained access control, audit, and sharing policies to Iceberg tables they didn't produce and don't store in the governing catalog's substrate.

"With Foreign Iceberg now generally available, Unity Catalog can govern Iceberg tables managed in other catalogs. Customers can discover, secure, query, and share external Iceberg tables through Databricks while leaving the data and source catalog in place."

(Source: sources/2026-05-28-databricks-advancing-apache-iceberg-on-databricks-iceberg-v3-ga-open-sharing-and-unified-governance)

Architectural shape

External producer                Governing catalog              Consumers
                                 (Unity Catalog)
┌──────────────────┐        ┌──────────────────────────┐    ┌─────────┐
│ Snowflake        │        │  UC catalog hierarchy    │    │ Spark   │
│ Horizon          │ ──────►│  - foreign Iceberg table │◄───┤ Trino   │
│ (owns table data │        │    (metadata pointer)    │    │ DuckDB  │
│  & metadata)     │        │  - ABAC policies         │    │ Flink   │
└──────────────────┘        │  - tags                  │    │ etc.    │
                            │  - audit log entry       │    └─────────┘
┌──────────────────┐        │  - share membership      │         ▲
│ AWS Glue         │ ──────►│                          │         │
│ Hive Metastore   │        │  Credential Vending      │─────────┘
│ Polaris          │        │  for short-lived         │  (vended creds
│ ...              │        │  scoped credentials      │   for direct
└──────────────────┘        │                          │   data reads)
                            └──────────────────────────┘

Three properties that make foreign Iceberg different from federation alone

  1. Data stays in place. UC doesn't copy or replicate the data; the foreign table's files remain in the producer's storage. "leaving the data and source catalog in place."
  2. Governance applies fully. Foreign tables can be tagged, ABAC-policied, classified, and audited as if they were native UC tables. "discover, secure, query, and share." This is what distinguishes foreign Iceberg from a thin federation pointer that only forwards reads.
  3. Credential vending extends to foreign tables. UC mints short-lived scoped credentials for federated tables — the same auth shape as managed tables — so external engines and downstream sharers don't need direct broad access to the producer's storage. The 2026-05-28 announcement names this explicitly: "Credential Vending for Foreign Iceberg (GA)".

What it enables

  • Single pane of glass across catalogs. Enterprises with Glue + Snowflake + Hive + UC simultaneously can govern, discover, and share Iceberg tables across all of them through one UC interface.
  • External-Sharing of Foreign Iceberg. The same announcement introduces (Public Preview) sharing foreign Iceberg tables externally via Delta Sharing — meaning a table managed by Snowflake Horizon can be shared via Delta Sharing's open Iceberg-REST endpoint to any Iceberg-compatible recipient, with UC as the sharing layer and the data still living in Snowflake's storage.
  • Migration optionality without churn. Customers don't have to migrate Iceberg tables out of their existing catalog into UC to benefit from UC governance; they register and govern in place.

What it doesn't change

  • The producer catalog is still authoritative for the table's metadata. UC sees the table through the producer's catalog protocol; UC's governance applies on top but doesn't replace the producer's snapshot management, schema-evolution, or commit semantics.
  • Performance characteristics depend on the producer. A foreign Iceberg table queried through UC pays whatever metadata-fetch cost the producer's catalog imposes, plus UC's policy-evaluation cost.

Caveats

  • Mechanism details light. The 2026-05-28 announcement positions foreign Iceberg as "register and govern" but does not document the wire protocol UC speaks to each external catalog (Glue API, Snowflake Horizon REST, Hive Metastore Thrift, etc.). Mechanism depth requires Databricks docs (catalog federation docs) and per-catalog connector documentation.
  • Federation-set as of the announcement: AWS Glue, Snowflake Horizon, Hive Metastore, Salesforce Data Cloud at GA; Google Cloud Lakehouse and Palantir in Preview; Workday named in body but not explicitly placed in GA/Preview taxonomy.
  • No quantitative numbers on metadata-fetch latency, policy-evaluation overhead on foreign tables, or scale limits.
  • Write path on foreign tables undisclosed. Whether UC permits writes to foreign Iceberg tables (and how those writes coordinate with the producer's catalog commit semantics) is not addressed in the announcing source.
  • Interaction with Iceberg v3 features. Iceberg v3 features (deletion vectors, row tracking, VARIANT) are stated to apply to "managed Iceberg tables, foreign Iceberg tables, and UniForm-enabled managed tables" — but the version-floor of the foreign producer's catalog must support v3 too, which is not addressed.

Seen in

  • sources/2026-05-28-databricks-advancing-apache-iceberg-on-databricks-iceberg-v3-ga-open-sharing-and-unified-governance — GA disclosure for Foreign Iceberg + Credential Vending for Foreign Iceberg on Unity Catalog. Eight named upstream catalogs (existing GA: Glue / Snowflake Horizon / Hive Metastore / Salesforce Data Cloud; Preview: Google Cloud Lakehouse / Palantir; named: Workday). Public Preview of External Sharing of Foreign Iceberg tables — UC becomes the sharing layer for tables it doesn't store. Three architectural properties named: data in place + full governance + credential vending.
Last updated · 542 distilled / 1,571 read