Skip to content

SYSTEM Cited by 2 sources

Lakehouse Federation

Lakehouse Federation is Databricks' query-federation surface inside Unity Catalog that exposes data in external systems as foreign catalogs governed by UC. A foreign catalog appears in UC alongside native Delta-Lake-backed catalogs; standard GRANT / row-filter / column-mask / tag-based ABAC policies apply to it the same way they apply to managed UC tables.

Stub page. First wiki source surfaces Lakehouse Federation as the mechanism by which an operational Lakebase Postgres database becomes a UC-governed foreign catalog, replacing Postgres-level role management with UC GRANT semantics.

Canonical wiki instance — operational DB as foreign catalog

From the 2026-05-15 Backstage-with-Lakebase Part 2 source: the Thoughtworks POC pointed Backstage at Lakebase and exposed the Backstage catalog as a foreign catalog (lakebase_bs) in Unity Catalog via Lakehouse Federation. Verbatim:

"Because Lakebase is natively embedded inside Databricks, Unity Catalog extends directly over the operational Postgres database. In this POC, we used Lakehouse Federation to expose the Backstage catalog as a foreign catalog (lakebase_bs) in Unity Catalog. Once it's there, standard UC grants control who can see what, no Postgres-level role management required."

This is the load-bearing architectural mechanism behind the post's operational↔ analytical governance unification thesis: the operational tier and the analytical tier no longer live in two security paradigms because they share the same catalog substrate. (Source: sources/2026-05-15-databricks-backstage-with-lakebase-part-2)

What's not disclosed

The Part 2 source asserts the federation capability but does not disclose mechanism:

  • How UC GRANT semantics translate to Postgres-level enforcement decisions across the federation boundary.
  • Whether row filters / column masks are pushed down to Lakebase or evaluated above the federation engine.
  • Latency envelope of a UC-governed Lakebase query vs a direct Postgres query.
  • Whether write-path enforcement uses the same path as read-path.
  • How the lakebase_bs foreign-catalog binding is created (declarative DDL? UI flow? bound at Lakebase project creation?).

The first three are likely to be addressed in either Part 3 (FinOps) of the series or future Databricks Lakehouse Federation disclosures.

Composes with

Seen in

  • sources/2026-09-25-databricks-from-data-to-dialogue-how-sp-global-energy-made-its-structured-data-estate-conversational — Federation as the substrate that makes non-Databricks data conversational — no ETL. S&P Global Energy's data spans Databricks and several non-Databricks sources. Where data lives outside Databricks, SMEs bring it in through Lakehouse Federation connectors so it can back a Genie Agent: "no data movement, no duplicate pipelines. The federated tables appear alongside native tables and inherit the same governance." Every agent question then runs through Unity Catalog on governed tables "native or federated alike." Best-practice lesson: "Use Lakehouse Federation before you build pipelines" — federation reaches "conversational" without a single new ETL job, and hot paths can be materialized later. A second canonical instance of federation-as-governed-foreign-substrate (complementing the Lakebase Part-2 operational-DB instance below), here as the reach mechanism for a conversational-AI estate.

  • sources/2026-05-15-databricks-backstage-with-lakebase-part-2 — First canonical wiki disclosure of Lakehouse Federation. The Thoughtworks Backstage POC uses Federation to expose the Backstage Postgres catalog as lakebase_bs foreign catalog in Unity Catalog; standard UC grants then govern access without Postgres-level role management. Mechanism for the operational↔analytical governance unification thesis. Capability- level disclosure only — federation engine internals not yet disclosed.

Last updated · 766 distilled / 2,225 read