Skip to content

CONCEPT Cited by 1 source

Operational ↔ analytical governance unification

Definition

Operational ↔ analytical governance unification is the architectural property that the operational tier (the OLTP database serving the application) and the analytical tier (the warehouse / lakehouse serving analysts) share a single governance substrate — one catalog, one access-policy model, one audit log, one billing surface — rather than being governed by two independent systems with their own RBAC, audit, and cost-tracking mechanisms.

The property is the structural answer to the compliance side- channel class: when the operational DB lives in a different security paradigm from the warehouse, every compliance question that spans the boundary ("who dropped this table?", "what was the data masking on the dev environment?", "how much did this ephemeral test environment cost?") requires cross-referencing multiple services, query languages, and access policies — turning the operational database into what the canonical source calls "a compliance side-channel."

Canonical instance — Lakebase + Unity Catalog (2026-05-15)

From the Backstage with Lakebase Part 2 source — the load-bearing architectural claim of the entire post:

"In a traditional stack, your application database and your data lake live in two entirely different security paradigms. The ownership graph for your infrastructure lives in Backstage, backed by an isolated RDS instance and governed by complex IAM roles and Postgres native grants. Meanwhile, your warehouse data is governed by the data team using Unity Catalog. … To audit a single table drop on RDS, you'd need to cross-reference CloudTrail for the IAM principal, pg_stat_activity or pgaudit logs for the SQL statement, and CloudWatch for the timestamp, three services, three query languages, three access policies. The operational database becomes a compliance side- channel."

The unified architecture replaces this:

"Because Lakebase is natively embedded inside Databricks, Unity Catalog extends directly over the operational Postgres database. … We didn't just change where the data lived; we changed where the access policy lived."

Mechanism: Lakehouse Federation exposes the operational Postgres as a foreign catalog (lakebase_bs) inside Unity Catalog. Standard UC GRANTs then govern access; system.access.audit captures every control-plane operation; UC system billing tables capture every cost line item. Three services collapse into one substrate with one query language.

Three structural payoffs

  1. Single audit substrate. Every control-plane action against the operational DB lands in system.access.audit — the same table that captures audit events for every other UC-governed asset, governed by the same row-level-security controls. "One SQL query instead of three services." (Source: sources/2026-05-15-databricks-backstage-with-lakebase-part-2)

  2. Single billing substrate. UC system billing tables capture compute costs broken down by branch identity dimensions, eliminating the "custom CloudWatch tagging that misses short-lived workloads" problem. See concepts/branch-level-cost-attribution.

  3. Single policy substrate. UC ABAC policies, governed tags, row filters, column masks — all evaluated against the operational DB the same way they're evaluated against analytical Delta tables. The same masking policies also propagate to database branches at creation time (concepts/branch-level-governance-propagation).

What it eliminates

Before (two paradigms) After (one substrate)
CloudTrail (IAM principal) system.access.audit
pg_stat_activity / pgaudit (SQL statement) system.access.audit
CloudWatch (timestamp + cost) system.access.audit + UC system billing tables
Postgres native grants (DB-tier RBAC) UC GRANT
Custom CloudWatch tagging strategy Native (project_id, branch_id, endpoint_id)
Manual masking on dev/test provisioning Branch-creation-time policy propagation
Three query languages (SQL + JSON Cloud-API + CloudWatch Logs Insights) One — SQL over UC system tables

What it depends on

  • Native embedding of the operational DB inside the analytical platform. "Because Lakebase is natively embedded inside Databricks, Unity Catalog extends directly over the operational Postgres database." This is the key architectural pre-condition — UC's reach is bounded by the platform boundary; an external Postgres on RDS cannot be unified this way without either migration into Lakebase or a different federation surface.
  • A query-federation layer that bridges policy. Lakehouse Federation is the wire-protocol-level mechanism that exposes the operational DB as a UC foreign catalog. Without it, "UC extends directly over" would be a marketing phrase rather than an enforceable property.
  • Identity-dimension uniformity across the substrate. The (project_id, branch_id, endpoint_id) triple has to be stamped the same way on audit events, billing events, and policy- evaluation events, or the "one SQL query" claim degrades to "one SQL query with three different join keys."

What it doesn't say

The 2026-05-15 source establishes the unification thesis and walks through three property instances (audit / cost / masking) but does not disclose:

  • Whether arbitrary operational-DB data-plane DML reaches system.access.audit (the worked example is control-plane: branch lifecycle); the "audit a single table drop on RDS" baseline is data-plane DDL — the strict claim about that before/after symmetry is not nailed down.
  • Performance overhead of UC-mediated access vs direct Postgres access.
  • The latency and consistency model between an operational write and its appearance in system.access.audit.
  • Whether the property generalises to operational stores other than Lakebase that aren't "natively embedded inside Databricks."

Sibling concepts

  • concepts/separation-of-duties-data-governance — the governance-team / steward / data-producer permission split that this unification preserves across both tiers; before, separation of duties had to be implemented twice (once for each paradigm).
  • concepts/branch-level-governance-propagation — the most operationally consequential consequence of the unification: branch-creation events (operational) propagate governance policies (analytical) without leaving the substrate.
  • concepts/single-platform-application-architecture — the application-tier dual: when the app, the operational DB, and the analytical store all live inside the same workspace + UC, the app authenticates as a service principal, queries UC over the SQL Statement API, and inherits UC access controls. This concept is the substrate-level version; the single-platform-app concept is the app-deployment-level version. They compose.

Seen in

  • sources/2026-05-15-databricks-backstage-with-lakebase-part-2 — First canonical wiki home for operational↔analytical governance unification. Unity Catalog absorbs the operational Postgres database via Lakehouse Federation as a foreign catalog (lakebase_bs); standard UC GRANTs replace Postgres-level role management; system.access.audit + UC system billing tables replace CloudTrail + pgaudit + CloudWatch cross-referencing; "one SQL query instead of three services." The compliance side-channel collapses into the catalog substrate.
Last updated · 542 distilled / 1,571 read