Skip to content

CONCEPT Cited by 2 sources

Headless BI semantic layer

Definition

A headless BI semantic layer is a single, governed, tool-agnostic layer in the data platform where business metrics and KPIs are defined once, and every consumer — first-party BI dashboards, AI agents, SQL notebooks, third-party BI tools — resolves the same definition. "Headless" means the layer is not coupled to any particular BI tool's rendering / interaction surface: the metric lives in the platform, and any consumer with permission can resolve it.

The defining property: the metric definition is a first-class object in the data platform's catalog, with its own SQL logic, join graph, and semantic metadata, queryable through a uniform contract (e.g. a MEASURE() function call). Consumers don't see each other; the layer doesn't care which one called.

The 2026-05-27 Databricks BI Serving Pointers source canonicalises this concept on the wiki via Metric Views: "a single, governed semantic layer where you define your data model and KPIs once, independent of any specific BI tool. […] AI/BI Dashboards, Genie, SQL notebooks, and third-party BI tools all resolve metrics from the same definitions."

What problem it solves

The pre-headless antipattern, named verbatim in the source:

"Most organizations have the same business metrics defined in multiple places — a revenue calculation in one BI tool, a slightly different one in another, a third variant in a SQL notebook someone wrote last quarter. Each definition drifts independently. Nobody's sure which one is right."

The structural problem: per-tool semantic layers — LookML in Looker, calculated fields in Tableau, DAX in Power BI, a SQL view someone forked into a notebook — produce definition drift. Three failure modes:

Failure mode Symptom
Numerical disagreement Dashboard A says revenue is $X; dashboard B says $Y; the tie-breaking conversation happens in Slack instead of in the platform.
Governance drift Each per-tool layer has its own access controls and audit surface; PII handling, row filtering, and column masking get re-implemented (often inconsistently) per tool.
AI agent ambiguity When an AI agent queries the data, it has no canonical definition to reason over — it must either guess (and drift) or be told via per-prompt scaffolding (which doesn't generalise).

Three structural properties

  1. Single source of truth. "Define a metric once, and every consumer — human or AI — gets the same answer." The metric logic + dimensions + grain + semantic metadata live in one versioned object in the platform's catalog.

  2. Tool-agnostic resolution contract. Consumers query the metric through a uniform mechanism (SQL MEASURE() in Databricks; equivalent abstractions in other systems). The contract is what the metric is, not how it gets computed — the platform owns the optimisation.

  3. Governance flows through the layer. Access control, row filters, column masks, and audit are applied at metric-resolution time, not at the BI-tool boundary. This means the same metric queried from a third-party BI tool gets the same governance as the in-platform dashboard.

AI grounding via semantic metadata

The architecturally-distinctive property of a headless semantic layer is that its metadata is machine-readable AI grounding context. From the Databricks source:

"The semantic metadata is what sets them apart. Fields like display_name, comment, and synonyms give AI systems the context they need to interpret business questions correctly. When a user asks Genie 'what was our revenue last week?', those annotations are how Genie maps natural language to the right measure and dimensions. No custom prompts, no separate glossary. The richer your metadata, the more accurately AI serves the right answer."

This is the same architectural shape as layered grounded context for a data agent from Cloudflare Skipper — both name the platform's metadata catalog as the substrate over which agents reason about user questions, replacing per-prompt scaffolding with platform-resident structured context.

The convergence is a load-bearing wiki signal: AI agents querying enterprise data require a governed semantic layer to ground in; the data-agent design problem dissolves once the semantic layer exists.

Generalisation: the semantic-layer family

Headless BI semantic layers sit in a broader family of "define once, consume anywhere" abstractions on the wiki:

Concept / system Domain Defining property
Headless BI semantic layer (this page) BI / analytics metrics Metric definitions live in the platform; every consumer resolves the same; metadata grounds AI agents.
concepts/semantic-layer-of-business-concepts MDM / master data A business-vocabulary layer between source schemas and a target logical model — used by Zalando's MDM.
concepts/unified-metric-semantics Operational metrics (Prometheus / observability) Same metric semantics across runtimes / regions.
systems/viaduct (Airbnb) API serving Federated GraphQL where types live in subgraphs but consumers see one schema.

The shared shape: a layer that decouples definitional authority (where the truth lives) from rendering authority (which UI / tool displays it).

Composition with materialization

The headless shape composes naturally with automatic materialization: because the metric is defined in the platform, the platform can automatically maintain pre-aggregated forms behind it and transparently rewrite consumer queries to hit the materialization. The consumer's query never changes; only the substrate's resolution path does.

This is the architectural value-add over per-tool semantic layers: you can't add transparent query rewriting to a Tableau calculated field, because the BI tool is the substrate. You can add it to a Metric View because the platform is the substrate.

What a headless semantic layer is not

  • Not a metrics framework like Prometheus's PromQL — PromQL is a query language over time series, not a definitional layer for business metrics.
  • Not a data-warehouse view — a SQL view is a query-rewriting primitive but lacks the semantic metadata (display_name / comment / synonyms), the cross-consumer resolution contract (MEASURE()), and the materialization
  • governance integration.
  • Not a dbt metrics layer (per se) — dbt's metrics layer is a related primitive in a different governance plane (the dbt project, not Unity Catalog); the conceptual shape is similar but the resolution surface is dbt-resident and the consumer story is more limited.
  • Not necessarily SQL-resident — the headless property doesn't require SQL; it requires a uniform tool-agnostic resolution contract. SQL is the convenient choice when the underlying engine is SQL-shaped.

Failure modes

  • Metadata under-investment. A Metric View with thin display_name / comment / synonyms won't ground AI agents well. The headless win on the AI side is proportional to metadata richness.
  • Over-flattened metric definitions. When everything becomes a Metric View, the layer accumulates definitional debt; some metrics belong in the SQL view layer or as ad-hoc queries.
  • Materialization as default rather than selective. The Databricks "Get started" guidance is "enable materialization for your highest-traffic metrics", not blanket-enable — pre-aggregation has refresh cost and storage cost.
  • Per-team divergence inside the layer. Without governance on metric naming and synonym discipline, two revenue Metric Views can still emerge from different teams — the layer prevents drift across consumer tools but not across producer teams.

Open questions / forward signals

  • Open-standard metric resolution. SPARK-54119 names the Apache Spark implementation of Metric Views; Unity Catalog OSS support is "coming". The headless-semantic-layer-as- open-standard thesis is on track but not yet shipped.
  • Cross-platform consumption. Whether a Metric View defined in Databricks Unity Catalog can be consumed by a non-Databricks AI agent via an open metric-resolution wire spec is the natural follow-on disclosure to watch for.

Seen in

  • sources/2026-05-27-databricks-bi-serving-pointers-maximizing-for-performance-and-tco — first wiki canonicalisation of headless-BI-semantic-layer as a concept. Names the four-consumer surface (AI/BI Dashboards / Genie / SQL notebooks / third-party BI), the semantic-metadata-as-AI-grounding shape, and the open-standard trajectory via SPARK-54119. Reserved for future ingests: cross-vendor consumption stories, the metric-resolution wire protocol, federation across multiple semantic layers.
  • sources/2026-06-10-lyft-metric-semantic-layer — Lyft's Metric Semantic Layer is a second canonical instance: a versioned Python package with YAML + Jinja metric definitions, distributed to all consumers. Adds the dual-owner governance model (Business Owner + Operational Owner) and an MCP server for AI agents as first-class access channel.
Last updated · 542 distilled / 1,571 read