Skip to content

SYSTEM Cited by 1 source

Lakewatch (Databricks)

Lakewatch is Databricks' agentic SIEM built on the security lakehouse. It consumes the same audit-trail substrate that Unity AI Gateway and Unity Catalog write — Inference Tables + UC audit logs — and turns it into AI-driven threat detection and response. First wiki disclosure 2026-05-20.

Definition (from the source)

"Lakewatch, Databricks' agentic SIEM built on the security lakehouse, takes this further still, turning the same audit trail into active security intelligence: AI-driven threat detection and response built on the lakehouse. Attackers are using agents. Defenders should too." — Source: sources/2026-05-20-databricks-governing-ai-agents-at-scale-with-unity-catalog

Structural insight: same substrate, dual purpose

The architectural payoff is the substrate-reuse property:

Unity AI Gateway + Unity Catalog
        │ writes
        ▼
   ┌───────────────────────────┐
   │  Inference Tables         │
   │  UC audit logs            │
   │  Usage-tracking tables    │
   │  Data-quality monitoring  │
   └───────────────────────────┘
            │
   ┌────────┴────────┐
   ▼                 ▼
audit /        Lakewatch
compliance     (agentic SIEM)
queries         │
                ▼
            threat detection
            + response

The same data that proves to auditors "what did our agents do?" powers detection of "what are attackers' agents trying to do to our systems?". The two consumers share the substrate but have different query shapes:

  • Audit / compliance — point queries against specific events for forensic reconstruction.
  • Lakewatch — continuous detection across the full audit stream, looking for anomaly patterns.

The asymmetry argument

"Attackers are using agents. Defenders should too."

The post's most direct framing of why agentic SIEM is structurally appropriate — and why a static-rules-only SIEM is structurally insufficient — for the agent-era threat model. If attackers can compose new TTPs at agent speed, defenders need detection that operates at agent speed too. Static-rule SIEMs were built for a threat model where adversary playbooks evolved on quarterly timescales; agentic adversaries change tactics within a single attack chain.

Why "lakehouse" (not just "log warehouse")

The post's framing is explicit: "built on the security lakehouse." The lakehouse property buys:

  • Joinable with business data — same as Pillar 2 (concepts/data-centric-ai-governance). Detection rules can join "this agent accessed PII" against "is this a customer that consented to this access?".
  • Same governance as the rest of the catalog — UC ABAC, row filters, column masks apply to security data too, so Lakewatch's own access surface is governed.
  • Open formats — Delta Lake substrate means detection rules can be written in standard SQL, no proprietary query DSL.
  • Cost-effective long retention — required for slow-moving APT detection where the indicators of compromise show up months after the initial intrusion.

What's not disclosed (significant gaps)

The post is the first mention of Lakewatch on the wiki and the disclosure is one paragraph. Not disclosed:

  • Detection rule syntax / engine.
  • Whether detection runs as batch queries or streaming detection.
  • Whether the agentic detection-loop is itself built on Databricks-hosted models or external.
  • Response automation surface (does it close incidents? page humans? pause agents?).
  • Whether Lakewatch consumes only Unity-AI-Gateway-written tables or also broader security-lake feeds (network, endpoint, identity).
  • Pricing / packaging.
  • Public availability (GA / beta / private preview).

The page is currently stub-quality — it canonicalises the name + framing but leaves the architecture as a wiki gap to be filled by future Databricks Lakewatch posts.

Relation to other wiki primitives

  • Sibling to AWS CloudTrail — both are durable-audit-substrate-as-detection-input. CloudTrail's input is AWS API calls; Lakewatch's input is AI-agent traffic + UC data access.
  • Companion to systems/bits-ai-sre (Cloudflare's SRE-loop agentic system) — both are agentic detection / triage loops, different domains. Bits AI SRE is for incidents; Lakewatch is for security threats.
  • Substrate-reuse pattern shared with patterns/inference-payload-table-for-audit + patterns/telemetry-to-lakehouse — both depend on operational data landing in the same governed lakehouse, queryable by both compliance teams and detection systems.

Seen in

Source

Last updated · 542 distilled / 1,571 read