Skip to content

SYSTEM Cited by 1 source

DuckDB

DuckDB is an open-source, in-process columnar analytical database (duckdb.org) — designed for single-node analytical workloads, embedded into application processes the same way SQLite is, with full SQL support and columnar execution.

The defining properties for this wiki:

  • In-process / embedded — no separate server; runs in the application's address space.
  • Columnar storage + vectorised execution — analytical query performance comparable to dedicated OLAP engines on single-machine workloads.
  • Open-format reader/writer — reads/writes Parquet, Delta, and Iceberg via extensions.
  • Single-node first — no built-in distributed execution; scales by machine size, not cluster size.

Seen in

  • sources/2026-05-14-databricks-expanded-interoperability-with-unity-catalog-open-apis — First wiki disclosure as a UC-integrated external engine. Named alongside Apache Spark and Apache Flink as one of three external engines that "can create and write to UC managed Delta tables with centralized governance and automatic optimizations" in the Beta. Composes with Delta Kernel (the Java/Rust library DuckDB leverages for UC-coordinated commits) and UC Credential Vending (the auth substrate). Specific role inside the article: DuckDB is the small-footprint single-node consumer in a list otherwise dominated by distributed cluster engines (Spark / Flink) — it represents the architectural class of "laptop or single-VM analyst doing managed-table writes against a governed catalog". Article framing: "thousands of customers use Unity Catalog to govern and access Delta Lake and Apache Iceberg tables, with dozens of integrations in the growing Unity Catalog ecosystem — from Apache Spark and Trino to DuckDB and Confluent Tableflow."
Last updated · 542 distilled / 1,571 read