Skip to content

SYSTEM Cited by 2 sources

Iceberg REGISTER / UNREGISTER APIs

The REGISTER and UNREGISTER endpoints are a pair of Apache Iceberg REST catalog operations that together let a table be handed off between independent Iceberg catalogs without copying, exporting, or rewriting a single data file. They turn "catalog portability" into a metadata-pointer transfer rather than a data migration. REGISTER was already part of the Iceberg REST spec; UNREGISTER was contributed by Databricks to the specification to complete the handoff (Source: sources/2026-10-05-databricks-unlocking-data-portability-preventing-catalog-lock-in-with-register-and-unregister).

Why they exist

In an open table format, the catalog is not where the data lives — schema, history, statistics, and the Parquet data files all sit in object storage, decoupled from compute. The catalog's job is narrow but critical: be the single authority for the table's latest metadata pointer, and coordinate commits so two writers can never silently overwrite each other. An engine queries a table by first asking the catalog to load it (returning the current metadata + storage location), then reading data files straight from the bucket.

Because the catalog is just the authority over a pointer, moving a table to a different catalog should be cheap — you only need to transfer who owns the pointer. REGISTER + UNREGISTER are the two halves of doing that transfer safely.

REGISTER

  • Request: a client POSTs the desired table name + the URI of the table's existing metadata.json location.
  • Action: after basic validation, the catalog "writes a single record linking the table name to the existing metadata.json. No data files are copied, moved, or altered."
  • Status: already defined in the Iceberg REST catalog specification.

REGISTER alone is the attach half of a handoff — and using it alone is unsafe (see below).

The split-brain hazard of REGISTER-alone

Iceberg catalogs are independent and do not communicate. Registering a table in catalog B while leaving it active in catalog A produces a split-brain: both catalogs believe they are the sole owner and will each coordinate commits. "Neither will throw an error, but the first write operation will fork the table, leading to inconsistent query results and silent data loss." A write made through Catalog A is invisible to Catalog B — "permanently destroying your single source of truth."

DROP TABLE cannot be used to clean up the old side, because DROP deletes the underlying data and metadata. Historically the ecosystem "lacked a standard way to cleanly terminate a catalog's management of a table" without destroying its data — so the REGISTER handoff was missing its second half.

UNREGISTER

  • Request (empty POST to the table resource): POST <uc-iceberg-rest-base>/v1/{prefix}/namespaces/{namespace}/tables/{table}/unregister
  • Action: removes the table's entry from the managing catalog "without touching a single underlying data file."
  • Response: the table's latest metadata location — the exact pointer the next catalog needs to assume commit coordination.

By making the original catalog explicitly relinquish control and emit the handoff pointer, UNREGISTER eliminates the split-brain risk.

The safe handoff (ordered three steps)

Catalog A (owner)                         Catalog B (new owner)
      │                                          │
      │ 1. UNREGISTER table ──────────────►      │   (files stay in place)
      │    - A forgets the table                 │
      │    - returns latest metadata location    │
      │                                          │
      │ 2. metadata-location pointer ───────────►│
      │                                          │
      │                             3. REGISTER (name, metadata.json URI)
      │                                          │   B becomes sole
      │                                          │   managing catalog
  1. UNREGISTER on Catalog A — removes A's entry, returns the metadata location. Data files are untouched.
  2. Receive the returned metadata location — the handoff token.
  3. REGISTER on Catalog B with that location — B becomes the table's sole managing catalog.

This ordering guarantees the invariant the design protects: the table has exactly one managing catalog at all times. It is a concrete instance of separating revoke from establish — revocation (UNREGISTER on A) must complete before establishment (REGISTER on B), or both catalogs transiently satisfy the ownership invariant and the single-owner property breaks.

Why it matters — catalog portability without lock-in

Because no data ever moves and ownership is a transferable pointer, organizations keep "the freedom to choose and move between catalogs as their architecture evolves." This is the structural basis for the claim that managed catalogs like Unity Catalog are "not lock-in" — the same portability argument that the open table format + open REST-catalog APIs make about data readability, now extended to catalog ownership itself.

Seen in

Caveats

  • Not the full migration runbook. The post explicitly notes a production migration "still involves operational steps, like safely stopping writers and repointing jobs" — deferred to a follow-up. The three API calls are the correctness core, not the complete operational procedure.
  • UC availability is private preview at publication. UNREGISTER is contributed to the Iceberg REST spec, but support in other catalog implementations (Apache Polaris, Snowflake Open Catalog, AWS Glue) is not stated.
  • No latency/scale numbers are given; this is a mechanism explainer.
  • The split-brain window of a naïve REGISTER-only move is silent — no error from either catalog; divergence surfaces only on later reads/ writes. The explicit UNREGISTER relinquish step is what closes it.

Source

Last updated · 854 distilled / 2,448 read