PATTERN Cited by 1 source
Push-pull schema detection¶
Pattern¶
Detect upstream schema changes through two complementary mechanisms:
-
Push-based: An upstream schema update generates a DDL CDC message that triggers an immediate comparison of the new schema against current downstream metadata (e.g., Iceberg catalog API). If a supported difference is found, schema evolution is invoked automatically.
-
Pull-based: A periodic (e.g., daily) comparison job independently checks online schemas against their offline counterparts and triggers evolution on drift.
Why both¶
- Push gives low-latency response — changes propagate as soon as they happen
- Pull is a safety net for anything the push path misses (CDC message lost, connector restart, edge cases)
The pull path also serves dual duty: if no drift is detected, it performs routine maintenance (e.g., small-file compaction for merge-on-read tables).
Seen in¶
- sources/2026-06-24-pinterest-automated-schema-evolution-in-pinterests-next-generation-db — canonical instance: Pinterest's CDC ingestion platform uses DDL CDC messages (push) + daily comparison job (pull) for schema drift detection