Skip to content

PATTERN Cited by 1 source

Contract-driven topic provisioning

Pattern

Make the per-node contract.yaml the single source of truth for which topics exist on the broker, validated by regex + enum, and materialised by a single extractor that runs in CI, runtime boot, and post-boot validation.

The pattern eliminates silent wiring failures from topic-name drift across independently- developed agents in a multi-repo Kafka-shaped system. The ingredients:

  1. Per-node contract — every node ships a contract.yaml declaring subscribe_topics: and publish_topics:. See concepts/contract-yaml-as-bus-surface.
  2. Regex + enum validation — topic names follow onex.{kind}.{producer}.{event}.v{N}, validated by a regex for syntax and a StrEnum for canonical-name membership. See concepts/regex-plus-enum-validation.
  3. No second copy — "there is no second operator-maintained registry, separate constant list hidden inside the runtime, or manually synchronized provisioning config."
  4. Single extractor, multi-call-site — see patterns/single-extractor-multi-call-site for the topology.
  5. Narrow-scope provisioner — creates missing topics only, does not reconcile partition counts / replication / retention.

Disclosed instantiation

OmniNode's ContractTopicExtractor "discovers approved packages, loads each contract.yaml, and returns the union of declared topics." The extractor runs in three independent places:

new_topic = NewTopic(
    name=spec.suffix,
    num_partitions=partitions,
    replication_factor=spec.replication_factor,
)
await admin.create_topics([new_topic])

Provisioning is "async, best-effort, and non-blocking." If a topic exists, the provisioner "leaves it alone." See the contract.yaml shape:

event_bus:
  subscribe_topics:
    - "onex.cmd.router.route-request.v1"
    - "onex.evt.router.scoring-decision.v1"
  publish_topics:
    - "onex.evt.router.routing-complete.v1"
    - "onex.evt.router.routing-failed.v1"

Why this works

Three properties interact:

  1. Single source of truth eliminates drift — there is exactly one reviewed location where each topic name lives. Drift requires two copies; the pattern removes the second copy.
  2. Mechanical validation eliminates typo-as-silent-failure — regex + enum reject typo'd names at PR review, runtime boot, and post-boot validation. The bug class never reaches the broker.
  3. Topology eliminates "works locally, breaks in prod" asymmetry — the same extractor runs in CI and runtime, so "a topic name can only be wrong in the contract."

When to use it

The pattern's value is proportional to:

  • Number of topics — scale of bug surface. The OmniNode disclosure names "around the fiftieth topic" as the crossing point at which constants-near-publishers stops working.
  • Repository fan-out — single-developer-per-topic ownership doesn't need this discipline; cross-repo topic sharing does.
  • Refactor frequency — refactors are the primary source of "old topics left behind" drift; the pattern catches it mechanically.
  • Cost of silent failure — agent coordination is the OmniNode case; payment processing or healthcare workflows are higher- cost variants.

When the pattern is overkill

  • Single-repo / single-developer ownership — the constant-near- publisher style is simpler and the drift risk is low.
  • <20 topics — the policing overhead of contract.yaml + regex
  • enum + extractor doesn't pay back at small scale.
  • Hard-coded topic-name choice already enforced by code review conventions — if your team is already disciplined and the topic count is bounded, the contract layer is ceremony.

What it doesn't address

  • Partition-count drift — provisioner creates with the contract's partition count, but doesn't notice if the contract later requests more.
  • Replication-factor drift — same.
  • Retention-policy drift — same.
  • Schema drift — payload shape is a separate problem; pair with a schema registry (see concepts/schema-registry).
  • Producer/consumer offset drift — consumer-group lag is a separate observability problem.

The post is candid: "That boundary is intentional. Creation is contract-owned; reconciliation is a different problem." The disclosed future-work shape: "diff against the contract spec, decide which mismatches are auto-correctable, surface the rest as explicit drift."

Sibling patterns

Seen in

  • sources/2026-06-02-redpanda-how-omninode-uses-redpanda-to-scale-ai-agent-workflows (2026-06-02, Redpanda Blog guest post by OmniNode founder Jonah Gray) — canonical wiki instance of this pattern. Provides the contract-yaml shape, regex+enum validation, three-call-site extractor topology (CI / runtime boot / post-boot validation), the narrow-scope creation-only provisioner with explicit no- reconciliation boundary, and the bug class (concepts/silent-wiring-failure) the pattern was built to eliminate. "The contract becomes the only reviewed location where wire-format topic names live. There is no second operator-maintained registry, separate constant list hidden inside the runtime, or manually synchronized provisioning config." Disclosed scale: 12 repositories, 100+ event types.
Last updated · 542 distilled / 1,571 read