Skip to content

PATTERN Cited by 1 source

Bootstrap Then Auto-Follow

Pattern

When a policy-based automation primitive only applies to resources created after the policy is installed, cover the pre-existing set with an imperative bootstrap pass before relying on the policy, rather than assuming the policy will backfill.

Concretely:

  1. Enumerate the set of resources the policy would match if it matched retroactively.
  2. Filter to the ones actually in scope (e.g. exclude system indexes).
  3. For each, call the policy's per-resource attachment API imperatively (idempotent: treat already-attached as success).
  4. Then install the pattern-based auto-follow / auto-attach policy so future new resources are covered automatically.

The pattern has a clean two-phase shape: imperative backfill → declarative forward-coverage.

When to use

Any policy-based attachment / replication / labeling system where the policy is new-only, and the target population has a non-trivial existing set you can't afford to leave unattached. Typical instances:

  • Elasticsearch CCR auto-follow policies (the canonical wiki instance).
  • Kubernetes admission-controller mutations that don't retro-apply to existing objects.
  • Service-mesh sidecar-injection labels that only take effect at pod (re)creation.
  • Cloud-resource tagging policies that apply forward from a given timestamp.
  • Database row-level-security policies that gate only new inserts.

Canonical shape (from the GHES rewrite)

The 2026-03-03 GHES search rewrite paper publishes the exact pseudocode:

function bootstrap_ccr(primary, replica):
  primary_indexes = list_indexes(primary)
  replica_indexes = list_indexes(replica)

  managed = filter(primary_indexes, is_managed_ghe_index)

  for index in managed:
    if index not in replica_indexes:
      ensure_follower_index(replica, leader=primary, index=index)
    else:
      ensure_following(replica, leader=primary, index=index)

  ensure_auto_follow_policy(
    replica,
    leader=primary,
    patterns=[managed_index_patterns],
    exclude=[system_index_patterns]
  )

Three structural properties worth calling out:

  • Two paths per pre-existing resource — ensure_follower_index (create the follower from scratch) vs ensure_following (attach follower semantics to an already-present index). The imperative backfill has to handle both "not present on follower yet" and "present but unmanaged."
  • System-indexes excluded from both the imperative pass and the forward-coverage auto-follow policy. The exclusion list is the discipline's weak point — any new resource the storage layer creates under the hood that doesn't match the pattern has to be anticipated.
  • Imperative pass first, then policy — the order matters. If you install the auto-follow policy first then run the bootstrap, the policy might try to attach the same resource concurrently with the imperative call.

Trade-offs / failure modes

  • Bootstrap idempotency needs to be strict. Re-running the bootstrap mid-attach (e.g. on installer retry) must not break the partially-attached state.
  • Policy drift: the pattern on the auto-follow policy and the filter on the imperative bootstrap must be the same set. Divergence means new resources land without follower attachment (policy narrower than bootstrap) or bootstrap skips things the policy would have matched (bootstrap narrower than policy).
  • System / hidden resources: storage layers create indexes / topics / tables for internal book-keeping. If one sneaks into the managed set, the bootstrap can attach it with unintended semantics; if one sneaks out of the exclude list over a version upgrade, it gets skipped.
  • Not a general backfill: if the policy is more expressive than attachment (e.g. lifecycle policy with retention) and the policy has effects on state, the bootstrap must reproduce those effects imperatively, not just attach-the-resource.

Canonical wiki instance

The GHES 3.19.1 CCR migration: each pre-existing managed index is followed imperatively before the auto-follow policy for managed_index_patterns is installed. One of the visible consequences of the pattern shape: a new GHES install could skip the bootstrap and rely only on the auto-follow policy (there's nothing pre-existing), whereas an upgrade install needs both passes. The migration code path branches on this. (Source: sources/2026-03-03-github-how-we-rebuilt-the-search-architecture-for-high-availability)

Seen in

Last updated · 200 distilled / 1,178 read