Skip to content

PATTERN Cited by 1 source

Virtual policy instance per application

Problem

You embed a policy engine in a single host process (ingress proxy, service mesh filter, API gateway) that needs to enforce different policies for many tenants. Spawning one host process per tenant is expensive and defeats the embedding choice. Collapsing all tenants into a single global engine instance risks policy / data collisions, cross-tenant blast radius inside the engine, and label-cardinality explosion in telemetry.

Solution

Run one logically isolated policy-engine instance per tenant inside the single host process. Each virtual instance has its own:

  • Policy bundle (named after the tenant ID).
  • Labels (carried on emitted metrics + spans + decision logs).
  • Decision cache / status buffer.
  • Control-plane polling state (last fetched bundle revision, status report cadence).

The host process multiplexes across virtual instances; there are no additional OS processes, no additional containers.

The grace-period GC seam

Routes come and go continuously (deploys, canaries, A/B tests); immediately destroying a virtual instance whenever its last referencing route is removed would thrash bundle loading + status bootstrap. A grace period keeps recently-unreferenced instances warm; GC reaps them after a timeout. If a route referencing the tenant returns during the window, the warm instance is reused.

Shape

one host process
├── virtual-instance(A)  ← bundle "A", labels "app=A"
├── virtual-instance(B)  ← bundle "B", labels "app=B"
├── virtual-instance(C)  ← bundle "C", labels "app=C"
│      (grace period active; last route removed 30s ago)
└── ...

Trade-offs

  • No hardware-level isolation. A memory-exhausting policy in one tenant still pressures the shared process. Mitigate with bundle-size caps + bounded decision buffers (patterns/bounded-telemetry-data-structures-for-policy-engine).
  • Bundle-load cold-start. A new tenant's first request pays bundle fetch + compile. Warm-ahead policies can hide this.
  • Grace-period tuning. Too short → bundle thrash on route churn. Too long → stale tenants pin memory.
  • Cardinality discipline. Every metric / span emitted per virtual instance must carry the tenant label — plan for the label-cardinality cost.

Generalises beyond OPA

This pattern is the in-process multi-tenant engine shape. It applies to other embedded engines (rule engines, feature-flag evaluators, template renderers, validators) that would otherwise need process-level isolation per tenant.

Seen in

  • sources/2024-12-05-zalando-open-policy-agent-in-skipper-ingress — Zalando instantiates one virtual OPA instance per application ID referenced in any Skipper route. "Inside Skipper, we create one virtual OPA instance per application that is referenced in at least one of the routes. This allows us to re-use memory and also provides a buffer against high-frequency route changes by having a grace period for garbage collection." Explicit differentiator from the vanilla Envoy OPA plugin shape ("you typically run one OPA process per application").
Last updated · 550 distilled / 1,221 read