Skip to content

CONCEPT Cited by 1 source

Per-interval tag combination limit

The technique of bounding the number of unique tag combinations emitted per query pattern per aggregation interval, even when every individual tag key is low-cardinality. Handles combinatorial explosion: six tag keys with ten values each has 10⁶ possible combinations, far more than either the per-key cardinality tracker would catch or the pipeline can sustain.

Why per-key cardinality is insufficient

concepts/per-pattern-tag-cardinality catches "one key has way too many distinct values on this pattern." That's the common case but not the only one. The second failure mode is combinatorial: each of N tags has a modest cardinality (say, 10 values) but the Cartesian product is explosive:

  • 1 tag × 10 values = 10 combinations (fine)
  • 2 tags × 10 values each = 100 combinations (OK)
  • 3 tags × 10 values each = 1,000 combinations (stressed)
  • 6 tags × 10 values each = 1,000,000 combinations (fatal for per-unique-combination aggregation)

None of those keys individually trips a per-key cardinality threshold. The explosion comes from composition, which requires a different observer.

The per-interval combination observer

Within each aggregation interval (15 seconds in the PlanetScale canonical case), track the set of aggregates keyed by the full tag-combination tuple. If the aggregate count for a given query pattern exceeds a threshold (50 in the canonical case), trigger greedy collapse.

Greedy highest-cardinality collapse

Canonical collapse algorithm (Source: sources/2026-04-21-planetscale-enhanced-tagging-in-postgres-query-insights):

  1. Identify the tag key with the highest distinct-value count among the aggregates for this pattern.
  2. Replace all values of that key with * (the collapse marker).
  3. Merge aggregates that are now identical.
  4. If the aggregate count is still above threshold, repeat with the next-highest-cardinality tag key.

Worked example from the post: five executions of one query pattern with 2 distinct controller values + 4 distinct host values = 5 unique combinations. If threshold = 2, host (highest cardinality, 4 values) is collapsed to *, leaving 2 aggregates (one per controller).

The greedy choice — collapse the highest-cardinality tag first — minimises the number of keys that must be collapsed to reach the threshold. A non-greedy choice (random, or collapse-all-uniformly) would collapse more keys than necessary.

Trade-off: sensitivity over time

The per-interval limit is reset every 15 seconds — different intervals may produce different collapse decisions for the same pattern if cardinality varies. This is intentional: collapse is per-interval, so UX can say "in this 15-second window, the host tag was collapsed" rather than "for the last hour, host was always collapsed". Fine-grained collapse recovery when load drops.

(Contrast with concepts/per-pattern-tag-cardinality, which uses a 1-hour sticky window — once a key trips the per-key threshold on a pattern, it stays collapsed for the hour even if the next 15s window would be fine. The two mechanisms deliberately use different time scales to cover complementary failure modes: per-key catches persistent high-cardinality keys with coarse recovery; per-interval catches transient combinatorial blowups with fine recovery.)

Concrete PlanetScale parameters (as disclosed)

  • Threshold: > 50 unique tag combinations per query pattern per 15s interval.
  • Collapse order: greedy by highest-cardinality tag first.
  • Scope: per-pattern, per-interval.
  • Recovery: interval ends → new interval starts tracking combinations from scratch.

Seen in

Last updated · 347 distilled / 1,201 read