Skip to content

CONCEPT Cited by 1 source

Gap identification (top-N cap)

Definition

Gap identification with a top-N cap is the agent-output design principle that when a critic or reviewer enumerates missing evidence, uncertainty, or unresolved questions, it should limit its output to the top-N most significant gaps — not an exhaustive list.

The structural motivation: exhaustive gap lists produce reader fatigue. The downstream consumer (a planner agent, a human supervisor, the next round of a loop) cannot act on twelve gaps simultaneously. Forcing the critic to triage concentrates attention on the gaps that most matter and produces actionable next-round questions rather than a paralysing shopping list.

Canonicalised by Slack's Security Engineering team in Spear's Timeline task (Source: sources/2026-04-13-slack-managing-context-in-long-run-agentic-applications):

"We limit gap identification to the top 3 most significant gaps. This focuses the Director's attention on what matters most rather than presenting an exhaustive list of unknowns."

Slack's three gap categories

Slack's Timeline task classifies gaps into three explicit types (Source: sources/2026-04-13-slack-managing-context-in-long-run-agentic-applications):

Gap type What it flags Example
Evidential Missing data that would strengthen conclusions "SSH login event not captured; session activity observed from 09:29:01Z but initiation event missing"
Temporal Unexplained periods between events "Gap between 09:30:39 package ops and 09:31:26 ALERT — triggering command not documented"
Logical Events that don't fit the emerging narrative "Secondary analyst searched wrong path — reduces confidence in independent verification"

The typology is causally distinct: evidential gaps are about what we don't know, temporal gaps are about time we can't account for, logical gaps are about events that don't make sense together. A single gap belongs to one of these three — they are mutually exclusive.

Why a hard cap (not "up to" or "if any")

Three structural reasons:

1. Forces triage in the Critic

A top-3 cap is a scoring constraint the Critic must enforce: for every identified gap beyond the third, the Critic must either replace a currently-listed gap (this one is more significant) or drop it (not significant enough for top-3). This is a forced-choice scoring task, which is much easier for an LLM than open-ended enumeration.

2. Produces actionable Director input

The Director's next question is informed by the gaps. A top-3 cap produces a next-round question candidate set of three — which the Director can triage into one specific question. An exhaustive gap list produces a candidate set of twelve or twenty — which is effectively noise.

3. Scales with investigation complexity

A top-N cap is an architectural constant (N=3 for Slack); the investigation complexity varies. This means Slack's Timeline always reports three gaps whether the investigation is trivial or sprawling. This is intentional: the Director's next question can only be shaped by one gap at a time, so the cap is sized for the downstream consumer's capacity, not the source data's complexity.

Choosing N: the Magical Number Seven lineage

Slack picks 3. The choice echoes a lineage of short-working-memory / short-prompt design patterns:

  • Miller's 7 ± 2 (1956) — the classical working-memory capacity claim.
  • The rule of three in writing / rhetoric / UX — three-item lists are memorable and act-able.
  • Top-3 / top-5 / top-10 in search-result UX — diminishing attention past rank 5.

Three is the bottom edge of this range — aggressive, but defensible given the Director's downstream consumption pattern (asks one question per round, so most gaps will wait for future rounds anyway).

When not to use top-N capping

  • Audit / compliance contexts. A SOC audit reviewer may need the complete gap list for regulatory records. Top-N capping optimises for agent-consumable output; audit contexts sometimes need exhaustive output.
  • Human supervisor's first-look. When the human supervisor wants a "what's unknown about this investigation" one-shot view, top-N is a good default — but a "show me all unknowns" toggle should be available.
  • Cross-investigation aggregation. If gaps are aggregated across many investigations (to identify systematic blind spots), per-investigation top-N capping loses signal. Aggregation contexts should persist the full list and cap only at presentation.
  • Hard cap vs. threshold-based cap. Slack's is a hard cap (top-3 always). An alternative is threshold-based ("any gap with significance above 0.8") — but this produces variable output length, which is harder for downstream consumers to plan around.
  • Cap with overflow indicator. A refinement Slack does not disclose is "top-3 + total count" — tell the Director there are 14 gaps total but only the top 3 are shown. This preserves the acting-set while signalling completeness. Open question whether Spear does this.
  • Cap applied per-category or globally. Slack's post phrasing "top 3 most significant" implies globally (across all three gap types). An alternative is top-1 per category for balance. The post does not disclose which.

Contrasts

  • vs. exhaustive listing — the default output shape if nothing constrains it. Reader fatigue + downstream paralysis.
  • vs. first-N listing (no ranking) — returning the first 3 gaps found. Cheaper for the critic but produces worse downstream decisions than ranked top-3.
  • vs. summary paragraph — producing a narrative paragraph that discusses gaps. Harder for the downstream agent to parse and act on; Slack's typed-gap output is structured for machine consumption.
  • vs. gap score + filter — the Critic could emit all gaps with per-gap significance scores and let the downstream consumer filter. Slack chose to push the triage into the Critic (cheaper, more explicit).

Seen in

Last updated · 470 distilled / 1,213 read