Skip to content

CONCEPT Cited by 1 source

AI re-review (incremental)

Incremental AI re-review is the discipline of re-running an AI code-review agent with full awareness of its own prior findings and the human responses to them, rather than treating each push as an independent, from-scratch review.

Why naive re-review fails

A re-run review that ignores history produces three failure modes:

  1. Re-flags already-fixed issues. The diff against mainline still contains the fix, so a naive reviewer sees the old code in context and re-raises it.
  2. Breaks comment-thread state. Every new run opens new DiffNote threads even when the same finding was already discussed and resolved.
  3. Ignores reviewer override. Humans explicitly replying "won't fix" or "acknowledged" get no durability — the next push re-opens the discussion.

The cumulative effect is that developers learn to either squash-before-review (defeating the iteration loop) or disable the AI reviewer (defeating the system).

The state the re-reviewer must consume

Cloudflare's AI Code Review coordinator, on every re-run, receives:

  • The full text of its last review comment — not just a hash; the actual prose verdict and enumerated findings.
  • A list of prior inline DiffNote comments it posted, with resolution status (open / resolved / closed).
  • User replies on each DiffNote — threaded, so the agent sees not just "human commented" but "human said X".
  • The new diff vs. the previously-reviewed baseline.

The judgment rules

The coordinator applies strict rules mapping prior-finding status + new evidence to re-emit behaviour:

Prior state Rule
Fixed (code no longer contains the issue) Omit from output → MCP server auto-resolves the DiffNote thread.
Unfixed (still present) Re-emit, even if unchanged, so the MCP server keeps the thread alive.
User-resolved (human marked resolved) Respect unless the issue has materially worsened.
User replied "won't fix" / "acknowledged" Treat as resolved.
User replied "I disagree" Read the justification and either concede (resolve) or argue back.

Cloudflare's framing: "If a developer replies 'won't fix' or 'acknowledged', the AI treats the finding as resolved. If they reply 'I disagree', the coordinator will read their justification and either resolve the thread or argue back."

Why "re-emit even if unchanged" is load-bearing

A reviewer that only emits new findings on re-run effectively resolves the old ones — the MCP comment server treats absence as closure. For persistent issues the author has not yet addressed, the agent must re-emit them to signal "still valid, don't close". This inverts the obvious intuition ("don't repeat yourself") for good operational reason.

Production implication

Cloudflare's first-30-day data point: the average MR is reviewed 2.7 times. Re-review is not an edge case; it is the majority of the total-run count. A system that degrades on re-run degrades on most of its workload.

Generalisation

The pattern applies to any long-lived AI-generated artefact consumed iteratively by a human:

  • AI-driven design reviews across document revisions.
  • AI-generated threat models that track fixes vs. re-introductions.
  • AI alert narratives over multi-incident lifecycles.
  • AI compliance scans re-run across quarters.

Rule: persist prior output, consume it on the next run, apply explicit state-transition rules, and re-emit unresolved items so the downstream tracking system preserves their status.

Seen in

Last updated · 200 distilled / 1,178 read