CONCEPT Cited by 1 source

Agent self-correction loop¶

Agent self-correction loop is an intra-trajectory mechanism by which an agent detects its own intermediate-step inconsistencies and revises earlier reasoning without an external test or oracle. The 2026-05-08 Databricks post on Genie names self-correction as a load-bearing structural capability for data agents, without which the agent would commit to wrong answers in the absence of verifiable tests.

The verbatim disclosure: "it requires the agent to have an ability to automatically correct itself when intermediate calculations reveal incorrect initial assumptions."

The shape¶

Within a single trajectory:

Plan
  │
  ▼
Step 1 (initial assumption A)
  │
  ▼
Step 2 (apply A, get intermediate result R)
  │
  ▼
[Inconsistency detector]
  │
  ├─→ R consistent with prior assumptions → continue
  └─→ R inconsistent with A → revise A → restart from Step 1'
                                                   │
                                                   ▼
                                                  ...

The agent is its own oracle — it doesn't need an external unit test to notice that "a column-level pricing detail contradicts my earlier revenue assumption." The detection comes from cross-checking intermediate results against earlier reasoning steps.

Why this is necessary for data agents¶

A coding agent can detect errors via the test suite: write code, run tests, see failures, fix. A data agent has no equivalent oracle for "the explanation is correct." Two failure modes the agent must detect internally:

Internal inconsistency — Step 5's calculation contradicts Step 2's stated assumption. Detect: the agent's own reasoning chain doesn't compose.
External-data inconsistency — the agent assumed Table X is the source of truth, but Step 7 found Table Y has more recent + authoritative numbers. Detect: re-read evidence and revise the source-of-truth assumption (concepts/source-of-truth-disambiguation).

Without self-correction, the agent commits to a chain of reasoning that becomes increasingly wrong as more evidence accumulates against its initial assumption.

Worked example (from Genie)¶

The post's worked example: a CFO asks why two enterprise dashboards report contradictory revenue spikes for the same product on different dates. The agent's trajectory:

Discovery — finds candidate tables, dashboards, pricing docs.
Investigation — extracts SQL, runs comparative analysis, investigates root cause.
Self-correction loop — "reconciliation" — when intermediate calculations reveal that an early assumption (e.g., "revenue is computed identically in both dashboards") is wrong, the agent revises and re-reasons.
Verification — final answer presented with the reconciled reasoning.

Without phase 3, the agent would have produced a plausible-but-wrong explanation grounded in the bad initial assumption.

Self-correction vs parallel thinking¶

These are different axes of correctness compensation:

Axis	Mechanism	Operates over
Self-correction	Detect intra-trajectory inconsistency; revise	One trajectory, multi-step
concepts/parallel-thinking-trajectory-sampling	Sample N trajectories; aggregate	N trajectories

They compose — Genie does both:

Each trajectory runs a self-correction loop (internal consistency).
N trajectories run in parallel and aggregate (cross-trajectory consistency).

This is double-redundant correctness signalling — neither alone is sufficient given the verifiable-test gap; together they bracket the agent's answer in two complementary ways.

What triggers self-correction (not disclosed for Genie)¶

The Databricks post does not specify how Genie's self-correction loop is implemented. Plausible mechanisms (from the broader literature):

Judge sub-agent — a separate LLM evaluates each intermediate step and flags inconsistencies.
Anomaly detection — statistical signals on intermediate results (e.g., "this revenue number is 100× higher than expected") trigger re-investigation.
Constraint checks — schema-level / business-rule constraints fail → revise.
Backtracking — when later steps depend on earlier assumptions that turn out wrong, unwind and re-plan.

The specific mechanism Genie uses is not publicly disclosed as of 2026-05-08.

When this fits / doesn't¶

Fits:

Agent operates without verifiable oracles (data agents, research agents, deep-investigation agents).
Multi-step trajectories where intermediate evidence can contradict earlier assumptions.
Cost budget allows replay / revision of partial reasoning chains.

Doesn't fit / breaks down:

Trivially-short trajectories (no room for inconsistency to develop).
Agent design with no judge sub-agent or constraint checker — nothing detects inconsistency.
Cost budget too tight to allow replay — must commit to first reasoning chain.
Pathological case: bad initial assumption causes all downstream reasoning to be self-consistent in a wrong way (no inconsistency signal). This is why parallel thinking complements — different trajectories explore different initial assumptions.

concepts/verifiable-test-gap-data-queries is the problem self-correction addresses (substitutes for missing oracle).
concepts/parallel-thinking-trajectory-sampling is the cross-trajectory equivalent; this is the intra-trajectory equivalent — they compose.
concepts/source-of-truth-disambiguation is one specific kind of self-correction trigger — assumptions about which source is authoritative get revised mid-trajectory.
patterns/four-phase-data-agent-trajectory is the broader pattern in which self-correction is phase 3.

Seen in¶

sources/2026-05-08-databricks-pushing-the-frontier-for-data-agents-with-genie — canonical first wiki disclosure of agent self-correction loop as a named structural capability for data agents. Verbatim: "the ability to automatically correct itself when intermediate calculations reveal incorrect initial assumptions." Positioned as load-bearing for the four-phase data-agent trajectory shape; composes with parallel thinking as complementary correctness- signalling mechanism.