# Your AI Agent Is Not Reflecting. It Is Defending Its First Answer

Canonical URL: https://huecki.com/en/blog/ai-self-reflection-defends-first-answer/
Markdown URL: https://huecki.com/en/blog/ai-self-reflection-defends-first-answer.md
Language: English
Published: 2026-07-17
Updated: 2026-07-17
Author: Dominic Hückmann
Topic: AI Agent Workflows
- Agent topics: Agent Evals, Context Engineering, LLM-native Engineering
- Tags: AI Agents, Multi-Agent Systems, Self-Reflection, LLM Judges, Agent Architecture
- Audience: developers building agent workflows, AI product engineers, teams evaluating self-review loops
- Concepts: multi-agent-debate, llm-as-a-judge, agent-evals, self-reflection
Thesis: Reflection improves an answer only when the system can introduce genuinely independent evidence or hypotheses; otherwise it may merely polish the first mistake.
Content status: field-note

## Summary

Asking one agent to reconsider its answer often produces a more confident defense of the same mistake. A bounded challenger-and-judge loop can create real alternatives, but only if disagreement, stopping, and judge bias are engineered explicitly.

## Description

Why repeated self-reflection can reinforce an AI agent's first mistake, how multi-agent debate creates useful disagreement, and where the judge introduces a new failure mode.

## Body

The first answer is wrong.

So you ask the agent to reflect.

It rereads its reasoning, adds three paragraphs, sounds more careful — and reaches the same wrong answer with greater confidence.

This is one of the uncomfortable limits of AI self-review: **more thinking is not necessarily new thinking**.

If the same model starts with the same assumptions, sees the same context, and evaluates its own reasoning, the second pass may not challenge the first. It may rationalize it.

Researchers Tian Liang and colleagues gave this failure a useful name: [Degeneration of Thought](https://arxiv.org/abs/2305.19118). Once a model has established confidence in an answer, it can become unable to generate a genuinely different hypothesis through self-reflection — even when its initial position is wrong.

Their proposed remedy is Multi-Agent Debate: one agent proposes, another attacks the proposal, and a judge decides whether the debate has produced a satisfactory solution.

That is more than a prompting trick. It exposes a design choice every agent team needs to make:

```txt
Do you want another pass over the same thought,
or do you want a system that can produce a competing thought?
```

## The circle problem that exposes the failure

The paper uses a deliberately deceptive question:

```txt
A small circle with radius r rolls around the outside of a fixed circle
with radius 3r. How many times does the small circle rotate during one trip?
```

The intuitive answer is three: the large circumference is three times the small one.

It is also wrong. The small circle completes an additional rotation because its center travels around the large circle. The correct answer is four.

In the paper's example, chain-of-thought produces three. Self-reflection then checks the formula, validates the calculation, and confidently preserves three. The reflection is coherent. Its frame is incomplete.

The opposing debater introduces the missing hypothesis: the calculation must include both rotation around the small circle's own center and its movement around the large circle. The judge then selects four.

The important move is not “thinking longer.” It is **creating a second causal model of the problem**.

## Why self-reflection gets stuck

The paper identifies three contributing factors.

First, **bias and distorted perception**. The initial answer is often produced by a familiar pattern. Once that pattern is active, the review uses it as the lens through which all later evidence is interpreted.

Second, **rigidity**. A reflection prompt usually asks the model to inspect its own work. It does not necessarily reward abandoning the work and starting from an incompatible premise.

Third, **limited external feedback**. Nothing new has entered the system. The agent has the same model, the same task, the same context, and its own previous answer anchoring the next generation.

This explains why “take another look” is a weak correction strategy for failures caused by a hidden assumption. It may work for arithmetic slips, missing fields, inconsistent wording, or facts recoverable from the existing context. It is much weaker when the model needs a different interpretation.

The practical distinction is:

```txt
Inspection failure → reflection may help.
Framing failure    → introduce an independent challenge.
Knowledge failure  → retrieve evidence or use a tool.
```

Calling all three “reasoning problems” leads to the wrong architecture.

## A debate is useful only if disagreement is real

Two agent labels do not automatically create two independent thinkers.

If both roles use the same model, identical context, and nearly identical prompts, they may reproduce the same assumptions. “You are Agent B” is not a diversity mechanism.

A useful challenger needs a different objective and, where possible, a different information path:

```yaml
proposer:
  objective: produce the strongest supported solution
  context: task, available evidence, product constraints

challenger:
  objective: find the first unsupported assumption or construct a counterexample
  context: task, proposal, verification tools

judge:
  objective: decide criterion by criterion, with abstention allowed
  context: task, claims, evidence, challenge — not agent identities
```

For a coding agent, the challenger should not merely say that the patch “could be improved.” Give it concrete attack surfaces:

- Find an input that breaks the proposed fix.
- Identify a changed behavior outside the issue scope.
- Check whether the test would pass before the patch.
- Search for a public API or configuration contract the patch violates.
- Name the strongest alternative root cause.

For a research agent:

- Find the central claim with the weakest source support.
- Construct the strongest rival explanation.
- Check whether the cited source actually entails the claim.
- Distinguish an observed fact from the author's inference.

The challenger should be adversarial toward the claim, not toward the proposer. That difference matters.

## More disagreement is not always better

The paper found that debate worked best at a modest level of “tit for tat.” Explicit disagreement increased diversity and improved results. Forcing the agents to disagree with every point did not.

At the extreme, the task changes. The agents optimize for winning rather than finding the truth. They preserve incompatible positions, repeat arguments, and turn uncertainty into polarization.

This is a warning for agent prompts such as:

```txt
You must find a flaw.
Never agree with the other agent.
Argue until one side wins.
```

Those instructions guarantee visible conflict, not useful conflict.

The better contract is:

```txt
Try to falsify the proposal.
If a claim survives the strongest available counterexample, mark it supported.
If the evidence is insufficient, abstain.
Do not manufacture disagreement after the material issues are resolved.
```

## The judge becomes the new bottleneck

Multi-agent debate does not remove the evaluator problem. It makes it more visible.

The judge in the paper had two jobs. After each round, it decided whether a correct solution had been found. If the debate hit its iteration limit, it extracted a final answer from the full history.

That creates at least four failure modes:

1. The judge stops too early because one answer sounds complete.
2. It continues after the useful disagreement has been resolved.
3. It selects the more persuasive argument rather than the better-supported one.
4. It loses important evidence inside an increasingly long transcript.

The authors also found a more direct bias: when debaters used different backbone models, the judge tended to prefer the debater built on the same model as itself.

That result should kill the naive idea of an impartial LLM referee. A judge is another model with preferences, positional effects, verbosity bias, and model-family affinity.

Do not ask it which agent “won.” Ask narrow questions:

```yaml
claim_correctness:
  verdict: supported | contradicted | insufficient_evidence
  evidence: exact test, source, calculation, or counterexample

constraint_compliance:
  verdict: pass | fail
  evidence: exact requirement and observed behavior

material_challenge_resolved:
  verdict: yes | no
  evidence: response to the strongest counterargument
```

Hide agent identities where possible. Run pairwise comparisons in both answer orders. If `A vs B` and `B vs A` produce different winners, abstain or send the case to human review.

Most importantly, keep deterministic evidence outside the debate. A test runner should decide whether tests passed. A schema validator should decide whether the output is valid. A policy gate should decide whether a forbidden tool was used. Do not make the judge infer facts that software can verify.

## Two agents were better than four

It is tempting to read a positive multi-agent result and add more agents.

The paper tested that too. Translation quality fell as the number of debaters increased from two to three and four. The authors' manual analysis pointed to context complexity: debaters forgot earlier views, while the judge struggled to extract the important information from the longer transcript.

More agents create more tokens, more duplicated arguments, more coordination cost, and a harder synthesis problem. They increase potential diversity, but also increase noise.

For a production workflow, start with the smallest topology that can introduce a competing hypothesis:

One proposer and one challenger are enough for the first version. Add specialized roles only when eval traces show a stable failure class that the current pair misses.

## A buildable pattern for coding agents

Suppose an agent proposes a fix for an intermittent login bug.

The weak loop is:

```txt
Agent: writes patch
Agent: reviews its own patch
Agent: says the patch looks correct
```

The stronger loop is:

```txt
1. Proposer writes the smallest patch and states its root-cause hypothesis.
2. Challenger tries to falsify that hypothesis with one counterexample.
3. Test harness runs target, regression, and adversarial cases.
4. Judge scores only unresolved semantic criteria.
5. System accepts, requests one revision, or escalates.
```

Use a strict round budget. Most tasks need one challenge and at most one revision. If the same objection survives two rounds, another debate round is unlikely to create new evidence.

```yaml
stop_when:
  - deterministic_checks_fail
  - all_material_claims_are_supported
  - judge_returns_insufficient_evidence
  - one_revision_is_exhausted
  - cost_or_time_budget_is_reached
```

This turns debate from theatre into a verification stage.

## What the paper does not prove

The result is useful, but it should not be stretched beyond its evidence.

The main experiments use GPT-3.5-Turbo, GPT-4-0314, and Vicuna models. The tasks are commonsense Chinese-to-English translation and a 200-question counter-intuitive arithmetic dataset. These are deliberately chosen tasks where a plausible first interpretation is often wrong.

That makes them a good test for anchoring. It does not prove that debate improves every agent workflow, every current model, or long-running autonomous systems.

The gains also come with extra model calls and a more complicated failure surface. On the arithmetic set, debate improved GPT-3.5 substantially over its baseline, but it still remained below GPT-4. A stronger model with a verifier may be cheaper and more accurate than several weaker agents debating.

So the right baseline is not “one call versus an impressive multi-agent architecture.” Compare:

- one stronger model
- one model with retrieval or tools
- self-reflection
- best-of-N with deterministic reranking
- proposer plus challenger plus judge

Measure accuracy, hard failures, latency, cost, and human correction rate on the same frozen tasks.

If debate does not win that eval, remove it.

## The real lesson

The interesting finding is not that AI agents should argue like humans.

It is that iterative reasoning needs a source of novelty.

Self-reflection reuses the same model, context, and initial anchor. It can catch mistakes, but it can also produce a sophisticated defense of the first answer. Multi-agent debate helps when it introduces a real counter-hypothesis. Then the judge creates a new risk: the system may select familiarity, rhetoric, or model-family affinity instead of truth.

The robust architecture separates the jobs:

```txt
Generator → creates a candidate.
Challenger → tries to falsify its central assumption.
Tools      → produce external evidence.
Judge      → resolves narrow semantic criteria.
Eval       → proves whether the entire loop is worth keeping.
```

Do not add agents because a diagram with five boxes looks advanced.

Add one independent challenge where your failures show that the first answer is anchoring the rest of the run. Give it evidence. Give it a stop rule. Test the judge for bias.

The goal is not more debate.

The goal is a system capable of changing its mind for the right reason.

## Sources

- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://arxiv.org/abs/2305.19118)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://arxiv.org/abs/2305.14325)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://arxiv.org/abs/2306.05685)

## FAQ

### Does self-reflection make AI agents more accurate?

Sometimes, but not reliably. If the same model starts from the same assumptions and receives no new evidence, reflection can reinforce its initial mistake instead of correcting it.

### Is multi-agent debate better than one AI reviewer?

It can produce more diverse hypotheses, especially on deceptive reasoning tasks. It still needs a calibrated judge, evidence checks, a stopping rule, and an external eval.

### How many agents should participate in a debate?

Start with two roles: a proposer and a challenger. The cited study found that adding more debaters reduced performance because the growing context became harder for both debaters and judge to process.

### Can the same model act as proposer, challenger, and judge?

It can, but role prompts do not create true independence. Use different context views, require evidence, test answer-order effects, and calibrate the judge against human labels.


## Related

- Explains why open-ended self-review also creates unnecessary changes.: ai-self-check-always-finds-something
- Shows how external evals decide whether an agent workflow actually improved.: agent-is-not-the-product-fitness-function-is
- Covers the stopping rules required by iterative agent loops.: agents-need-exit-conditions

## Source References

- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://arxiv.org/abs/2305.19118) (paper)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://arxiv.org/abs/2305.14325) (paper)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://arxiv.org/abs/2306.05685) (paper)