15. When Reasoning Breaks: Self-Correction, Verifier Reliability, Overthinking

1. Fix your mistakesverbatim published numbers

You have almost certainly tried this yourself: you ask a model a question, the answer comes back wrong, so you send it straight back with “check that again and fix any mistakes.” No hints, no tools, just re-reading. Measured across two rounds of exactly that, accuracy goes down.

The reason is structural: the same weights that produced the error are asked to find it, so the review mostly re-approves the answer, and when the model does change something, it flips correct answers to wrong more often than the reverse.^{reference 1} Nothing new enters the system; “are you sure?” just adds noise, and on CommonSenseQA one round of it cost GPT-3.5 half its score.

2. Thinking too hardour run · 100 problems · Qwen2.5-3B-Instruct · greedy

Step 1 was a revision you asked for. This one the model performs on its own: left running, a chain keeps writing, and what it writes can replace an answer it had already reached. Raise the token cap and page 6's accuracy curve climbs, though most of the climb is chains that were being cut off mid-thought finally reaching an end. But a rising curve only shows the balance of two things happening at once. Each doubling of the budget wins some answers and loses others, and the curve reports only the difference. The figure below separates them.

Every rung loses answers as well as gaining them, and those losses were never drawn until now. The top rung gains nothing, for a reason that is about the axis rather than about thinking: under greedy decoding a chain that has already stopped itself is not run again at a larger budget, it is the same generation, and 94% of these end themselves before 2048 tokens. The axis runs out before the model does.

So look at one of the losses. The budget above still drives what follows. Watch a single problem: intermediate_algebra/190, minimise (x+y)(y+z) given xyz(x+y+z) = 1. The answer is 2.

Cut at 128, 256, 512 tokens and forced to commit, it answers 2 three times over. Allowed to run to its own 818-token ending, it answers 4√3/3. The step that costs it is lit, and every line after it is correct arithmetic carrying a wrong assumption to a confident finish. It breaks on Qwen2.5-1.5B-Instruct and Qwen2.5-7B-Instruct too.

Not a strategy. Stopping every problem at 512 tokens scores 35% against 43% for finishing, and the best cut per problem reaches 49% for about 821 tokens where plain resampling reaches 49% at the same cost. This is page 7's coverage without page 8's selector: a better answer is in the pile and nothing we tested finds it.

The one aggregate claim here, and it is about capability rather than budgets: the better the model, the less of this it does.

Length alone is worth something. Guess right from wrong by how long the chain is and you score 0.26 over page 8's whole pile, because a correct candidate is the longer one only about a quarter of the time. Invert it, prefer the shorter chain, and you score 0.74, beating 2 of the models that page asked to judge, at no cost. Wrong answers elsewhere burn ~225% more tokens amid ~4× more strategy switches: thrashing, not depth.^{reference 2reference 3}

3. Scoring gone wrongour run · the same 100 problems · page 8's judges

Step 2 leaned on a judge, and page 8 sold one: selection beats voting, up to the ceiling its reliability sets. That sentence quotes reliability as though it were a property of the judge. It is not. It is a reading taken at a setting, and the setting is a knob page 8 turned two pages ago.

Temperature is what buys the candidate diversity page 8 was selling. It is also what blinds whatever has to pick from those candidates. Take one chain and pick nothing, and reliability drops 26 points across the sweep. Put a model asked to judge in charge of the picking and it drops anyway.

Only the line trained to score holds, and not by judging better: it sits mid-pack at the cold end and never improves. It finishes 11 points clear because everything else came down to meet it. What this sweep measures is the verifier, not the reasoning, and with an imperfect one, more resampling eventually lowers true accuracy outright.^{reference 4}

Candidate depth is held at 16 throughout, because reliability decays with that too. And reliability is undefined where no candidate is right, so the denominator shrinks as the temperature rises, which makes this decline the conservative reading.

4. What still works

Feedback from outside the model. Self-correction fails unaided; give the reviser real information, a failing unit test, a calculator, a proof checker, and revision has something to push against. Page 9 is why that is not a platitude: 1,062 of 1,580 real chains (67%) carry no closed-form numeric claim anywhere, so a calculator never fires at all on two chains in three.

Verifiers, where answers are checkable, and trained ones by preference. Selection with a judge (page 8) still beats blind voting, up to the ceiling its reliability sets. Step 3 adds which judge: the trained scorer held across the whole sweep while every model asked to judge lost ground.

Training, not prompting. The mid-chain “wait, let me re-check” that RL taught DeepSeek-R1 (page 14) was learned against verified rewards, not requested politely at inference time.