Act 2 · The Toolbox

Download the notebook (zip, 10.1 MB) ↓

8. Best-of-N and Verifiers: Generate, Then Select

Counting asks what is most common. Judging asks what is best. They are not the same question, and on 100 problems scoring 16 candidates and keeping the best takes 43% to 60%: 21 problems fixed against 4 broken, p < 0.001. Then the part nobody promised us: page 7's vote, reading the same candidates, gets there too.

1. Judge one batch two ways

Page 7 counted the chains and kept the answer most of them agreed on. Here the same pile is read a second way, one candidate at a time by a judge that scores each on its own. That judge reads one finished candidate and returns a single number, which is where that candidate's dot sits on the axis below. It never sees the tally, so the rightmost dot is not counting in disguise. Below is a problem where the two rules disagree, and beside it both rules across all 100 problems.

  • measured 2026-08-07
  • Qwen2.5-3B-Instruct
  • 100 MATH-500 problems, 16 chains each

Find the greatest common divisor of 3339, 2961, and 1491.

16 chains. The vote takes the longest row: 3 on 9 of 16, and it is wrong. The verifier, a second model trained to score finished solutions rather than to write themreference 1, takes the rightmost dot: 21, which is right. The answer is 21.

3 (9)21 (5)1 (1)189 (1)what the verifier scored it →
  • a correct chain
  • a wrong one
  • what the verifier kept
0255075100124816tiescandidates the rule may look atproblems solved (%)
  • best-of-N, trained verifier
  • majority vote
  • one chain, on average (46%)
  • a correct answer is present

The vote dips below one chain at two candidates, and that is the rule rather than noise: with only two, any disagreement is a tie, and page 7 counts a tie as no answer. Voting needs a crowd before it beats not voting.

Click a dot to read the chain behind it.

2. What each rule fixed, and what it broke

One row per selection rule, each against the same single chain on the same problems. The majority-vote row is page 7's counting rule, re-run here on chains this page generated itself.

  • measured
  • 100 problems
  • every rule reads the same chains
fixedbrokehand-written length heuristic-8 · p = 0.152answer log-probability0 · p = 1.000best-of-N, 1.5B generalist+7 · p = 0.189best-of-N, the writer itself+8 · p = 0.115best-of-N, 1.5B math specialist+10 · p = 0.052majority vote+13 · p = 0.007best-of-N, 7B generalist+13 · p = 0.007best-of-N, trained verifier+17 · p < 0.001problems, against one fixed chain
Every rule against the same fixed chain per problem, which solves 43%, on the same 100 problems, tested where the two disagree. The trained verifier fixes 21 and breaks 4. And the heuristic scorer from the source we build onreference 2 breaks 16 and fixes only 8: every candidate here ends in a boxed answer, so its boxed bonus cancels and the rule reduces to shortest wins. A judge that grades presentation is worse than not choosing at all.

3. What a judge is worth

Five judges read the same candidates, and one of them is the trained verifier from the figures above. It is Skywork/Skywork-o1-Open-PRM-Qwen-2.5-1.5B, a process reward model read at the final token only: one number for the whole solution. Grading the steps is the next page.

  • measured
  • 5 judges
  • 4 temperatures each
majority vote, 68%506070801.5B math specialistsolves 64% on its ownmeasured here59%7B generalistsolves 60% on its ownfrom page 665%the writer itselfsolves 43% on its ownfrom page 660%1.5B generalistsolves 22% on its ownfrom page 664%trained verifierscores only, never answers73%picks a correct candidate (%)
Five judges spanning size, subject and training objective, each averaged over four temperatures with the bar spanning its best and worst. As you can see from the graph, being good at solving the problems is not the same thing as being good at judging them. This run shows that the best solver on the ladder (the 1.5B math specialist) ranked last as a judge, and the only judge whose mean sits above the vote is the trained verifier, which ironically cannot solve any of these problems on its own.

The 7B generalist and 1.5B math specialist solve more of these problems than the model that wrote the candidates, which solves 43%. Nothing here separates being a better judge from being a better solver. Set them aside and the judge that never answers is still no better than the vote.

4. Where this leaves us

Selection works. Against the same candidates, so does counting: the verifier is +4 problems ahead of the vote, p = 0.454, which is not a difference. Two routes to the same place, and one of them is free.

That contest is ours. Counting against selection is not one anyone else ran: self-consistency and best-of-N are introduced in different chapters of the source we build onreference 2 and are never set against each other there.

And page 7's leftovers are still mostly leftovers. A right answer was in the pile on 78% of problems, and on the problems where one exists the vote returns it 68% of the time against the trained verifier's 73%. Judging recovers a little of what counting discarded, not most of it.reference 3

So the question is not how many candidates, nor even how good the judge is, but what the judge is reading. Every rule here grades the final answer and none of them looks at whether step three followed from step two. Turn the page.