# Page 11 — The Keystone, measured: training vs scale vs test-time compute

Everything page 11 draws from its own instrument, and the generations behind it.

```bash
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
jupyter lab page_11_training_vs_scale.ipynb
```

`RUN_MODE` is set to `REPLAY`. Run all: it loads no model and touches no GPU. Arm 0 re-derives
the accuracy-against-FLOPs family chart, and gate 1 checks it against pages 6, 7 and 8's shipped
aggregates. Those three `out/` directories are the arm-0 inputs; unpack pages 6-8's own bundles
beside this one, or flip to `FULL` on a 16 GB card to regenerate the measured arms from scratch.

## What is in here

| file | what |
|---|---|
| `page_11_training_vs_scale.ipynb` | the notebook that produced everything below |
| `harness.py` | shared plumbing: model loading (8-bit for the 7B on a 16 GB card), resume log, provenance hashes |
| `out/page_11/traces.jsonl` | **4,300 generations.** The general family (1.5B, 3B, 7B) on a fresh MATH-500 level 1-3 band, plus the math-specialist arm, cap 2048 |
| `out/page_11/derived.json` | the aggregates the page plots, plus the claims and pre-registered branch it is gated on |
| `out/page_11/manifest.json` | model revisions, hashes, seeds, versions, dates |

## What the page reads, and what it leaves in the file

The page plots the **no-crossing** result (sampling and voting never beat the bigger model's one
answer at matched FLOPs, on either difficulty band) and the **coverage-versus-selection**
mechanism (the right answer is in the pile far more often than any realizable selector returns
it). The specialist arm is measured and kept in `traces.jsonl` and `derived.json`, but the page
does not plot it: that comparison belongs to the frontier-reasoners page, not the keystone.

Temperature sampling is not reproducible across machines, so **the transcripts are the
provenance**. The analysis is deterministic from them. The grading harness is Sebastian Raschka's
[reasoning-from-scratch](https://github.com/rasbt/reasoning-from-scratch) (`load_math500_test`,
`render_prompt`, `extract_final_candidate`, `grade_answer` from ch03). Grading is not ours to
defend; the sweep and every number are ours.
