Status: the measured numbers
Results
Section titled “Results”Measured, dated, reproducible — the numbers below are the state of 2026-10-19. Retractions are published next to the numbers (the retracted list); three artifacts of this lane (b1night, medium_big2, sft9) were invalidated on 2026-10-19 by the expert-bank cache defect and are named in AGENTS.md §3.2.
| what | number |
|---|---|
medium pretrain (mix1, web+wiki+ru+math mixture) |
BPB 1.936 held-out (196,608 B window) after 1.47 GB seen |
| speed, medium (b24 × s1024) | 40.9k tokens/s, ~12 GiB peak, one RTX 5060 Ti |
| speed, b1 (b8 × s512) | ~8k tokens/s eager, 13 GiB |
| first downstream numbers (zero-shot, n=1000, random = 25%) | HellaSwag 30.5%, ARC-Easy 27.2%, synth-math 24.0% — at 0.37B tokens seen |
| test suite | 968 tests, ruff clean |
The bench row is a floor, not a comparison: 30.5% HellaSwag is above the 25% random
baseline and below SmolLM2-135M’s 42.1 — and the two are not the same measurement
(this harness is byte-level acc_mean over n=1000; the published number is token-level
acc_norm under lm-eval). The protocol bridge is unbuilt, so treat every cross-model
row here as an order-of-magnitude statement and nothing finer. The data gap is the
reason: 0.4B byte-tokens seen against 13B-2T for the models in that table.
Every mechanism in the tree was admitted by a 3-seed A/B on a fixed protocol, and the losers are recorded, not deleted quietly (the full ledger, with the delta and the verdict for each arm, is docs/reviews/2026-10-19-model-scorecard.md): the ColRow fix (−0.043 BPB, the largest single win), depth-3 (−0.007), dropless routing, Triton Newton–Schulz (52 dispatches → 1), optimizer CUDA-graph (−11.7 ms/step), bf16 Muon momentum (−1.23 GiB), Engram in host RAM (bit-identical). Rejected by measurement: NorMuon (+0.072), schedule-free Muon (+0.043), SIGReg (+0.148), LOOM (tie), MSA at short context (tie), ternary weights from scratch (+0.04), KDA recompute flag (tie).