maxopt2 lane findings — three no-graph levers
Lane: maxopt2. Base: main 0822403. Control = gbench2_control recipe
(small, b8, s512, max_iter=4, Fp8 factors, Muon+ ColRow ns=8, JEPA 0.05
- KoLeo, dspark off, no engram, retract-every 4, seed 1). Control warm
step: 485.8 ms (first read), 457.3 ms (quiet-card repeat), both in
~/logs/gbench2_control.log. NOTE vs the lane brief: the brief says “depth 2”; the control snapshotcheckpoints/gbench2_control.config.tomlcarriesmax_iter = 4, and that is what every control number quoted on this lane ran. Flagged, not silently “fixed” — A/B cells below use the control’s actual shape (b8 s512 max_iter=4).
Eval window for every BPB quoted: eval_batches x batch x seq_len =
20 x 8 x 512 = 81920 B (printed per run; the byte count on the eval line
is the authority).
Lever 1 — TSCT retraction cadence
Section titled “Lever 1 — TSCT retraction cadence”A/B --retract-every 1 vs 5 vs 10, 2000 steps, 2 seeds each, held-out
BPB at steps 500/1000/1500/2000 in one window.
(results land per cell; logs ~/logs/maxopt2_re_s.log; rows into benches/history.tsv)
Lever 2 — JEPA teacher out of the hot loop
Section titled “Lever 2 — JEPA teacher out of the hot loop”--jepa-precompute 2000 --jepa-targets then a 2000-step run with
--jepa-targets. Atlas (DM_LAUNCH_ATLAS=1) before/after, warm ms/step,
and the 2k-step quality delta vs the in-loop-teacher run at the same seed.
Lever 3 — Muon+ head-wise Q/K NS: batched
Section titled “Lever 3 — Muon+ head-wise Q/K NS: batched”The head-wise Q/K group ran one Newton-Schulz per head (12 x the matmul
chains). Now one stacked NS over [12, 64, 768] with identical per-slice
math (orthogonalize_batched, crates/dormouse-train/src/optim.rs).
Gate: cargo test -p dormouse-train --lib optim (equivalence +
existing headwise tests), opt ms before/after from the atlas.
Results table
Section titled “Results table”| lever | ms/step before | ms/step after | BPB (held-out) | verdict |
|---|---|---|---|---|
| 1 | ||||
| 2 | ||||
| 3 |