TSCT diagnostics + the 1.05 comment — 2026-10-01
Lane: the external review’s pre-100k checklist, items 1 and 2
(docs/research/2026-10-01-alphaxiv-answers-batch-3.md, “ПРЕДЗАПУСКОВОЙ ЧЕКЛИСТ”).
Gates tonight’s 100k launch.
What landed
Section titled “What landed”Three rare diagnostic series on the eval line, plus the comment fix. Cadence: eval steps only (default every 500). Every number is a host-side read of already-materialised factors at a boundary the run already syncs on — no new device sync was introduced (§1.3).
(a) Gram error of the FP masters, BEFORE and AFTER retraction
Section titled “(a) Gram error of the FP masters, BEFORE and AFTER retraction”No new metric. LinearLike::max_ortho already is the per-entry masters
reading (‖UᵀU−I‖_F / rank, AGENTS §2.3), so the pair is that function called
either side of retract_tsct* in train_loop. Ladder rung 2: it already
existed in this codebase.
(b) Gram error of the EFFECTIVE FORWARD factors at the current alpha
Section titled “(b) Gram error of the EFFECTIVE FORWARD factors at the current alpha”burn_spectral::ortho_error_forward(u, alpha) =
ortho_error_per_entry(ste_ternary_annealed(u, alpha)) — the same function
SpectralLinear::forward calls, at the layer’s own alpha. Computed on the
ternary-shaped effective factor, never on the raw master, which is precisely
the metric that would look perfect while the forward degrades.
(c) The |s| spectrum: max, min, count of near-off ranks
Section titled “(c) The |s| spectrum: max, min, count of near-off ranks”burn_spectral::spectrum_stats(s, 1e-3) → (max, min, off). off counts
|s_i| < 1e-3 (“почти выключенные ранги”), summed across every factor in the
model. 1e-3 is ~3 orders below the per-entry magnitude of a [768,64]
orthonormal factor (~3.6e-2); it is a reporting threshold, nothing branches on
it.
THE MEASUREMENT, and why it matters
Section titled “THE MEASUREMENT, and why it matters”On a freshly built [768,64] TSCT factor (fp32, CPU, rank 64 — the trainer’s
shape), measured with the new functions:
| metric | value |
|---|---|
masters on the manifold (‖UᵀU−I‖_F/64) |
2.27e-7 |
forward factor at alpha = 0 |
2.27e-7 (identical — the cross-check) |
forward factor at alpha = 0.25 |
1.64e-2 |
forward factor at alpha = 0.5 |
3.08e-2 |
forward factor at alpha = 0.75 |
4.33e-2 |
forward factor at alpha = 1 (the shipped default) |
5.39e-2 |
And on a real SpectralLinear::new(768, 768, 64) layer:
| metric | value |
|---|---|
| masters | 8.06e-8 |
forward at alpha = 1 |
7.96e-2 |
forward at alpha = 0 |
8.06e-8 |
|s| at init |
max 1.0, min 1.0, off 0 |
The gap is ~10⁶ and it is the reviewer’s warning, confirmed. The masters
are on the manifold to fp32 precision; the factor the forward actually
multiplies by is 5-8 orders of magnitude further from it. The ternary
projection with its 0.7·mean dead zone is not an orthogonal map, and
max_ortho — the only metric the trainer had — cannot see that by
construction: it is a function of the masters alone.
Two consequences that are facts about tonight’s run, not forecasts:
- The trainer’s
max_ortho1e-3 latch cannot see this. It reads 8e-8 on a factor whose forward factor is at 8e-2, so the latch never fires and the degradation it was installed to catch is invisible to it. The latch’s threshold is not wrong for what it measures (master drift) — it is just not a forward-health metric, and nothing in the log claimed it was. The live 2-step run above is the same thing at step 1:aft = 2.12e-8(latch quiet) againstfwd = 6.59e-2(66× over the latch’s own threshold). alphais 1.0 everywhere and nothing in the repo sets it (grep set_alpha→ onlyburn-spectral’s ownSpectralLinear::forwardand its diag example). So today’s 100k run is in thealpha = 1column: the worst one, and the column the metric now reports. Whether that 8e-2 per-entry Gram error hurts BPB is exactly what the pre-100k A/B (checklist item 3) is for; this lane measures it, it does not decide it.
The eval-line field
Section titled “The eval-line field”tsct=<bef>/<aft>/<fwd>/<smin>/<smax>/<off>@a=<alpha>bef/aft— masters’ per-entry Gram error either side of THIS step’s retraction.-whenretract_everydid not fire on the eval step (two identical numbers printed as a pair would read as a measurement and mean “we did not measure it” — the same reasoning that put names into thekda_askeddestructuring).fwd— forward-factor Gram error, worst over factors, at the layer’s alpha.smin:smax— the|s|spectrum’s range over every factor.off— near-off rank count, summed.a=— the largest alpha folded in.fwdis a function of alpha, so a number without it is not comparable across runs.max/sumconventions matchmax_ortho(worst-case) excepts_min, which is a min because a min is what “a rank died” looks like, andoff, which is a sum because one dead rank in one factor is the fact being reported.
None (no field at all) when the model has no TSCT factors — a
--set use_tsct=false run prints nothing rather than a row of zeros.
A REAL eval line (2-batch CPU run, --preset nano, 2 steps)
Section titled “A REAL eval line (2-batch CPU run, --preset nano, 2 steps)”step 1 EVAL ce=5.423 bpb=7.824 BEST over 256 B (fixed window) fused kda=0/0 asked=0 bwd=0 declined=0 ops=0 node_bwd=0 norm=0/0 muon_skipped=0/0 engram=8/8 fb=0/0 tsct=6.90e-6/2.12e-8/6.59e-2/1.000e0/1.000e0/0@a=1.00Read the tsct= field left to right:
| slot | value | what it says |
|---|---|---|
bef |
6.90e-6 | masters’ per-entry Gram error BEFORE this step’s retraction |
aft |
2.12e-8 | …and AFTER: the retraction does its job, 325× better |
fwd |
6.59e-2 | the factor the forward actually multiplies by |
smin:smax |
1.000e0 : 1.000e0 | s is all-ones at step 1, as initialised |
off |
0 | no dead ranks yet |
a |
1.00 | the annealing alpha the fwd number is at |
This is the whole finding in one line. The retraction is doing its job
(aft is 325× better than bef) and the masters are then held at 2.12e-8,
far under the 1e-3 latch — while the factor the forward multiplies by reads
6.59e-2, which is 66× OVER that same latch’s threshold. The latch cannot
fire: it is a function of bef/aft alone, and those are the two healthy
numbers. Nothing in the log before this field could have told a reader that
the deployed factor is 66× past a limit the model is nominally inside.
Item 1: the 1.05 comment
Section titled “Item 1: the 1.05 comment”burn-spectral’s POWER_ITERS doc and both mul_scalar(1.05) sites claimed
1.05 was a safety factor covering the power-iteration error, and the old
comment justified it as “converges from below, so the true sigma_max stays
inside the basin”. False as measured (2026-09-30, AGENTS §3.7 tail): over
200 random [768,64] factors the estimate is worse than 5 % on 32–47 % of
draws and the prescaled sigma_max reached 1.087.
Rewritten to what is true, with the arithmetic:
- Cubic suppression. Expanding about
s = 1,p(1+e) − 1 = (5/2)·e³ + (15/8)·e⁴ + (3/8)·e⁵— linear and quadratic terms cancel exactly (p′(1) = 0), so error is damped cubically. This is why the mis-estimate is benign: measured prescaled values up to 1.087 still returnσ = 1.000000000. - The basin, which is the real property.
p(s) − s = 0ats² = 1ands² = 7/3, sign negative between, so the attracting interval is0 < x₀ < √(7/3) ≈ 1.5275. A 1.05 prescale therefore tolerates an estimate as bad as1.5275/1.05 = 0.687— a 31 % underestimate — i.e. 6× the error the factor was documented as covering. POWER_ITERSbounds the estimate, not the factor; it isp′(1) = 0that absorbs the residue.
The literal 1.05 is now const SIGMA_OVERSHOOT (three call sites), so the
comment and the arithmetic cannot drift apart again.
Why c* is not learned (stated in the comment, because “just make it
learnable” is the obvious wrong next move): c multiplies the whole factor, so
a gradient on it is a gain-control mechanism — the optimizer would shrink
the factor to cut the loss, which is the exact thing the retraction exists to
prevent. c* = argmin_c max_{x∈[ℓ,u]} |p³(x/c) − 1| is measurable offline from
saved factors if the margin ever stops being enough.
Tests (all red→green by perturbation, not by literal)
Section titled “Tests (all red→green by perturbation, not by literal)”burn-spectral (cargo test -p burn-spectral --lib, ndarray):
| test | what makes it red without the metric |
|---|---|
ortho_per_entry_reports_a_known_perturbation |
fixture [768,64] with diag(1+eps) on the first k rows ⇒ Gram error exactly ((1+eps)²−1)/√k in closed form. eps = 0 must read < 1e-7; eps = 0.05 must read 1.281248e-2 vs closed form 1.281248e-2. An unperturbed read cannot produce it. |
ortho_per_entry_is_zero_on_the_manifold |
a retraction applied to an already-orthonormal factor must not report drift |
ortho_forward_departures_from_the_masters_as_alpha_rises |
three properties: at alpha = 0 exactly equals the masters’ metric to 1e-12; at alpha = 1 is > 100× it; monotone in between. Plus assert_eq! that the masters’ metric returns the SAME number for any alpha — that is the defect this series exists to expose. |
spectrum_stats_reports_max_min_and_dead_ranks |
fixture (2.0, 1e-4, −0.5, 0.0, 1.25) ⇒ (2.0, 0.0, 2); tolerance moves only the count (1 / 2 / 3 at 1e-4 / 1e-3 / 1.0); an all-alive spectrum reads 0, so a constant-returning metric fails |
Why the perturbation fixture is built rather than applied to a real factor:
perturbing an existing orthonormal U by ε moves the off-diagonal Gram
entries by ε·U_ij, so the expected value would depend on the fixture’s own
cross terms instead of on the measurement. The diag(1+eps) fixture has none,
which is what makes the expected number a property of the METRIC.
dormouse-train (cargo test -p dormouse-core -p dormouse-data -p dormouse-train --lib):
| test | what it pins |
|---|---|
tsct_field_format_is_golden_and_alpha_zero_is_the_cross_check |
the exact tsct= string; a dense linear prints no field; at alpha = 0 the forward metric equals max_ortho to 1e-12 relative (the built-in cross-check); at alpha = 1 they differ by > 100×; missing sides render - |
tsct_diag_runs_on_a_real_two_batch_cpu_eval |
the walk survives the real train_loop eval path (Run harness: batch 2, eval_batches 2, eval_every 1) over every factor incl. the ones lm_head’s fold owns — a unit test cannot see a fold that forgot lm_head |
cfg_retract_every_default_is_one |
the bef/aft pair is fully populated only because retract_every is 1. The external review asks for every-4th: if that moves, this assertion fires and the condition to re-check is eval_every % retract_every == 0 |
Carried, not fixed (other lanes / owner decisions)
Section titled “Carried, not fixed (other lanes / owner decisions)”max_ortho’s 1e-3 latch is not a forward-health metric, per the measurement above. Changing the latch to fire onfwdis a POLICY change with a one-way fallback attached and belongs to the owner, not this lane. The field now reports both so the decision can be made from a log.alphahas no schedule. Nothing in the repo sets it; the 100k run is atalpha = 1. Annealingalphais checklist item 3’s business.- A train at
alpha = 1runs the 8e-2 forward factor today. That is a measurement, not a verdict: whether it costs BPB is the A/B. - Under a non-Fp32
--quant(the sm_120 auto default is Fp8),fwdreports the TERNARY arm, not the deployed one —LinearLike::forwardtakesforward_quant(quant_factor(master)) wheneverquant != Fp32, and the ternary factor is not what it multiplies. Named here rather than silently averaged in. Follow-up if it matters: foldquant_formatinto the walk and report both. - Checklist item 3 (intervals 4 / 16 / none, ≥2 seeds, BPB on equal bytes AND equal wall-clock) and item 4 are untouched — they are runs, not code.
Discipline note
Section titled “Discipline note”Worktree wt/tsct-diag, branch off 3e1812b. /home was at 98 % with 12 G
free — a cold worktree build is ~41 GB, so this lane reused the shared warm
target/ dirs (CARGO_TARGET_DIR=…/target and …/vendor/dormouse-fused/target)
through tools/build_lock.sh. All cargo went through the build lock. No
cleanup was needed: the lane created zero bytes of target dir.