Schedule-free Muon — does it exist, and what of it survives our port?

Date: 2026-10-16. Read-only: docs/research/ only; no code, no config, nothing committed. Ask: the owner will not accept a schedule-free AdamW — they want it for Muon, our primary optimizer. Does a schedule-free (or warmup-free / LR-schedule-free) Muon exist, and if not, is the minimal derivation sound? Method: arXiv MCP, 14 queries across 6 angles (SF×Muon, SF theory, Muon theory, schedule elimination, clipping/stability, NorMuon), every candidate id resolved with get_abstract; two papers read by section (AMUSE 2605.22432 §5/§9/§10; SF-NorMuon 2605.23061 §3.1/§3.2); one citation graph. Repo read: port/src/dormouse/train/optim.py (our Muon), port/src/dormouse/train/lr.py (WSD), AGENTS.md §2.3, docs/protocols/AB-PROTOCOL.md, docs/research/2026-10-16-new-tech-scan.md. Marks: [A] = the paper’s own words (abstract or section read) · [ours] = repo fact, file named · [S] = SPECULATION (my inference, no source). Ids verified 2026-10-16.

TL;DR

1. It exists, twice, since May 2026 — the baseline scan is wrong about the conclusion, not the window. 2605.22432 AMUSE is schedule-free Muon at 124M/720M/1.3B, 2605.23061 SF-NorMuon is schedule-free spectral (NorMuon) at 125M/772M with a stationarity theorem [A].
2. The scan’s line 54 (“no real id found” for schedule-free Muon) is true only of 2026-07..10, and its row 27 dismissal (“SKIP — schedule-free was already evaluated”) conflated schedule-free AdamW with schedule-free Muon: 2026-10-07-technology-super-research.md #3 evaluated ScheduleFree+ 2605.19095, an AdamW/LLM result that never names Muon [A].
3. Nothing newer than 2026-05-21 exists in the home window. ti:"schedule-free" returns 8 papers, newest 2607.09167 (theory, Lyapunov analysis of SF-GD/SGD) [A]; no 2026-07..10 SF-Muon appeared.
4. The transfer of SF theory to a non-gradient update map is already published for the exact-polar case — the SF-Spectral terms (descent + noise + SF-drift + momentum-tracking) hold with the polar factor because polar is the LMO over the spectral-norm ball [A, 2605.23061 Thm 2.1]. Our port sits outside that theorem in three places (§3): 5-step NS + ColRow instead of polar(), no weight decay on the Muon group (optim.py:234-237), and a per-step polar retraction of the TSCT factors (AGENTS.md:406).
5. Verdict: DERIVE — adopt AMUSE’s shape as the arm template, not its numbers; gate the three port-specific mechanisms before the BPB is believed. The 2k-step A/B below is likely to tie, and a tie there is not a refutation: the published SF gains are stated at ≥1000 tokens/param [A, 2605.19095].

1. Candidates (STEP 2) — 25 ids examined, 9 rows, ranked by fit to port/

#	candidate (id, submitted)	what it actually does [A unless marked]	scale tested	Muon-compatible?	verdict
1	AMUSE 2605.22432 2026-05-21 (v2)	SF + Muon: gradients evaluated at Y_t=(1-β_t)Z_t+β_t X_t with time-varying β_t=1−(T₀/t)^ρ(1−β_init); no LR schedule, keeps linear warmup; trust-region-style state is (params at y, z, momentum m), x interpolated in place for eval then reverted [§5.4]	124M / 720M / 1.3B Llama, FineWeb-100B, seq 512, batch 256/1984/2048; 2.1B/16.3B/≈26B tokens; + vision	yes — same polar+ColRow-class update; non-Muon params go to SF-AdamW [§5.4]	ADOPT (arm template)
2	SF-NorMuon 2605.23061 2026-05-21 (v2, JPMorgan)	SF spectral descent with an explicit momentum buffer M_t, Z_{t+1}=Z_t−ηλZ_t−η̂P̂_t, X_{t+1}=(1−c)X_t+cZ_{t+1} with c_{t+1}=η_t²/s_t (not 1/(t+1)); row-wise adaptive normalization (NorMuon); weight decay at the fast iterate Z	125M + 772M, FineWeb-100B, 1–8× Chinchilla; matches tuned AdamW, trails cosine NorMuon by ~0.03 nats	yes — this is the closest published neighbour of our exact update	ADOPT (algorithm + theorem)
3	SF-Spectral theorem (inside #2) 2605.23061 §3.1	Assumptions: Frobenius L-smoothness, unbiased bounded-variance gradient, bounded diameter; bounds (1/T)ΣE‖∇f(Y_t)‖_* (nuclear-norm dual) with an explicit SF-drift term and a momentum-tracking term; Õ(T^{−1/4}) noisy, Õ(T^{−1/2}) noiseless; wd at Z lifts the bounded-diameter assumption (Lemma 3.1)	theory	yes — stated for polar(M_t), i.e. the exact LMO, not for NS-5	DERIVE (as the proof skeleton)
4	ScheduleFree+ 2605.19095 2026-05-18, Defazio · Through the River 2507.09846 2025-07-14	LR-free + schedule-free for LLMs (“greatly outperforms WSD”, +31% at 1000 tokens/param); SF-AdamW implicitly performs weight averaging without memory overhead	LLM scale / theory+LM	no — AdamW-shaped	READ-ONLY (this is the paper our baseline actually evaluated; the mechanism we are borrowing)
6	TrasMuon 2602.13498 2026-02-13 · OptMuon 2606.08783 2026-06-07	trust-region adaptive magnitude for orthogonalized momentum (“experiments without warmup” confirm stability); closed-loop AdaGrad-Norm-style magnitude on the polar direction, Õ(T^{−1/2}+σ^{1/3}T^{−1/3})	vision+language / theory only in the abstract	yes (Muon-class)	SKIP as the arm (both confound schedule-elimination with a new magnitude rule); cite TrasMuon for the warmup-free claim
8	SOLAR 2609.34681 2026-09-28	learned online LR policy (bounded residual on a base schedule, PPO controller) for AdamW and Muon, 60M–1B, MoE to 3B	60M–1B	yes	SKIP — it learns a schedule; opposite of removing schedule state, and it adds a reward loop
8	Musec 2609.11655 2026-09-10 · MuCon 2605.26459 2026-05-26 · ADANA 2602.05298 2026-02-05	spectral clipping instead of spectral flattening (the real ids behind the “MuonClip”-class idea); log-time (β₁,β₂,λ) schedules, +40% compute vs tuned AdamW and benchmarked against Muon	LLM / 45M–2.6B	yes / n/a	SKIP for this arm (a direction change or a horizon-aware schedule — either would confound)
9	2609.04577 2026-09-04 · 2608.28557 2026-08-28	preferred LR schedule reverses across the overtraining axis; optimizer survey incl. schedule-free	51M–253M, OT 1×–256× / survey	—	READ-ONLY (the standing caution against a 2k-step schedule verdict)

2. Baseline correction (the line this file was asked to test)

2026-10-16-new-tech-scan.md:54 — “schedule-free Muon (pre-July only) — no real id found” — is correct as a statement about the 2026-07..10 window and wrong as a statement about the field: the two ids are 2605.22432 and 2605.23061, both 2026-05-21, both resolved above. :91 (row 27, AMUSE — SKIP (already answered)) is the consequential error, because 2026-10-07-technology-super-research.md:24,152 assessed ScheduleFree+ 2605.19095 — an AdamW/LLM paper that never names Muon [A]. A schedule-free AdamW result is not evidence about a schedule-free Muon one; :151 repeats the conflated reason. This is a verdict change of the same kind the scan files for other rows.

3. Which part of the SF theory transfers, and which does not (STEP 3)

Transfers [A, 2605.23061 §3.1]. The SF-Spectral theorem is already a theorem about a non-gradient update map: the polar factor is the linear minimization oracle over the spectral-norm ball, so it pairs with the gradient through the nuclear norm (⟨G, polar(M)⟩ large ⟺ ‖G‖_* large), and the proof never needs the update to equal −∇f. It needs only a descent inequality at Y_t, a bounded update norm (ours: ‖O‖_F ≈ √r after Normalize_d — exactly what colrow_normalize fixes), and a bounded diameter. The momentum buffer and the averaging each cost an explicit term inside the bound [A: “the third term is the price of schedule-free drift … the explicit momentum … at the cost of a tracking error”]. So: yes, iterate averaging survives a non-gradient update map, in print, for the exact-polar case.

Does not transfer — three port-specific gaps, each a gate, not a sentence:

• S1 · our update is not polar(M). optim.py:116 computes colrow_normalize(zeropower_via_ns(buf, 5)) — the Jordan quintic, 5 steps, then columns-then-rows normalization (optim.py:80-84); SF-NorMuon’s Algorithm 1 line 7 is P_t ← polar(M_t) [A]. Finite-step-NS Muon has its own analyses (2601.19156, 2609.39595, 2610.03306, 2609.33152) and SF averaging has its own (2508.06743, 2607.09167, 2411.07061) [A] — but no published theorem composes the two. [S]: the composition almost certainly keeps order Õ(T^{−1/4}) at a worse constant; that is my inference, not a citation.
• S2 · our Muon group has NO weight decay. optim.py:234-237 builds Muon(muon_params, lr, momentum, ns_steps); only the AdamW group gets weight_decay=wd [ours]. SF-NorMuon: “weight decay at the fast iterate Z_t … is necessary for long-horizon training with schedule-free methods”, and Lemma 3.1 is what lifts the bounded-diameter assumption [A]. Our arm as the code stands runs outside the published assumption set — gate it with decay at Z, or the paper’s algorithmically permitted norm-bounded variant.
• S3 · our TSCT factors are retracted onto the Stiefel manifold every step. AGENTS.md:406: “train_loop calls model.retract_tsct(cfg.retract_iters) every step”, with the per-entry max_ortho latch at 1e-3 [ours]; TSCT factors are Muon-routed (optim.py:5-8) [ours]. The per-step map is therefore R(W − η·O(M)) with R the polar retraction, and the averaged iterate x is a convex combination of retracted iterates — not on the manifold: for orthonormal U₁,U₂, (1−c)U₁+cU₂ has column norms ≤ 1 with equality only when U₁=U₂. (a) The eval forward at x feeds a non-orthonormal U,V: the effective weight U diag(s) Vᵀ stays continuous and well-defined, so x is a legal parameter point, but the orthonormality contract the latch and the factor quantization rely on does not hold there. (b) R is the closest-point projection onto a non-convex set, and the projection step in Lyapunov/Fejér arguments needs convexity. [S] — no published theorem covers SF averaging composed with a retraction; I expect the empirical river mechanism (evaluate where oscillation is small, report the average) to be unaffected while the proof is unavailable for the retracted block.

The minimal sound recipe for this port (no code written):

1. Keep model params at y (forward, the retract_tsct call site and the firewall stay where they are) and add one extra buffer z over the Muon-routed params only — AMUSE §5.4 does exactly this and reports no extra memory versus AdamW [A].
2. y_t = (1−β_t) z_t + β_t x_t, β_t = 1 − (T₀/t)^ρ (1−β_init), β_init = 0.6, ρ = 0.8, T₀ = the existing warmup token count [A, 2605.22432 Eq. 5].
3. Gradient at y → the existing momentum buffer (µ=0.95) → NS(5) → ColRow (unchanged) → z ← z − η·scale·O, η constant, no decay phase [A].
4. x_t materialized only for evaluation: lerp y,z in place, eval, restore [A, §5.4] — and retract_tsct on y only, while logging x’s factor column-norm deficit (‖UᵀU−I‖_F/rank) every eval: that one number decides whether x is a model on the same manifold family as y.
5. Everything 1-D (norms, s scales, iteration embeddings, Engram tables) stays on the current AdamW (or SF-AdamW) path; gate S2 (decay or norm bound at z) before the arm claims the theorem.

Failure modes to gate before believing a BPB: a β_t bug leaves y≈z — silent, the arm becomes constant-LR Muon and looks merely mediocre; masked NaN steps shrink the effective step count and SF’s c_t is step-indexed, so the average is not the average of the run it thinks it ran; the max_ortho latch is one-way and persists in the checkpoint [ours, AGENTS.md:409], so record its step-0 state and refuse comparison if it flips; and x-only wins must be reported as x-only wins.

4. A/B design (STEP 4) — if derived

Protocol. small, pure CE (--jepa-weight 0 --dspark-weight 0), one batch size and one seq_len across all arms and seeds (window = eval_batches × batch × seq_len, AB-PROTOCOL.md:37) [ours]; fixed operating depth; 3 seeds per arm; 2000 steps; verdict metric = the BPB curve and the x↔y gap, not the endpoint (a horizon-dependent mechanism judged at one endpoint is the mistake 2609.04577 names) [A].

Arms (3): A0 = WSD control (live recipe, lr.py:20-48); A1 = constant LR, no averaging (same warmup, no decay) — the cheapest null, and the one AMUSE’s Figure 6 says loses badly [A: “constant learning rate Muon benefits significantly from both EWA and learning rate decay”]; A2 = constant LR + SF averaging per §3, returning x. Without A1 the arm confounds “dropped the schedule” with “added the average” — one-factor discipline, AB-PROTOCOL.md:389 [ours].

Void-check counters (read before any verdict): (a) byte count identical on every eval line of all nine runs — otherwise the table is void regardless of the numbers; (b) fused kda=<f>/<b> with b>0 in every arm (the control precondition) — kda=<f>/0 is a network whose attention is frozen at init; (c) muon_skipped= / finalize present and stable, proving the Muon path ran; (d) the NaN-firewall skipped-step count = 0 in every run, for the SF reason above; (e) one new counter is required — steps at y, x materializations, reverts — because a schedule-free arm whose x was never materialized is the 8fa5d4c failure class, and only a counter proves the arm ran. No code is edited here; the design lists it.

Pre-registered prediction [S]: A2 ≲ A1 ≲ A0 on the held-out BPB curve, with the largest gap at the end of the run. I predict the 2000-step endpoint ties (published SF gains are stated at ≥1000 tokens/param [A, 2605.19095]; 2k steps × batch × 512 bytes is orders below that), and that a tie will be misread as a refutation. Decision rule: A2 wins iff it beats A1 by more than the pooled A0/A1 seed spread on any window; a tie at 2k escalates to one 10×-step arm before deletion, which is the only reading that separates “SF does not work here” from “we measured the wrong horizon”.

5. Rejected with one line each (so nobody re-screens them)

• 2609.11655 Musec, 2605.26459 MuCon — spectral clipping variants; a direction change, not schedule elimination. Real ids, so the scan’s “MuonClip — no real id” should read “the Kimi-report name has no id; the peer-reviewed equivalents are these two”.
• 2602.13498 TrasMuon — adaptive magnitude reintroduces a schedule-shaped knob; keep only its warmup-free evidence line. 2606.08783 OptMuon — closed-loop magnitude, theory but no verified LM scale in the abstract.
• 2609.34681 SOLAR — learns an LR policy (base schedule + PPO controller); the opposite of removing schedule state. 2602.05298 ADANA — log-time (β,λ) schedules, still horizon-aware, AdamW-shaped. 2511.07767 Adaptive Polyak steps — SF-SGD/SF-Adam with a loss-value oracle; no matrix geometry.
• 2605.08980 Muon does not converge on convex Lipschitz — a caution about which proof class to trust, not a recipe; error feedback fixes it and degrades Muon in practice [A]. 2609.39595, 2610.03306, 2601.19156, 2609.33152, 2605.13079, 2606.08388 — Muon convergence / spectral dynamics under smoothness: the right citations for S1’s gap, none of them SF. 2508.06743, 2607.09167, 2411.07061, 2502.02431 — SF theory for GD/SGD: proof ingredients, not an SF-Muon result. 2609.30957, 2608.28557 — context only.
• MuonClip, LoMu, PSGD Muon, PolarGrad — rechecked 2026-10-16: MuonClip and LoMu still have no arXiv id (0 hits, all: queries); PSGD AND Muon → 0 hits; PolarGrad DOES have an id — 2505.21799, 2025-05-27, Lau/Long/Su — on “the necessity of learning rate warmup for Adam”, and including Muon scaled by the nuclear norm [A]. The scan is wrong for PolarGrad, right for the other three.

6. Sources (ids verified with get_abstract on 2026-10-16)

SF×Muon/spectral: 2605.22432 · 2605.23061 · 2605.19095 · 2507.09846 · 2602.13498 · 2606.08783 · 2609.34681 · 2609.11655 · 2605.26459 · 2602.05298 · 2505.21799. Theory: 2607.09167 · 2508.06743 · 2411.07061 · 2502.02431 · 2511.07767 · 2605.08980 · 2610.03306 · 2609.39595 · 2601.19156 · 2609.33152 · 2605.13079 · 2606.08388 · 2609.04577 · 2608.28557 · 2609.30957. Repo: port/src/dormouse/train/optim.py:80-84,100-119,121-184,187-205,234-237 · port/src/dormouse/train/lr.py:20-48 · docs/protocols/AB-PROTOCOL.md:31-51,78-85,208-233,377-398 · docs/research/2026-10-16-new-tech-scan.md:54,91,151 · docs/research/2026-10-07-technology-super-research.md:24,152 · AGENTS.md:406-412,1172.

7. Open questions

1. Does retract_tsct on y keep x’s factors inside the observed NS-3 retraction floor (≈6e-5 per entry, AGENTS.md:409)? Only our own measurement answers it — and it decides whether x needs its own retraction.
2. Is decay-at-Z compatible with the Engram/tables exemption and the µ=0.95 default? SF-NorMuon calls it necessary; nobody has tested it on a retracted TSCT block. And at what token count does an SF-Muon gain exceed our seed spread? Published SF statements are relative to horizons we cannot reach in 2k steps — honest answer today: “unknown”.