Technology battery — 2026-10-20/21 (owner directive: "протестить все и включить все технологии")

One card, sequential arms, medium (141.9M), 2000 steps, 3 seeds, eval window 81 920 B for every seq-512 arm (MSA arms: 2 x 2048 x 20 = 81 920 B, the same number). Data: the 230 GB mixture (configs/mix_big.toml) on the filtered corpus_bin_filtered.bin stream; eval eval_mix/eval_mix.bin. Arm flags are the only difference from the control: ns_kind=triton, opt_graph=true, moe_kernel=triton, moe_capacity=0 (dropless), engram_ram=true, bf16=true, tf32=true, max_iter=3. Verdict rule (AGENTS §1.2): a win must beat the control’s own seed spread.

Results

arm	BPB mean	spread	Δ vs control	verdict
control (seq 512)	3.0750	0.0084	—	baseline
attnres (use_attnres=true)	3.0118	0.0129	−0.0632	WIN (7.5x the control spread)
dspark (dspark_weight=0.1)	3.0733	0.0096	−0.0017	TIE — off in pretraining
msa off @ seq 2048	3.0744	0.0099	−0.0006	(the 2048 control)
msa on @ seq 2048	3.0702	0.0045	−0.0042 vs its own control	TIE at 2k — kept for long-context (it costs nothing measurable and halves the seed spread)
rank 32 (E=256)	3.1024	0.0083	+0.0274	LOSS — rank 64 stays
rank 16 (E=256)	3.2106	0.0212	+0.1356	LOSS (4x the spread)
rank 8 (E=256)	—	—	—	CRASH: CUDA kernel error at launch; not chased (the curve above already decides)
loom @ depth 4	—	—	—	not run: the chain was cut for the training launch; the 9.2M A/B already measured a tie

Small-polygon ByteFlow (recorded 2026-10-20, 2M params, b4 s512, 3 seeds):

arm	BPB mean	spread
control (raw bytes)	3.0447	0.0308
channel (patches)	2.3581	0.0053

−0.686 BPB, 22x the control spread — but at +46% parameters and 1.64x slower per step, so the capacity-matched control is still owed.

What the numbers say

1. attnres is real. It was the one mechanism carried since 2026-09-30 with no A/B (“A/B or delete” debt), and on medium it wins by 0.063 BPB — the second-largest measured win after ColRow (−0.0428 on the same polygon class) and the largest of this battery. It goes into the next training recipe.
2. MSA is free at 2k. It does not pay for itself over 2048 tokens (nothing to be sparse about) and it does not hurt; its value is the long-context run, where the alternative is quadratic attention. Keep it on, report the tie.
3. Capacity lives in the expert WIDTH, not in more experts per bit. Halving rank costs 0.027 BPB, quartering it costs 0.136 — so the “compress harder to fit more experts” trade is a net loss at this scale. TSCT rank 64 stays.
4. dspark as a pretraining aux is a tie, consistent with the 2026-10-18 aux A/B. Its value, if any, is speculative decoding, not the base objective.
5. The ByteFlow medium pair did not run: with bf16=false (the ByteFlow net refuses bf16) the MoE Triton grouped GEMM needs 196 KiB of shared memory against the 101 KiB ceiling. Fix: run that pair with moe_kernel=padded on both arms. Queued — it is an architecture decision, not a knob.

Files

Logs: ~/logs/battery/ (chain.log, one .log + .jsonl per arm). Driver: /tmp/opencode/battery.sh (not in the repo: it is a one-shot). The recipe it fed is configs/mix_v2.toml (the 230 GB mixture + the 126 MB real codeparrot-clean code slice), committed with this report.

The morning model (newmodel2)

Launched after the battery with its winners: medium, seq 2048, batch 4, 12 000 steps (98 MB), use_msa=true + use_attnres=true, rank 64, dropless Triton MoE, Enram in host RAM, configs/mix_v2.toml (the 230 GB mixture plus the 126 MB real codeparrot-clean code slice). opt_graph=false: at this shape the capture refused (3 parameters without .grad at capture time) and the fallback costs a 562 MB allocation attempt at a 15 GiB working set.

step	3000	4000	5000	9000	10000	11000	12000
BPB	2.435	2.363	2.277	2.151	2.146	2.151	2.082

Window 163 840 B (20 x 4 x 2048), eval_mix.bin. Curve monotone, decay visible at the end. MSA counters live: msa_triton=13029, msa_iter_gate=17994, msa_iter_skip=4965 (27.6% of iterations skipped by the adaptive gate), msa_truncated=478 588; attnres_write=17994.

Bench on the same checkpoint (n=500 per task): ARC-Easy 0.270, HellaSwag 0.308, synth-math 0.240 — macro 0.2727, i.e. above the 60k-step mix1 checkpoint’s 0.257 on a 12k-step run, and with 2048-token context.

Honest: this is an architecture run (98 MB seen), not a chat base — mix1 (4.18 GB, BPB 1.883) remains the best pretrained checkpoint. newmodel2 proves the 2048 + MSA + attnres + code path trains cleanly and lands downstream numbers above the longer run’s.

Still owed

• ByteFlow at medium scale: the pair cannot run in fp32 with moe_capacity=0 — moe_kernel=padded is refused by dropless routing and the Triton grouped GEMM needs 196 KiB of shared memory against a 101 KiB ceiling (measured twice). Legal paths: moe_capacity=2.0 + padded (drops on both arms; measured 90k dropped assignments in a 20-step smoke) or naive (slow). The fp32 step is ~2.5-3 s, so the pair costs ~75 min/run.
• ByteFlow capacity-matched control at the small polygon (the +46% parameter confound).
• loom at depth 4 on medium (the chain was cut for the training launch; the 9.2M A/B measured a tie).
• rank 8 crashes in a CUDA kernel at launch — not chased, the rank curve already decides.