4-bit ACTIVATION quantization for training from scratch — the 2025-26 literature, ranked for the port

Date: 2026-10-18. Read-only: this file only; no config or code touched, nothing committed (the docs-site/tools/manifest.mjs line is owed, §8). Question (owner): what does the 2025-26 literature say about making 4-bit activation quantization better for training from scratch (QAT/STE) — not PTQ of a frozen model? Survey and rank: group scales, rotations, outlier migration, learnable clipping, hybrid precision, stochastic rounding, MoE-specific, and anything after 2026-06. Method: ~45 arXiv MCP calls on 2026-10-18 — search_papers (18 queries), search_paper_text (15), get_abstract (33 IDs, each resolved individually), download_paper (4 full texts). No 429 fallback was needed: every ID below resolved through the MCP (get_abstract / download_paper); the one rate limit hit was Semantic Scholar’s citation_graph, which says of itself “this is NOT an empty graph” — so no citation-following was possible (§8). Marks: [A] = authors’ abstract · [T] = full text read this session · [FS] = trained from scratch · [PTQ] = post-training/inference · [ours] = our file:line · [S1/S2] = SPECULATION (S1 = sourced mechanism, transfer unmeasured; S2 = no source at all).

0. ID corrections (two in the brief, one in our code)

1. 2504.12285 is NOT BitNet a4.8 — it is BitNet b1.58 2B4T Technical Report (2025-04-16), 8-bit per-token absmax activations, no 4-bit activations, no groups [T]. BitNet a4.8 = 2411.04965 (2024-11-07) [T]. Our repo cites the wrong ID for a4.8 in three places: port/src/dormouse/model/act_quant.py:9-12, port/src/dormouse/config.py:888, and the A/B driver’s docstring. None of the a4.8 mechanisms is in 2504.12285.
2. 2002.00356 is NOT LSQ — it is a condensed-matter paper on manganite magnetoresistance. LSQ = 1902.08153 (2019-02-21) [A].
3. Both the wrong IDs and the “a4.8 group scales” phrase come from our own transcription of the burn tree (act_quant.rs:70-104); §3.1 shows the papers have no group scales at all.

1. TL;DR

1. [T] a4.8’s recipe is a layer-split, not a group-scale recipe. 4-bit for the inputs to the attention and FFN layers; the intermediate states (before FFN down-projection and attention output-projection) are sparsified (Q-Sparse, TopK 50%) + 8-bit — the hybrid exists and is the paper’s headline. “group” and “per-token” do not occur in the paper at all; the INT8 formula is γ = max(|X|); full-INT4 uses an absmean scale β = 2·mean(|X|) for the FFN-down inputs [T 2411.04965 §3.1-3.2].
2. [T] BitNet v2 is the from-scratch validation of the rotation axis: online Hadamard (activation side only) + per-token absmax for 8-bit / absmean for 4-bit; at W1.58A4 with no rotation the 1.3B and 3B runs diverge; with rotation, a8 matches b1.58 and a4 degrades “minimally” — 1.3B/3B/7B trained from scratch [T 2504.18415 §3.1-3.3].
3. [ours] Our v2_int4 arm diverges from the paper twice: the port uses per-token absmax for 4-bit where the paper uses absmean (hadamard.py:186-220 says so in its own docstring), and the config refuses v2_ at d_model=768* (not 2^k, config.py:909-922) — small is 768, so the v2 arm cannot run on the preset our A/B uses.
4. [A] The 2026 counter-trend: Hadamard is no longer the automatic answer. Averis (2603.10444) beats “NVIDIA’s Hadamard-based outlier-smoothing method” in W4A4G4 FP4 training (Qwen3 0.6B/100B tokens, 7B-A1.5B MoE/50B tokens) by subtracting the coherent mean before quantizing; UE5M3 (2609.02846) omits RHT entirely and lowers an 8B/190B-token loss with wider (E5M3) block scales.
5. [A] Stochastic rounding is a gradients/optimizer-states technique, never a forward-activation one in everything surveyed; and 2605.09825 (Llama-3.1-8B pretraining, native FP4) reports SR fails to stabilize once Wgrad is quantized, while deterministic Hadamard restores stability.
6. [A] Fine-grained scaling for training is block 16-32, not 128: NVFP4 block-16/E4M3 was measured optimal over block sizes and formats (2505.19115); MXFP4 is block-32/E8M0; 2609.02846 block-16/E5M3; 2607.24953 2D blocks. act_group=128 has no source in the activation-training literature — it is a port convention, not an a4.8 recipe.
7. [ours] Our A/B cannot resolve any of this: control spread 0.0124 BPB, all four arms inside it (deltas ≤ 0.0044); the literature’s effects are quoted at 1.3B-8B. The next A/B should pick the axis with the cheapest faithful arm, not the axis with the biggest paper number.

2. The table — ranked by fit to “4-bit activations, training from scratch”

#	candidate (id, date)	mechanism, one line	measured on (scale, FS/PTQ)	claimed gain (source’s own)	our effort (seam, LOC)	class
T1	BitNet a4.8 2411.04965 2024-11-07	4-bit inputs to attention+FFN; intermediate states sparse (TopK 50%)+8-bit; absmean 2·mean|X| for FFN-down under full INT4; 2-stage 8→4	700M-1.3B-3B-7B FS	parity with b1.58 at equal training cost; 55% params activated; 3-bit KV	sparse+8 half absent: new seam in act_quant.py/experts.py, ~50-150 LOC + gold test	[T]
T2	BitNet v2 2504.18415 2025-04-25	online Hadamard (activation-only) → per-token int quant; absmax(8)/absmean(4)	1.3B/3B/7B FS	a8 == b1.58; a4 minimal degradation; A4 without rotation diverges	wired (hadamard.py) but refused at d=768; absmean missing; arm add ~5 LOC, preset/baseline extra	[T]
T3	Averis 2603.10444 2026-03-11	isolate coherent mean before FP4 quant (reductions + subtraction only)	Qwen3 0.6B dense/100B tok + 7B-A1.5B MoE/50B tok, W4A4G4 FS	loss gap 1.19%/0.81% vs 2.05%/1.10% for Hadamard-based smoothing	pre-step in act_quant.quantize_act or a new format, ~25-35 LOC + test	[A]
T4	UE5M3 block scaling 2609.02846 2026-09-02	E2M1 payload + unsigned E5M3 block scales (block-16); SR on backward grads only; no RHT	Nemotron-H 8B, ~190B tok FS	lower final loss than TE nv recipe; +21.2% tput when RHT+BF16-final removed	new format family in mxfp.py-style module, ~100-200 LOC	[A]
T5	Transposition-invariant 2D blocks 2607.24953 2026-07	2D-block FP4 with transposition-invariant scales + truncation-free scaling + SR; MXFP8 for Q/K	dense ≤7B + 30B MoE, ≤100B tok FS	stable end-to-end FP4 training	new scale geometry, ~150-250 LOC	[A]
T6	MXFP4 on native HW 2605.09825 2026-05-11	controlled Fprop/Dgrad/Wgrad study; interventions: SR, RHT, deterministic Hadamard	Llama 3.1-8B, C4 pretraining FS	Wgrad is the degradation driver; SR+RHT fail; deterministic Hadamard restores stability	read as evidence (picks the rotation variant)	[A]
T7	Full-Stack FP4 2607.04422 2026-07-05	NVFP4 projections + LoRA-SVD subspace in BF16 + mixed-precision attention	3B, 64B tok FS	0.838% loss gap vs BF16; 2.5-2.83x Root speedup	reference for attention split only	[A]
T8	Format-Aware Fusion 2610.00053 2026-09-04	fused FP4 producers; MXFP4 + row-gradient SR + Hadamard weight-grad preconditioning	8B, 160B tok FS	2.11% above bf16 loss endpoint at 37.2K tok/s/GPU	systems-side, not our lane	[A]
T9	UFP4 2606.20381 2026-06-18	E2M1 “shrinkage bias” vs uniform E1M2/INT4; RHT on all 3 GEMMs, SR only on dY	Dense 1.5B, MoE 7.9B/124B FS	lower BF16-relative degradation than E2M1 baselines	format-choice evidence (we already have int4)	[A]
T10	AdaHOP 2604.02525 2026-04-02	outlier-pattern-aware Hadamard + outlier extraction to a high-precision path	FS MXFP4, LLM training	BF16-level quality at MXFP4; 3.6x memory, 1.46x speedup	too structural for now	[A]
T11	FP4 All the Way 2505.19115 2025-05-25	FQT; NVFP4 block-16/E4M3 optimal; SR backward+update, RTN forward	7B, ≤200B tok FS	BF16-comparable downstream; √3-noise threshold	block-16 evidence for any grouping A/B	[A]
T12	QuEST 2502.05003 2025-02-07	Hadamard normalization + MSE-optimal fitting + trust gradient estimator	Llama-type, W4A4 and W1A1 FS	optimal at 4-bit, stable to 1-bit	gradient-estimator alternative to STE, ~50+ LOC	[A]
T13	HALO 2501.02625 2025-01-05	Hadamard rotations in fwd+bwd, all matmuls low precision	LLaMA fine-tune, 8-bit	near-FP-equivalent, 1.41x speedup	fine-tune lane, not pretrain	[A]
T14	QuaRot 2404.00456 2024-03-30	randomized Hadamard, 4-bit weights/acts/KV	LLaMA2-70B PTQ	≤0.47 WikiText-2 PPL loss, 99% zero-shot	read-only (PTQ)	[A]
T15	SpinQuant 2405.16406 2024-05-23	learned rotation matrices	LLaMA-2 7B/13B PTQ	gap to FP 2.9 pts (W4A4KV4)	read-only (PTQ)	[A]
T16	SmoothQuant 2211.10438 2022-11-18	offline outlier migration act→weight, W8A8	OPT/BLOOM/Llama etc. PTQ	1.56x speedup, 2x memory, negligible loss	read-only (§3.3)	[A]
T17	LSQ 1902.08153 / PACT 1805.06085	learnable step size / learnable clip α	CNNs (ImageNet), 2-4 bit	LSQ: SOTA 2/3/4-bit; PACT: 4-bit W+A≈FP	not wired; see §3.4	[A]
T18	Max-Window Scale Estimation 2605.26189 2026-05-25	QAT scale estimation failure modes; DTS max over a 64-step window	OpenPangu-1B, HiF8 W8A8 QAT	amax saturation invisible in loss; 0.43% MMLU drop after fix	read as evidence for any learned-scale arm	[A]
T19	Q-PACE 2610.09183 2026-10-06	second-order sensitivity → per-layer precision allocation during QAT	LLMs ≤4B FS/QAT	beats fixed mixed-precision recipes at lower memory	allocation policy, ~100+ LOC	[A]
T20	Sinks/rescaling 2601.22966 2026-01-30	attention+residual sinks; outlier-driven rescaling is functional	LLM training	clipping outliers while keeping norm degrades training	argument against naive clipping	[A]

Secondary (verified [A], not ranked): MxMoE 2505.05799 (PTQ, linear-block sensitivity + divergent expert activation frequencies) · CodeQuant 2604.10496 (PTQ MoE, learnable rotation + clustering) · JustQuant 2609.33601 (4-bit act PTQ/QAT-by-distillation, DiT/dLLM) · ThinQuant 2609.36120 and SpectrumQuant 2610.20303 (rotation calibration, PTQ; fixed-vs-learned) · HyQuant 2608.27875 (attention hybrid precision: sink tokens + local window kept high, inference) · MEC-Quant 2509.15514 (x-bit activation QAT, CV) · SR survey 2603.06060 · SR theory 2511.00874 · Practical FP4 MoE 2603.02731 · 2B4T 2504.12285 · b1.58 2402.17764.

3. The eight questions, answered

3.1 Group scales / fine-grained scaling — act_group=128 is REFUTED as an a4.8 recipe

[T 2411.04965]: the INT8 quantizer is γ/127·RoundClip(127/(γ+ε)·X, −128, 127), γ = max(|X|); the hybrid is which layers get 4-bit (Gaussian-like attention/FFN inputs) vs sparse+8-bit (sharp intermediate states); the full-INT4 ablation uses an absmean scale β = 2·mean(|X|) for the FFN-down inputs “as its inputs have larger outliers”; training uses a two-stage 8→4-bit recipe. Searches for group and per-token in 2411.04965 return zero passages; 2504.18415 also has zero group passages; 2504.12285 says “absmax… applied per-token” for its 8-bit activations [T]. The fine-grained-scaling evidence in training is block 16 (NVFP4/E4M3, measured optimal in 2505.19115; block-16/E5M3 in 2609.02846) or 32 (MXFP4/E8M0) or 2D blocks (2607.24953) — never 128. If the group axis is tested, test 16/32; 128 has no source.

3.2 Rotations — fixed Hadamard is validated for FS training; learned rotations are PTQ-only

Fixed: BitNet v2 (FS, 1.3B-7B; activation-only rotation suffices; no-rotation A4 diverges) [T]; QuEST (FS, W4A4→W1A1) [A]; HALO (fwd+bwd) [A]; AdaHOP (FS MXFP4) [A]; and inside the FP4 pretraining recipes 2607.04422 / 2610.00053 / 2606.20381 [A]. Deterministic > randomized: 2605.09825 found deterministic Hadamard restores stability where RHT and SR fail [A]. Learned: SpinQuant, ThinQuant, SpectrumQuant, CodeQuant are all PTQ calibration [A] — no from-scratch training validation of learned rotations was found. Counter-evidence to Hadamard-as-default: Averis beats a Hadamard baseline in W4A4G4 training [A]; UE5M3 omits RHT [A].

3.3 Outlier migration (SmoothQuant class) in QAT-from-scratch — no source found

SmoothQuant is explicitly training-free PTQ with an offline per-channel migration [A]; the migration presumes frozen weights. Every 2025-26 successor found is PTQ-side (DuQuant++-class, CodeQuant, SpectrumQuant, ThinQuant). The training-side analogues of outlier migration are mean-split (Averis), Hadamard/RHT, and outlier extraction (AdaHOP). [S2] I found no paper applying SmoothQuant-style migration in from-scratch training, and I assume the per-channel scale is not invariant under a moving weight.

3.4 Learnable clipping / step (LSQ, PACT) — canonical, but 2025-26 LLM training did not pick it up

LSQ 1902.08153 (learnable step size; 2-4 bit, ImageNet) and PACT 1805.06085 (learnable clip α) are verified [A]. 2025-26 successors found: Max-Window 2605.26189 (scale estimation, DTS, amax saturation — a failure mode of delayed scale estimates, invisible in training loss) [A]; BATQuant 2603.16590 (learnable block-wise, PTQ) [A]; JAB 2609.34866 (STE with learnable scales, inside a PTQ fitter) [A]. No 2025-26 result found where LSQ/PACT-style learnable clipping beats fixed absmax/absmean scales in from-scratch LLM training — the field moved the difficulty into rotations/mean-split and format block scales instead. [S1] On our STE graph a learnable scale is a cheap arm (a parameter per block), but it fights max_value()’s “block max lands on the format max” invariant and needs its own gold test.

3.5 Hybrid precision by layer/channel — the split that matters is which layers, not attention-vs-FFN bits

a4.8: 4-bit attention/FFN inputs, 8-bit intermediate states (FS) [T]. BitNet v2: 4-bit all linears after Hadamard, BOS KV heads retained at 8-bit, 3-bit KV otherwise (FS) [T]. 2607.24953: MXFP8 for Q/K, FP4 elsewhere (FS) [A]. 2607.04422: softmax-sensitive ops in BF16 (FS) [A]. Q-PACE: learned per-layer allocation (FS/QAT, ≤4B) [A]. PTQ-side: MxMoE (linear blocks vary in sensitivity), 2609.34866 (which matrix matters more than any sensitivity estimate), HyQuant (sink tokens + local window high) [A]. Our attention >= 8 rule is consistent with a4.8 and conservative vs v2/2607.24953 — but in the port it is dormant: attn_path has no call site (grep 2026-10-18), so no quantized seam currently sits inside attention.

3.6 Stochastic rounding for activations — not a forward-pass technique in any source found

SR appears as: backward/update rounding in FQT (2505.19115: SR backward+update, RTN forward) [A]; “selective stochastic rounding to backward gradients” (2609.02846) [A]; row-gradient SR (2610.00053) [A]; “SR only on dY” (2606.20381) [A]; theory vs batch size (2511.00874) [A]; survey (2603.06060) [A]. And the negative: SR fails to stabilize Wgrad in native MXFP4 pretraining (2605.09825) [A]. No source found applies SR to forward activation quantization. [S2] I assume the reason is that SR needs a high-precision master to round from, which a forward activation does not have.

3.7 MoE-specific activation quantization — block-level, not expert-input-level

2603.02731 (671B MoE FS): MXFP4 for activations and expert-parallel communication, FP8 for the core MoE compute; −14.8% peak activation memory (11.8 GB), +12.5% throughput, “without degrading convergence” [A]. Averis: 7B-A1.5B MoE, W4A4G4 FS [A]. 2607.24953: 30B MoE FS [A]. UFP4: MoE 7.9B/124B [A]. PTQ-side MoE: MxMoE (divergent expert activation frequencies create heterogeneous compute) and CodeQuant [A]. No paper found that quantizes expert-input distributions differently per expert during training — every training-side MoE paper quantizes the block uniformly. [S2] Per-expert scale rules are an open, untested axis.

3.8 Newer than 2026-06, directly on 4-bit activations in pretraining

2607.04422 (2026-07-05), 2607.24953 (2026-07), 2609.02846 (2026-09-02), 2610.00053 (2026-09-04), 2609.33601 (2026-09-27, PTQ/QAT distillation), 2610.09183 (2026-10-06). The direction of travel: block-scale FP4 (E4M3/E5M3) + outlier handling by Hadamard or mean-split + SR restricted to gradients + attention sub-paths held at higher precision. Nothing newer than 2026-06 changes the ranking of §5.

4. What our A/B already says — and its power

~/logs/ab_act_quant/*.jsonl, final eval BPB per seed (81920 B window, 3 seeds, 2000 steps, b8·s512, act_group=0) [ours, re-read 2026-10-18]:

arm	s1	s2	s3	mean	spread	Δ vs control
none	2.2011	2.1973	2.2097	2.2027	0.0124	—
int8	2.2055	2.1915	2.2097	2.2022	0.0182	−0.0005
int4	2.2125	2.2054	2.2035	2.2071	0.0090	+0.0044
fp4	2.2141	2.2036	2.1950	2.2042	0.0191	+0.0015

Every arm is inside the control’s own spread — a tie, and the pre-registered rule deletes on a tie. The honest reading is not “activation quantization is free”; it is “this instrument cannot see the effect”. The literature’s effects are 0.8-2% loss gaps at 1.3B-8B; at 9.48M params the whole quantization is a perturbation below the seed noise. Any next A/B at this scale is a mechanism-ran check, not a quality verdict; the quality verdict needs a bigger preset (b1, d=2048).

5. The ranked shortlist for the next A/B (pre-registered here, before any run)

Verdicts on the brief’s two natural axes, from sources:

• act_group=128 — REFUTED. No occurrence of “group” in 2411.04965 or 2504.18415; 2B4T is per-token 8-bit absmax; the training literature’s granularity is block 16/32. The axis is still runnable (the flag exists), but it would be testing our convention, not a recipe — and 128 is the coarsest, least-sourced setting.
• v2 Hadamard — CONFIRMED as the best-sourced training axis, with a port blocker. 2504.18415 trains from scratch at 1.3B-7B and its A4 arm diverges without the rotation. Blocker: the port refuses v2_* at d_model=768 (config.py:909-922); small is 768, medium is 1024, b1 is 2048.

S	arm (exact A/B)	why this first	LOC / seam	cost & gate
S1	4-bit scale rule: absmax → absmean. Arm int4_absmean (and fp4 stays e2m1): scale β/7 with β = c·mean|X|, c=1 (v2) and c=2 (a4.8’s FFN-down choice); control = today’s int4. 3 seeds, same window.	Cheapest new axis (both BitNet papers use absmean at 4-bit; our absmax is the divergence)	~20-30 LOC: act_quant.quantize_act (:276-320) scale branch + a config key (config.py:388-394) + gold test in tests/test_act_quant.py	one extra arm × 3 seeds ≈ +12 min; gate = the arm’s act_quant_arms counter > 0 (ADR-0019) and the eval window prints 81920 B
S2	v2_int4 / v2_int8 arms in the driver, on a power-of-2 width: add the arms (ab_act_quant.py:110), run on medium (d=1024) or a new small2k preset; re-baseline the control there.	The only from-scratch 4-bit-activation recipe with a published divergence ablation; the quantizer is already wired and tested	~5 LOC driver + a preset/baseline decision (config-only); the FWHT path is hadamard.v2_quantize	re-baseline at a different width (window must print identical bytes across arms); no code risk
S3	Averis mean-split pre-step (fp4m/int4m): subtract the per-token mean, quantize, add it back.	The 2026 paper that beat Hadamard in W4A4G4 training at 0.6B-7B; if any smoothing helps at 9.48M, this is the cheapest challenger to test against S1/S2	~25-35 LOC in act_quant.quantize_act (or a wrapper) + a gold test	same 3-seed protocol; it composes with S1’s scale rule (mean-split + absmean)

Explicitly not on the list: SR for forward activations (no source, §3.6); SmoothQuant-style migration in QAT (no source, §3.3); learnable clipping (no from-scratch LLM source, §3.4); per-expert scale rules (no source, §3.7); attention below 8 bits (papers only go there with rotation/sink retention — revisit after S2).

6. VERIFIED (sourced)

Mechanisms read from full texts [T]: a4.8’s layer split + sparse/8-bit hybrid + γ=max(|X|) + β=2·mean(|X|) + two-stage recipe + 55% activation + BOS-at-4-bit (2411.04965); BitNet v2’s per-token absmax(8)/absmean(4), activation-only Hadamard, no-rotation divergence at 1.3B/3B, FS at 1.3B/3B/7B, PTQ comparison vs QuaRot/SpinQuant (2504.18415); 2B4T’s per-token 8-bit absmax (2504.12285); b1.58’s absmean weights + 8-bit activations (2402.17764). Abstracts read and IDs individually resolved [A]: QuaRot, SpinQuant, SmoothQuant, LSQ, PACT, Averis, UE5M3, 2607.24953, 2605.09825, 2607.04422, 2610.00053, UFP4, AdaHOP, FP4 All the Way, QuEST, HALO, Max-Window, Q-PACE, 2601.22966, MxMoE, CodeQuant, JustQuant, ThinQuant, SpectrumQuant, HyQuant, MEC-Quant, SR survey, SR theory, 2603.02731. Ours [file:line]: the four arm means and spreads (§4, re-read from the JSONL); the v2 2^k refusal (config.py:909-922); attn_path has no call site (grep); small d=768 / medium d=1024 / b1 d=2048; the wrong a4.8 citation (act_quant.py:9-12, config.py:888). Tooling: no arXiv MCP rate limit; the only rate_limited was Semantic Scholar’s citation_graph (2504.18415), which is not an ID verification.

7. SPECULATION (my inference)

• [S1] absmean-4bit will transfer to our activations: the mechanism is sourced at 1.3B-7B (2504.18415, 2411.04965), not measured at 9.48M; at our scale it may tie like everything else.
• [S1] Averis’s mean-split will beat Hadamard at our scale: sourced at 0.6B/100B and 7B-MoE/50B [A], not at ≤10M params, and its baseline is NVIDIA’s smoothing, not our FWHT arm.
• [S2] No source applies SmoothQuant-style migration or SR to forward activations in from-scratch training; I assume the reasons are moving weights (migration) and the missing high-precision master (SR), but I found no paper stating either.
• [S2] Per-expert input-scale rules in MoE training are untested — no paper found.
• [S2] The act_group=128 in our burn transcription came from somewhere I could not find (not the four BitNet papers’ text); it may be an implementation convention. I cannot answer where it came from.

8. Could not verify / open questions

1. How BitNet v2 handles its own non-2^k widths (its 3B-class dims are 3200 in the b1.58 family): the retrieved text has no pad and no 3200 passage — [S2] padding or a composite construction, unverified. Our port refuses padding on principle (hadamard.py:112-117), so this matters for S2.
2. The authors’ code for a4.8 (microsoft/BitNet) was not read — a group-scale option could exist there even though the paper has none. The paper text is the evidence quoted here.
3. No citation-graph follow-ups (Semantic Scholar rate limit) — the sweep is search-based only.
4. medium’s param count was not measured here (README lists only small 9,476,993 and b1 1,066,354,113); read it with dormouse info before budgeting S2.
5. Owed per §1.8: a docs-site/tools/manifest.mjs line for this document (one-file mandate, no commit).

9. Sources

Full texts (arXiv MCP, 2026-10-18): 2411.04965 (a4.8) · 2504.18415 (v2) · 2504.12285 (2B4T) · 2402.17764 (b1.58). Abstracts (all resolved individually): 2404.00456 · 2405.16406 · 2211.10438 · 1902.08153 · 1805.06085 · 2502.05003 · 2501.02625 · 2505.19115 · 2511.00874 · 2603.06060 · 2603.10444 · 2603.02731 · 2604.02525 · 2604.10496 · 2605.09825 · 2605.26189 · 2606.20381 · 2607.04422 · 2607.24953 · 2609.02846 · 2609.33601 · 2609.36120 · 2610.00053 · 2610.09183 · 2610.20303 · 2608.27875 · 2601.22966 · 2509.15514 · 2505.05799. Repo: port/src/dormouse/model/act_quant.py (:108, :276-339) · hadamard.py (:129-220) · config.py (:117, :388-394, :884-937) · experts.py (:271-274, :398-400) · tsct.py (:293-305) · port/scripts/ab_act_quant.py (:110) · ~/logs/ab_act_quant/*.jsonl (2026-10-18) · docs/research/2026-10-16-budget-savers.md, docs/reviews/littlebit-design-2026-10-15.md.