demo

The small polygon — 9,476,993 parameters — running in your browser as a plain-JavaScript inference engine. No server, no WebGPU, no WASM: the same .dormouse artifact dormouse serve --export loads.

the engine

bytes are the vocabulary: There is no tokenizer: the prompt is UTF-8 bytes and every position is one of 256 symbols. Type anything — Russian, code, binary — and it is already in the model's vocabulary.
KDA linear attention: The gated delta rule, one byte at a time, with the recurrent state carried across bytes: n_heads × head_dim² = 49,152 fp32 values per iteration slot (196 KB), independent of how long the context gets. There is no KV cache to grow.
the weight-shared loop: One block, four iterations: a controller gates the KDA arm, the Engram memory and the experts per iteration, and the readout is the average over iterations. The parameters do not multiply with depth.
TSCT low-rank experts: Three routed experts plus an always-on shared one, each a U·diag(s)·Vᵀ factorization. In the artifact the factors are 2-bit ternary codes (four per byte) with their scale in the header; the engine unpacks them to fp32 at load.
Engram memory: 25,000 rows per order over byte n-grams of orders [2, 3, 4] — FNV-1a hashed, 32 wide, gated into the memory branch behind the hard floor min(w_mem, lam_max). 33% of the model's parameters, and a lookup, not a matmul.

Verified against the Python reference by tests/test_demo_engine.py: the engine's logits agree with decode._step_logits to a max |Δlogit| of 1.058e-6 on small (tolerance 1e-4), with an identical greedy byte at every prefix.

run it

dormouse demomodel.dormouse

loading the engine…

prompt — bytes, not tokens

bytestemperature

temperature 0 is greedy — the demo's own rule; the Python decoder refuses it rather than approximating it.

output—

what this is not

the 9.5M small polygonan A/B instrument
a chatbotno SFT, no chat template
the top-k dropless MoE pathmoe_topk = 0, the dense blend
a ternary GPU kernelcodes unpack to fp32 at load
bf16 computethe engine is the fp32 arm
a fast pathone thread, ~30M MAC/byte

This is the model the project's A/B protocol is run on, not a release. It predicts the next byte; it has never been instruction-tuned, and nothing here is a claim about language quality. What the page does demonstrate is the architecture: the arithmetic the Python trainer computes is the arithmetic this engine computes — the gate above is that statement, and it is checked on every test run. The artifact carries the weights only; the loss, the optimizer and the training step live in the checkpoint, which is a different file for a different job.

The weights are a .dormouse artifact written by dormouse export small --ckpt-name … --engram-dtype fp16; it is 16.3 MB (15.51 MiB), so it is generated rather than committed. Status has the measured numbers, including the retractions.