ThRetNet / Reference / v1.3
Reference releases
Portable architecture and correctness reference for the bifurcated-prompt model.
- Package
- 1.3.0 · Latest
- Runtime
- Python 3.10+
- Parameters
- 12,716,064
- Files
- 38
Archive SHA-256bf72a7bd222ff0745a178726dc9bd5a15bf9386730b3b05cf7602228428588e8
Available releases
Choose a version
Why v1.3
The prompt memory splits in two
The defining change in v1.3 is the bifurcation of the prompt memory bank into reasoning-owned and finalization-owned banks. During prompt prefill, the same tokens are processed in two synchronized branches. Each subsequent phase therefore reads a prompt representation formed in its own class space, preventing conflicting class interpretations from competing inside a shared prompt bank.
Because the prompt is no longer treated as an independent compute phase, v1.3 eliminates the prompt-phase read heads and prompt-only class-phase parameter sets. Structured butterfly projections separately reduce the weight of the memory mixer. Together, these changes reduce the complete per-layer block from 999,120 to 715,520 parameters—about 28 percent.
That reduction made room to increase the MLP width from 464 to 512 and expand the network from 10 to 13 layers while reducing the representative model from about 13.4 million to 12.7 million parameters. The residual width remains 256.
Release comparison
What changed
v1.3 changes the scope of the reference package as well as the model. The older training harness and capability builders remain available in v0.1.
| Area | v1.3 · Latest | v0.1 · Previous |
|---|---|---|
| Primary purpose | Architecture, correctness, and portability reference | Runnable training and atomic-capability reference |
| Prompt memory | Reasoning-owned and finalization-owned prompt banks; four banks total | One shared prompt bank; three banks total |
| Prompt phase | Two synchronized prompt branches; no prompt-only read heads or class table | Prompt, reasoning, and finalization are independently parameterized phases |
| Geometry | 13 layers · residual width 256 · MLP width 512 | 10 layers · residual width 256 · MLP width 464 |
| Parameters | 12,716,064 in the README example configuration | 13,415,232 in the published package configuration |
| Included material | Architecture specification, checkpoint and parity tests, portable model, isolated DGX Spark artifact | Tokenizer, UGI, CopyPrompt, AtomicOrder, training and generation tools |
Selected release
v1.3: latest reference
This release is a correctness-oriented FP32 PyTorch reference with parallel, recurrent, prefill, and step interfaces. Its DGX Spark GB10 source artifact is isolated from the portable package and is not imported or built by default. The tokenizer, UGI implementation, atomic-capability builders, and training and generation CLIs remain in v0.1 rather than this archive.
v1.3 repository snapshot
Files
- ThRetNet-Reference-v1.3
- README.md
2.8 KB · MD