# ThRetNet Quantum CPA v1.3 Architecture ## Fixed geometry | Quantity | Value | | --- | ---: | | Residual width | 256 | | Layers | 13 | | Retention heads | 8 | | Key width per head | 32 | | Value width per head | 48 | | MLP width | 512 | | Memory banks per layer | 4 | | Compute phases | reasoning, finalization | | Parameter and model-math precision | IEEE FP32 | The vocabulary size and class-table size are supplied by the checkpoint/task contract. The class-table size is at least 12. The token embedding and output classifier share one parameter tensor. ## Regions and chronological cuts A complete scored row contains prompt, reasoning, and finalization regions. The first consumed EOR ends the prompt and is processed by reasoning. The second consumed EOR changes subsequent processing to finalization. A final EOR ends generation without itself being consumed. Prompt content is evaluated in two synchronized branches: reasoning-owned and finalization-owned. These create the two prompt memory banks used by the later phases. Let `P_l`, `R_l`, and `F_l` denote the prompt, reasoning, and finalization cells at layer `l`. Chronological cut `c` contains every valid member of: ```text P_c, R_(c-1), F_(c-2) ``` There are 15 cuts for 13 layers. A cut expresses the true inter-region dependencies while leaving independent cells available for concurrent execution. It is an execution organization of architectural pieces, not a claim that every intermediate datum is itself an architectural component. ## Memory banks Every layer owns four recurrent retention banks: | Bank | Written from | Read during | | --- | --- | --- | | `P_R` | reasoning-owned prompt branch | reasoning | | `P_F` | finalization-owned prompt branch | finalization | | `R` | reasoning | reasoning and finalization | | `F` | finalization | finalization | Reasoning reads `P_R` and `R`. Finalization reads `P_F`, `R`, and `F`. There is no synthetic null bank. The finalization read of `R` sees the unchanged forward value but is detached for backward ownership. Thus finalization loss trains its own quantum read head but does not send a gradient through the already completed reasoning memory state. This is a training-semantic boundary, not an additional network component. ## Quantum read heads and memory writers A quantum read-head component is one conceptual component whose dense 256-to-256 weight is selected according to the phase and memory bank being read. Five independent weight groupings exist per layer: ```text reasoning / P_R reasoning / R finalization / P_F finalization / R finalization / F ``` These groupings are initialized from one bit-identical reference draw and then train independently. The component behavior is otherwise identical; only the selected weights change. Four independent memory-writer groupings exist per layer: prompt reasoning, prompt finalization, reasoning, and finalization. Each key writer is a seven-stage radix-4 identity-plus-delta butterfly over width 256. Each value writer applies the same global butterfly followed by 64 independent norm-preserving 4-to-6 lifts, producing 384 values arranged as 8 heads of width 48. ## Retention Each head uses decay ```text gamma_h = 1 - 2^(-(5 + h)), h in [0, 7]. ``` Active memory follows normalized RetNet retention. The reference provides both the parallel causal equation and a literal recurrent equation. Frozen-bank reads multiply the current quantum query by the retained key/value state. The established parallel path divides a read by detached absolute score mass; the recurrent path reads the normalized recurrent state directly. Headwise RMS removes that positive scale in the zero-epsilon limit, but its fixed `1e-5` epsilon leaves small finite FP32 differences between full-row parallel logits and token-step recurrent logits. Cached generation is a third observation: prefill seeds retained state through the production parallel path, then each generated token advances that cache recurrently. The release tests all three forms against the corresponding current v1.3 behavior rather than claiming false identity between them. Every visible bank read is independently headwise RMS-normalized. A phase-owned meta query and bank-derived meta keys select across the visible real banks with a softmax. The selected 384-wide value is reduced to width 256 by 64 learned 6-to-4 reductions followed by the global butterfly. ## Class-phase attention and position Class-phase attention occurs once, before layer zero. A shared classifier emits unit-L2 class coefficients. Reasoning and finalization have separate class-vector tables and distinct scalar phase directions. Their blended vector is rotated by the same continuous global position clock used throughout the model and added to the token embedding. There is no prompt-only class table. Prompt content uses the reasoning-owned and finalization-owned parameter selections of the two synchronized branches. ## Layer structure Each cell in every layer performs: 1. Layer normalization. 2. Phase-specific ingress memory shock. 3. Retention memory operation. 4. Phase-specific egress memory shock and residual addition. 5. Layer normalization. 6. Phase-specific ingress MLP shock. 7. `256 -> 512 -> 256` MLP with tanh-approximate GELU. 8. Phase-specific egress MLP shock and residual addition. A vocabulary shock is applied around class-phase attention and decoding. Shock operations are add-then-scale transformations with independent reasoning and finalization offsets and scale deltas. ## Butterflies The square butterfly uses radix 4, width 256, and stage digits: ```text 0, 1, 2, 3, 2, 1, 0 ``` Every stage is an exact fixed identity plus a trainable delta. Rectangular butterflies use deterministic orthonormal six-point DCT columns as fixed bases; only their deltas are trainable. ## Portability and optimized execution The portable source expresses these equations with ordinary PyTorch operations and contains no CUDA import or device-specific scheduler. The supplementary DGX Spark artifact specializes the same parameter names and geometry for GB10, `aarch64`, and `sm_121`. It does not redefine the architecture.