Experiment 3 · python r1 · original
SOURCE: work/lm_understand-logs-4d09ba6d6a7e891dd8063974dfe81b3d78e3494f/exp3_language/runs/python_r1
Recorded prompts and outputs are reproduced below without shortening. CLI reasoning entries are emitted summaries. Raw records preserve metadata, notices and original failed resume attempts.

RECORDED INSTRUCTION FILE
Answer the user's request.


========================================================================
TURN t001
MESSAGE SENT TO CODEX
Below is a small character-level language model: a 4-layer transformer with 6 attention heads per layer, which reads up to 128 characters and outputs scores (logits) for the next character. This is exactly how it computes (numpy):

```python
import numpy as np
VOCAB = "\n " + "abcdefghijklmnopqrstuvwxyz"   # index 0 = newline, 1 = space, 2..27 = a..z
HEADS, LAYERS = 6, 4

def layer_norm(x, w, b):                     # over the last axis, eps 1e-5
    mu = x.mean(-1, keepdims=True)
    var = ((x - mu) ** 2).mean(-1, keepdims=True)
    return (x - mu) / np.sqrt(var + 1e-5) * w + b

def next_char_logits(text, p):
    """text: up to 128 characters (only the last 128 are used); p: dict of the weight arrays.
    Returns the 28 logits for the character that comes next."""
    ids = [VOCAB.index(c) for c in text[-128:]]
    n, d = len(ids), p["tok_emb"].shape[1]
    hd = d // HEADS
    x = p["tok_emb"][ids] + p["pos_emb"][:n]                  # (n, d); positions 0..n-1 from the start of the text
    for l in range(LAYERS):                                    # layer l uses the tensors named <name>.<l>
        h = layer_norm(x, p[f"ln1_w.{l}"], p[f"ln1_b.{l}"])
        q, k, v = h @ p[f"W_q.{l}"].T, h @ p[f"W_k.{l}"].T, h @ p[f"W_v.{l}"].T
        out = np.zeros((n, d))
        for j in range(HEADS):                                 # head j uses dimensions j*hd .. (j+1)*hd - 1
            s = slice(j * hd, (j + 1) * hd)
            att = q[:, s] @ k[:, s].T / np.sqrt(hd)            # (n, n)
            att = att + np.triu(np.full((n, n), -np.inf), 1)   # causal: position i sees positions 0..i
            att = np.exp(att - att.max(-1, keepdims=True))
            att = att / att.sum(-1, keepdims=True)
            out[:, s] = att @ v[:, s]
        x = x + out @ p[f"W_o.{l}"].T
        h = layer_norm(x, p[f"ln2_w.{l}"], p[f"ln2_b.{l}"])
        x = x + np.maximum(0.0, h @ p[f"W_fc.{l}"].T) @ p[f"W_proj.{l}"].T   # ReLU MLP, hidden 4 d
    h = layer_norm(x, p["lnf_w"], p["lnf_b"])
    return h[-1] @ p["W_out"].T + p["b_out"]                   # logits of the next character (28)
```

Training: from random initialization (every matrix entry drawn from a normal distribution with standard deviation 0.02, or 0.007071 for W_o and W_proj; LayerNorm weights 1 and biases 0; b_out 0), next-character prediction with AdamW (30,517 steps of 256 random 128-character windows, 1,000,000,000 characters in total; learning-rate warm-up then cosine decay; weight decay 0.01 on the matrices).

The training text was ONE of these three corpora:
- python: Python source code (about 36,600 files of Python libraries, docstrings and comments removed; 320 million characters)
- romaji: Japanese literature (the Aozora Bunko collection, about 17,000 works) written in romaji (Latin letters, Hepburn-style romanization; 564 million characters)
- english: English text (English Wikipedia articles; 500 million characters)
and it was used EITHER as is OR with the characters of every line randomly shuffled (each line keeps its letters and its length, the order of the characters inside the line is random, the order of the lines is kept).
Normalization (the same for all three corpora, before anything else): lowercased; in each line the leading spaces (indentation) were kept, every other character that is not a letter a-z became a space, runs of spaces were collapsed to one and trailing spaces removed. So the text contains only newlines, spaces and the letters a-z (the 28 characters of VOCAB).

WHAT YOU CAN USE. The weights inside the four transformer blocks, exactly as trained (float32): for layer l = 0, 1, 2 and 3, ln1_w.l, ln1_b.l, W_q.l, W_k.l, W_v.l, W_o.l (attention) and ln2_w.l, ln2_b.l, W_fc.l, W_proj.l (MLP). The input layer (tok_emb, the 28 x d character embeddings, and pos_emb, the 128 x d position embeddings) and the output layer (lnf_w, lnf_b, W_out, b_out) are NOT available. The d = 384 dimensions of the residual stream carry no labels. Head j of layer l uses rows 64j .. 64j+63 of W_q.l, W_k.l and W_v.l and columns 64j .. 64j+63 of W_o.l.

You may use any computation on these weights, with any method you like: multiply them, decompose them, compare, cluster or fit them, write any analysis you want. Two things are NOT allowed:
1. A forward pass: do not run the network or any of its layers (LayerNorm, attention with its softmax, the MLP with its ReLU) on any input, whether real, random or constructed, in code, in your head or on paper. Treating the weight matrices as linear-algebra objects (products, eigenvalues, singular vectors, applying one weight matrix to directions taken from other weights) is fine; simulating what the network does to an input is not.
2. Search: do not look anything up (there is no internet access in any case).

HOW YOU COMPUTE. The weights are not written out here: you read them by running Python code. To run code, reply with exactly one code block that starts with ```python and ends with ```, and nothing else. We run it and reply with its printed output. Available: numpy, scipy, scikit-learn, PyTorch (CPU), pandas, sympy, networkx. Before your code runs, numpy is imported as np and W is a dict of the weight arrays by name: W['W_q.0'] is 384 x 384, W['W_fc.0'] is 1536 x 384, W['W_proj.0'] is 384 x 1536, W['ln1_w.0'] has 384 entries, and so on; matrices are (out, in), as used in the code above. There is nothing else to read (no other files, no network). Each run is a fresh process (nothing is kept between runs), may take up to 600 seconds, and its output is cut at 20,000 characters. You may run code up to 100 times. When you are done, reply WITHOUT a code block, with your final answer in the order given in the question.

QUESTION. What was this network trained on, and how does it use the context? Answer in this order:
Observations: <what you see in the shown weights and what it suggests; quote the weights you rely on>
Heads: <one line per head, L0H0 .. L0H5, L1H0 .. L1H5, L2H0 .. L2H5, L3H0 .. L3H5: what it most likely does, as one of: 'previous' (attends to the previous character), 'local' (a few characters back), 'broad' (spread far back), 'copy' (induction: attends to the character right after an earlier occurrence of the current character or context, so the network can repeat earlier text), or 'unclear'; then your evidence>
Reasoning: <which training text fits these observations, and why>
corpus: <python | romaji | english>
order: <as is | shuffled>
probabilities: python=<p> romaji=<p> english=<p>; as is=<p> shuffled=<p>
Equal probabilities are not allowed. Even if the evidence is weak, give your best judgement.
CLI OUTPUT IN EVENT ORDER
ERROR
Code Mode is unavailable because code-mode host is disabled. Code mode will fail closed; enable `features.code_mode_host` and install `codex-code-mode-host`.

AGENT_MESSAGE
```python
import numpy as np
from scipy.linalg import svdvals
from threadpoolctl import threadpool_limits
threadpool_limits(4)

def summary(a):
    a = np.asarray(a, dtype=np.float64)
    return (float(np.linalg.norm(a)), float(a.mean()), float(a.std()), float(a.min()), float(a.max()))

def spectral(a):
    s = svdvals(np.asarray(a, dtype=np.float64), check_finite=False)
    e = s*s
    return [float(s[0]), float(e[:1].sum()/e.sum()), float(e[:4].sum()/e.sum()),
            float(e[:16].sum()/e.sum()), float(e.sum()/e[0])]

print("Keys:", sorted(W))
for l in range(4):
    print("\nLAYER", l)
    for name in ["ln1_w", "ln1_b", "ln2_w", "ln2_b"]:
        print(name, "norm,mean,std,min,max:", np.round(summary(W[f"{name}.{l}"]), 5))
    for name in ["W_q", "W_k", "W_v", "W_o", "W_fc", "W_proj"]:
        a = W[f"{name}.{l}"]
        print(name, "Frobenius:", round(float(np.linalg.norm(a)), 4),
              "smax,energy1,energy4,energy16,stable_rank:", np.round(spectral(a), 4))

print("\nHEADS: raw weight products only; no inputs or forward passes")
print("QK = diag(ln1_w) Q.T K diag(ln1_w); OV = O V diag(ln1_w)")
print("Fields: QK[Frob,smax,E1,E4,E16,stable_rank,symmetry,trace/Frob]; OV[same]; QK bias norms")
for l in range(4):
    g = np.asarray(W[f"ln1_w.{l}"], dtype=np.float64)
    b = np.asarray(W[f"ln1_b.{l}"], dtype=np.float64)
    for j in range(6):
        sl = slice(64*j,64*(j+1))
        q = np.asarray(W[f"W_q.{l}"][sl],dtype=np.float64)
        k = np.asarray(W[f"W_k.{l}"][sl],dtype=np.float64)
        v = np.asarray(W[f"W_v.{l}"][sl],dtype=np.float64)
        o = np.asarray(W[f"W_o.{l}"][:,sl],dtype=np.float64)
        a = (q*g).T @ (k*g) / 8
        c = o @ (v*g)
        def stats(m):
            f = np.linalg.norm(m)
            return np.array([f]+spectral(m)+[
                np.sum(m*m.T)/(f*f), np.trace(m)/f])
        biases = [np.linalg.norm((k*g).T @ (q@b)/8),
                  np.linalg.norm((q*g).T @ (k@b)/8)]
        print(f"L{l}H{j}", "QK", np.round(stats(a),4),
              "OV", np.round(stats(c),4), "bias", np.round(biases,4))
```


PERMITTED EXPERIMENT HARNESS RECORD: python
{
  "t": 1792713193.722969,
  "s": 18.9,
  "code": "import numpy as np\nfrom scipy.linalg import svdvals\nfrom threadpoolctl import threadpool_limits\nthreadpool_limits(4)\n\ndef summary(a):\n    a = np.asarray(a, dtype=np.float64)\n    return (float(np.linalg.norm(a)), float(a.mean()), float(a.std()), float(a.min()), float(a.max()))\n\ndef spectral(a):\n    s = svdvals(np.asarray(a, dtype=np.float64), check_finite=False)\n    e = s*s\n    return [float(s[0]), float(e[:1].sum()/e.sum()), float(e[:4].sum()/e.sum()),\n            float(e[:16].sum()/e.sum()), float(e.sum()/e[0])]\n\nprint(\"Keys:\", sorted(W))\nfor l in range(4):\n    print(\"\\nLAYER\", l)\n    for name in [\"ln1_w\", \"ln1_b\", \"ln2_w\", \"ln2_b\"]:\n        print(name, \"norm,mean,std,min,max:\", np.round(summary(W[f\"{name}.{l}\"]), 5))\n    for name in [\"W_q\", \"W_k\", \"W_v\", \"W_o\", \"W_fc\", \"W_proj\"]:\n        a = W[f\"{name}.{l}\"]\n        print(name, \"Frobenius:\", round(float(np.linalg.norm(a)), 4),\n              \"smax,energy1,energy4,energy16,stable_rank:\", np.round(spectral(a), 4))\n\nprint(\"\\nHEADS: raw weight products only; no inputs or forward passes\")\nprint(\"QK = diag(ln1_w) Q.T K diag(ln1_w); OV = O V diag(ln1_w)\")\nprint(\"Fields: QK[Frob,smax,E1,E4,E16,stable_rank,symmetry,trace/Frob]; OV[same]; QK bias norms\")\nfor l in range(4):\n    g = np.asarray(W[f\"ln1_w.{l}\"], dtype=np.float64)\n    b = np.asarray(W[f\"ln1_b.{l}\"], dtype=np.float64)\n    for j in range(6):\n        sl = slice(64*j,64*(j+1))\n        q = np.asarray(W[f\"W_q.{l}\"][sl],dtype=np.float64)\n        k = np.asarray(W[f\"W_k.{l}\"][sl],dtype=np.float64)\n        v = np.asarray(W[f\"W_v.{l}\"][sl],dtype=np.float64)\n        o = np.asarray(W[f\"W_o.{l}\"][:,sl],dtype=np.float64)\n        a = (q*g).T @ (k*g) / 8\n        c = o @ (v*g)\n        def stats(m):\n            f = np.linalg.norm(m)\n            return np.array([f]+spectral(m)+[\n                np.sum(m*m.T)/(f*f), np.trace(m)/f])\n        biases = [np.linalg.norm((k*g).T @ (q@b)/8),\n                  np.linalg.norm((q*g).T @ (k@b)/8)]\n        print(f\"L{l}H{j}\", \"QK\", np.round(stats(a),4),\n              \"OV\", np.round(stats(c),4), \"bias\", np.round(biases,4))\n",
  "out": "Keys: ['W_fc.0', 'W_fc.1', 'W_fc.2', 'W_fc.3', 'W_k.0', 'W_k.1', 'W_k.2', 'W_k.3', 'W_o.0', 'W_o.1', 'W_o.2', 'W_o.3', 'W_proj.0', 'W_proj.1', 'W_proj.2', 'W_proj.3', 'W_q.0', 'W_q.1', 'W_q.2', 'W_q.3', 'W_v.0', 'W_v.1', 'W_v.2', 'W_v.3', 'ln1_b.0', 'ln1_b.1', 'ln1_b.2', 'ln1_b.3', 'ln1_w.0', 'ln1_w.1', 'ln1_w.2', 'ln1_w.3', 'ln2_b.0', 'ln2_b.1', 'ln2_b.2', 'ln2_b.3', 'ln2_w.0', 'ln2_w.1', 'ln2_w.2', 'ln2_w.3']\n\nLAYER 0\nln1_w norm,mean,std,min,max: [6.7241 0.2677 0.2147 0.0165 1.0134]\nln1_b norm,mean,std,min,max: [ 0.7172  0.0006  0.0366 -0.1215  0.1418]\nln2_w norm,mean,std,min,max: [17.2723  0.8168  0.3312  0.0468  1.6412]\nln2_b norm,mean,std,min,max: [ 1.353   0.018   0.0667 -0.5537  0.1663]\nW_q Frobenius: 22.7038 smax,energy1,energy4,energy16,stable_rank: [ 5.0385  0.0493  0.1778  0.5305 20.3046]\nW_k Frobenius: 23.2143 smax,energy1,energy4,energy16,stable_rank: [ 6.2042  0.0714  0.1958  0.5535 14.0005]\nW_v Frobenius: 18.1973 smax,energy1,energy4,energy16,stable_rank: [ 4.5533  0.0626  0.1907  0.5408 15.9722]\nW_o Frobenius: 19.5995 smax,energy1,energy4,energy16,stable_rank: [6.678  0.1161 0.2271 0.4185 8.614 ]\nW_fc Frobenius: 51.6496 smax,energy1,energy4,energy16,stable_rank: [18.1104  0.1229  0.2298  0.4395  8.1335]\nW_proj Frobenius: 45.8056 smax,energy1,energy4,energy16,stable_rank: [10.9319  0.057   0.1241  0.3034 17.5569]\n\nLAYER 1\nln1_w norm,mean,std,min,max: [9.5046 0.4754 0.0964 0.1979 1.2096]\nln1_b norm,mean,std,min,max: [ 3.0327  0.022   0.1532 -1.0505  0.9103]\nln2_w norm,mean,std,min,max: [20.0543  1.0111  0.1582  0.0139  1.4707]\nln2_b norm,mean,std,min,max: [ 2.8769  0.0275  0.1442 -0.3744  0.3652]\nW_q Frobenius: 31.6763 smax,energy1,energy4,energy16,stable_rank: [ 8.9845  0.0804  0.2024  0.4647 12.4303]\nW_k Frobenius: 28.6017 smax,energy1,energy4,energy16,stable_rank: [10.9672  0.147   0.321   0.5501  6.8013]\nW_v Frobenius: 28.4686 smax,energy1,energy4,energy16,stable_rank: [ 5.3505  0.0353  0.1262  0.3844 28.3105]\nW_o Frobenius: 26.151 smax,energy1,energy4,energy16,stable_rank: [ 4.1901  0.0257  0.0783  0.2426 38.9526]\nW_fc Frobenius: 80.2609 smax,energy1,energy4,energy16,stable_rank: [20.7957  0.0671  0.1558  0.4206 14.8956]\nW_proj Frobenius: 75.1192 smax,energy1,energy4,energy16,stable_rank: [ 7.6249  0.0103  0.0358  0.1103 97.0588]\n\nLAYER 2\nln1_w norm,mean,std,min,max: [14.4574  0.7343  0.0712  0.3445  0.9047]\nln1_b norm,mean,std,min,max: [ 2.6214  0.0244  0.1315 -0.9417  0.7326]\nln2_w norm,mean,std,min,max: [25.082   1.275   0.1126  0.0775  1.5018]\nln2_b norm,mean,std,min,max: [ 2.911   0.0188  0.1474 -0.4157  0.3635]\nW_q Frobenius: 37.4829 smax,energy1,energy4,energy16,stable_rank: [ 5.0672  0.0183  0.0621  0.2013 54.7179]\nW_k Frobenius: 36.3557 smax,energy1,energy4,energy16,stable_rank: [ 8.2935  0.052   0.1273  0.2535 19.2163]\nW_v Frobenius: 34.7465 smax,energy1,energy4,energy16,stable_rank: [ 3.89    0.0125  0.0476  0.1701 79.7848]\nW_o Frobenius: 34.2053 smax,energy1,energy4,energy16,stable_rank: [ 4.4416  0.0169  0.0534  0.1609 59.3076]\nW_fc Frobenius: 79.4541 smax,energy1,energy4,energy16,stable_rank: [14.8285  0.0348  0.0646  0.1655 28.7103]\nW_proj Frobenius: 77.4765 smax,energy1,energy4,energy16,stable_rank: [10.0946  0.017   0.0517  0.1553 58.9061]\n\nLAYER 3\nln1_w norm,mean,std,min,max: [15.1046  0.7694  0.0469  0.451   0.8778]\nln1_b norm,mean,std,min,max: [ 1.8619  0.0102  0.0945 -0.5141  0.6285]\nln2_w norm,mean,std,min,max: [24.6574  1.2552  0.0888  0.2619  1.4568]\nln2_b norm,mean,std,min,max: [ 2.4972  0.014   0.1267 -0.5112  0.3808]\nW_q Frobenius: 33.9401 smax,energy1,energy4,energy16,stable_rank: [ 4.1544  0.015   0.0528  0.169  66.7429]\nW_k Frobenius: 33.2214 smax,energy1,energy4,energy16,stable_rank: [ 4.8875  0.0216  0.0582  0.1695 46.2013]\nW_v Frobenius: 38.9389 smax,energy1,energy4,energy16,stable_rank: [ 4.6968  0.0145  0.0507  0.1776 68.7339]\nW_o Frobenius: 39.0173 smax,energy1,energy4,energy16,stable_rank: [ 9.1753  0.0553  0.1082  0.2289 18.0831]\nW_fc Frobenius: 78.1887 smax,energy1,ener"
}

========================================================================
TURN t002
MESSAGE SENT TO CODEX
OUTPUT of run 1 of at most 100:
```
Keys: ['W_fc.0', 'W_fc.1', 'W_fc.2', 'W_fc.3', 'W_k.0', 'W_k.1', 'W_k.2', 'W_k.3', 'W_o.0', 'W_o.1', 'W_o.2', 'W_o.3', 'W_proj.0', 'W_proj.1', 'W_proj.2', 'W_proj.3', 'W_q.0', 'W_q.1', 'W_q.2', 'W_q.3', 'W_v.0', 'W_v.1', 'W_v.2', 'W_v.3', 'ln1_b.0', 'ln1_b.1', 'ln1_b.2', 'ln1_b.3', 'ln1_w.0', 'ln1_w.1', 'ln1_w.2', 'ln1_w.3', 'ln2_b.0', 'ln2_b.1', 'ln2_b.2', 'ln2_b.3', 'ln2_w.0', 'ln2_w.1', 'ln2_w.2', 'ln2_w.3']

LAYER 0
ln1_w norm,mean,std,min,max: [6.7241 0.2677 0.2147 0.0165 1.0134]
ln1_b norm,mean,std,min,max: [ 0.7172  0.0006  0.0366 -0.1215  0.1418]
ln2_w norm,mean,std,min,max: [17.2723  0.8168  0.3312  0.0468  1.6412]
ln2_b norm,mean,std,min,max: [ 1.353   0.018   0.0667 -0.5537  0.1663]
W_q Frobenius: 22.7038 smax,energy1,energy4,energy16,stable_rank: [ 5.0385  0.0493  0.1778  0.5305 20.3046]
W_k Frobenius: 23.2143 smax,energy1,energy4,energy16,stable_rank: [ 6.2042  0.0714  0.1958  0.5535 14.0005]
W_v Frobenius: 18.1973 smax,energy1,energy4,energy16,stable_rank: [ 4.5533  0.0626  0.1907  0.5408 15.9722]
W_o Frobenius: 19.5995 smax,energy1,energy4,energy16,stable_rank: [6.678  0.1161 0.2271 0.4185 8.614 ]
W_fc Frobenius: 51.6496 smax,energy1,energy4,energy16,stable_rank: [18.1104  0.1229  0.2298  0.4395  8.1335]
W_proj Frobenius: 45.8056 smax,energy1,energy4,energy16,stable_rank: [10.9319  0.057   0.1241  0.3034 17.5569]

LAYER 1
ln1_w norm,mean,std,min,max: [9.5046 0.4754 0.0964 0.1979 1.2096]
ln1_b norm,mean,std,min,max: [ 3.0327  0.022   0.1532 -1.0505  0.9103]
ln2_w norm,mean,std,min,max: [20.0543  1.0111  0.1582  0.0139  1.4707]
ln2_b norm,mean,std,min,max: [ 2.8769  0.0275  0.1442 -0.3744  0.3652]
W_q Frobenius: 31.6763 smax,energy1,energy4,energy16,stable_rank: [ 8.9845  0.0804  0.2024  0.4647 12.4303]
W_k Frobenius: 28.6017 smax,energy1,energy4,energy16,stable_rank: [10.9672  0.147   0.321   0.5501  6.8013]
W_v Frobenius: 28.4686 smax,energy1,energy4,energy16,stable_rank: [ 5.3505  0.0353  0.1262  0.3844 28.3105]
W_o Frobenius: 26.151 smax,energy1,energy4,energy16,stable_rank: [ 4.1901  0.0257  0.0783  0.2426 38.9526]
W_fc Frobenius: 80.2609 smax,energy1,energy4,energy16,stable_rank: [20.7957  0.0671  0.1558  0.4206 14.8956]
W_proj Frobenius: 75.1192 smax,energy1,energy4,energy16,stable_rank: [ 7.6249  0.0103  0.0358  0.1103 97.0588]

LAYER 2
ln1_w norm,mean,std,min,max: [14.4574  0.7343  0.0712  0.3445  0.9047]
ln1_b norm,mean,std,min,max: [ 2.6214  0.0244  0.1315 -0.9417  0.7326]
ln2_w norm,mean,std,min,max: [25.082   1.275   0.1126  0.0775  1.5018]
ln2_b norm,mean,std,min,max: [ 2.911   0.0188  0.1474 -0.4157  0.3635]
W_q Frobenius: 37.4829 smax,energy1,energy4,energy16,stable_rank: [ 5.0672  0.0183  0.0621  0.2013 54.7179]
W_k Frobenius: 36.3557 smax,energy1,energy4,energy16,stable_rank: [ 8.2935  0.052   0.1273  0.2535 19.2163]
W_v Frobenius: 34.7465 smax,energy1,energy4,energy16,stable_rank: [ 3.89    0.0125  0.0476  0.1701 79.7848]
W_o Frobenius: 34.2053 smax,energy1,energy4,energy16,stable_rank: [ 4.4416  0.0169  0.0534  0.1609 59.3076]
W_fc Frobenius: 79.4541 smax,energy1,energy4,energy16,stable_rank: [14.8285  0.0348  0.0646  0.1655 28.7103]
W_proj Frobenius: 77.4765 smax,energy1,energy4,energy16,stable_rank: [10.0946  0.017   0.0517  0.1553 58.9061]

LAYER 3
ln1_w norm,mean,std,min,max: [15.1046  0.7694  0.0469  0.451   0.8778]
ln1_b norm,mean,std,min,max: [ 1.8619  0.0102  0.0945 -0.5141  0.6285]
ln2_w norm,mean,std,min,max: [24.6574  1.2552  0.0888  0.2619  1.4568]
ln2_b norm,mean,std,min,max: [ 2.4972  0.014   0.1267 -0.5112  0.3808]
W_q Frobenius: 33.9401 smax,energy1,energy4,energy16,stable_rank: [ 4.1544  0.015   0.0528  0.169  66.7429]
W_k Frobenius: 33.2214 smax,energy1,energy4,energy16,stable_rank: [ 4.8875  0.0216  0.0582  0.1695 46.2013]
W_v Frobenius: 38.9389 smax,energy1,energy4,energy16,stable_rank: [ 4.6968  0.0145  0.0507  0.1776 68.7339]
W_o Frobenius: 39.0173 smax,energy1,energy4,energy16,stable_rank: [ 9.1753  0.0553  0.1082  0.2289 18.0831]
W_fc Frobenius: 78.1887 smax,energy1,energy4,energy16,stable_rank: [14.3731  0.0338  0.0652  0.1466 29.593 ]
W_proj Frobenius: 85.0885 smax,energy1,energy4,energy16,stable_rank: [28.1752  0.1096  0.2685  0.7207  9.1203]

HEADS: raw weight products only; no inputs or forward passes
QK = diag(ln1_w) Q.T K diag(ln1_w); OV = O V diag(ln1_w)
Fields: QK[Frob,smax,E1,E4,E16,stable_rank,symmetry,trace/Frob]; OV[same]; QK bias norms
L0H0 QK [ 0.1137  0.0887  0.6084  0.9803  0.9938  1.6437  0.1808 -0.4465] OV [4.3232 3.427  0.6284 0.9782 0.9978 1.5915 0.1036 0.3853] bias [0.0185 0.0161]
L0H1 QK [ 0.878   0.6014  0.4691  0.8225  0.9825  2.1316 -0.0996 -0.5564] OV [ 3.492   0.9977  0.0816  0.2774  0.7829 12.2507  0.0048  0.0475] bias [0.0738 0.1057]
L0H2 QK [0.1747 0.132  0.571  0.8741 0.9836 1.7515 0.4027 0.3123] OV [2.0255 1.0554 0.2715 0.735  0.9864 3.6836 0.0091 0.0061] bias [0.0235 0.0795]
L0H3 QK [ 0.2396  0.1646  0.4721  0.9452  0.995   2.1184  0.209  -0.781 ] OV [1.6238 1.0268 0.3999 0.6717 0.9638 2.5008 0.0428 0.1465] bias [0.057  0.1489]
L0H4 QK [ 0.0093  0.0045  0.2333  0.4424  0.7671  4.2872  0.023  -0.4017] OV [17.164  13.8775  0.6537  0.9945  0.9998  1.5297  0.0696  0.3518] bias [0.0009 0.0014]
L0H5 QK [ 0.9716  0.41    0.1781  0.4647  0.9151  5.6164 -0.2279  1.2505] OV [ 3.7103  1.0607  0.0817  0.2937  0.8197 12.2366 -0.0001 -0.6652] bias [0.0509 0.0664]
L1H0 QK [ 0.6446  0.3538  0.3013  0.637   0.9406  3.3191  0.02   -0.4775] OV [13.4437  4.9469  0.1354  0.4465  0.9116  7.3855  0.0835 -1.2128] bias [0.8392 0.8416]
L1H1 QK [ 1.5863  1.1468  0.5227  0.8237  0.9724  1.9132 -0.003  -0.8032] OV [ 9.5953  2.9286  0.0932  0.316   0.8088 10.7351  0.0596 -1.1678] bias [3.1501 2.1826]
L1H2 QK [ 1.2384  0.8093  0.4271  0.8409  0.97    2.3412 -0.0153 -0.5996] OV [11.4977  3.3252  0.0836  0.3059  0.8276 11.9561  0.0717 -1.2953] bias [2.1999 1.8979]
L1H3 QK [ 1.3117  0.9143  0.4858  0.851   0.9717  2.0583 -0.0428 -0.5311] OV [10.0448  3.4201  0.1159  0.3632  0.879   8.626   0.0507 -1.0525] bias [2.7194 2.2078]
L1H4 QK [ 1.2528  0.852   0.4624  0.8389  0.9728  2.1624 -0.0363 -0.468 ] OV [ 9.5599  3.0704  0.1032  0.363   0.8779  9.6946  0.0441 -1.093 ] bias [2.5182 1.946 ]
L1H5 QK [ 1.463   1.0096  0.4762  0.8243  0.9723  2.1001  0.0101 -0.8248] OV [ 9.8289  2.8188  0.0822  0.284   0.795  12.1583  0.0705 -1.3273] bias [2.4602 1.8112]
L2H0 QK [ 2.0464  0.5184  0.0642  0.2121  0.5791 15.5805 -0.0029 -0.2845] OV [18.7016  4.0788  0.0476  0.1655  0.5169 21.0226  0.08   -1.9232] bias [0.8951 0.7758]
L2H1 QK [ 1.9469  0.4596  0.0557  0.1849  0.5341 17.9419  0.0397  1.292 ] OV [21.4533  6.411   0.0893  0.2322  0.6051 11.1977  0.0711 -1.8751] bias [0.6381 0.3339]
L2H2 QK [ 2.8055  0.709   0.0639  0.2346  0.6511 15.6583  0.0319  0.5324] OV [16.7872  3.4921  0.0433  0.1588  0.5117 23.1086  0.0678 -1.7551] bias [1.5837 1.5815]
L2H3 QK [ 1.9768  0.5709  0.0834  0.2122  0.5687 11.9897 -0.0118 -0.206 ] OV [20.6209  4.6307  0.0504  0.1741  0.5221 19.8301  0.0902 -1.8701] bias [1.0369 1.2654]
L2H4 QK [2.2031 0.7829 0.1263 0.2841 0.6169 7.9187 0.0004 0.0247] OV [18.4176  4.0021  0.0472  0.1749  0.5515 21.1785  0.0964 -2.1041] bias [1.2005 1.3405]
L2H5 QK [ 1.6549  0.4205  0.0646  0.2118  0.544  15.4887 -0.0056 -0.4237] OV [23.7711  6.3519  0.0714  0.2127  0.5645 14.0055  0.0526 -1.4279] bias [0.6363 0.4924]
L3H0 QK [ 2.0781  0.4054  0.0381  0.1422  0.4558 26.2735  0.0198  0.8438] OV [24.5048  5.5215  0.0508  0.1639  0.4941 19.6967  0.0673 -1.9719] bias [0.2989 0.3023]
L3H1 QK [ 1.6954  0.3227  0.0362  0.1359  0.4478 27.6063  0.0219  1.3128] OV [25.729   7.2188  0.0787  0.2031  0.5765 12.7031  0.0819 -1.9283] bias [0.2636 0.2534]
L3H2 QK [ 2.0613  0.4339  0.0443  0.1511  0.4553 22.5722 -0.0067  0.3486] OV [26.8659  9.0071  0.1124  0.2368  0.557   8.8968  0.074  -1.8824] bias [0.3424 0.2865]
L3H3 QK [ 1.0538  0.2628  0.0622  0.1887  0.4999 16.0832 -0.0072  0.0023] OV [37.9438 14.6757  0.1496  0.3298  0.722   6.6848  0.0658 -1.3482] bias [0.2584 0.1604]
L3H4 QK [ 1.735   0.4007  0.0533  0.1758  0.4846 18.7462  0.0041  0.0025] OV [28.4456  9.176   0.1041  0.2782  0.6026  9.61    0.0392 -1.231 ] bias [0.3069 0.2391]
L3H5 QK [ 2.1176  0.4042  0.0364  0.1337  0.4409 27.4514  0.2212  3.8   ] OV [23.6935  9.9785  0.1774  0.3245  0.7239  5.638   0.2049 -2.3992] bias [0.3101 0.308 ]

```
CLI OUTPUT IN EVENT ORDER
ERROR
Code Mode is unavailable because code-mode host is disabled. Code mode will fail closed; enable `features.code_mode_host` and install `codex-code-mode-host`.

AGENT_MESSAGE
```python
import numpy as np
from scipy.linalg import svd, svdvals
from threadpoolctl import threadpool_limits
threadpool_limits(4)

for name in ["W_q.0", "W_k.0", "W_v.0", "W_fc.0", "W_proj.0", "W_proj.3"]:
    a = np.asarray(W[name], dtype=np.float64)
    s = svdvals(a, check_finite=False)
    print(name, "first 48 singular values:", np.round(s[:48], 4))
    print("singular values 64,96,128,192,256,320,384:",
          np.round(s[np.array([64,96,128,192,256,320,384])-1],4))

A, C, Q, K = {}, {}, {}, {}
for l in range(4):
    g = np.asarray(W[f"ln1_w.{l}"],dtype=float)
    for j in range(6):
        sl = slice(j*64,(j+1)*64)
        name = f"L{l}H{j}"
        Q[name] = np.asarray(W[f"W_q.{l}"][sl],dtype=float)*g
        K[name] = np.asarray(W[f"W_k.{l}"][sl],dtype=float)*g
        A[name] = Q[name].T @ K[name] / 8
        C[name] = np.asarray(W[f"W_o.{l}"][:,sl],dtype=float) @ (
            np.asarray(W[f"W_v.{l}"][sl],dtype=float)*g)

names = list(A)
print("\nL0 QK eigenvalues with greatest absolute value:")
for n in names[:6]:
    z = np.linalg.eigvals(A[n])
    z = z[np.argsort(-np.abs(z))][:16]
    print(n, np.round(z,4))

print("\nL0 QK pairwise normalized Frobenius inner products:")
print(np.round([[np.sum(A[a]*A[b])/np.linalg.norm(A[a])/np.linalg.norm(A[b])
                 for b in names[:6]] for a in names[:6]],3))
print("\nL1 QK pairwise normalized Frobenius inner products:")
print(np.round([[np.sum(A[a]*A[b])/np.linalg.norm(A[a])/np.linalg.norm(A[b])
                 for b in names[6:12]] for a in names[6:12]],3))

print("\nEarlier-head output to later-head query/key weight composition.")
print("Each entry: ||Q_l C_earlier||F/(||Q_l||F ||C_earlier||F), and analogous K; random-isotropic scale ~0.051.")
for l in range(1,4):
    earlier = [f"L{a}H{b}" for a in range(l) for b in range(6)]
    print("earlier:", " ".join(earlier))
    for j in range(6):
        n = f"L{l}H{j}"
        qc = [np.linalg.norm(Q[n]@C[e])/np.linalg.norm(Q[n])/np.linalg.norm(C[e]) for e in earlier]
        kc = [np.linalg.norm(K[n]@C[e])/np.linalg.norm(K[n])/np.linalg.norm(C[e]) for e in earlier]
        print(n, "Q", np.round(qc,3), "K", np.round(kc,3))

print("\nLayer-0 effective input-weight joint spectrum:")
m = np.concatenate([W[f"{name}.0"]*W["ln1_w.0"] for name in ["W_q","W_k","W_v"]])
u,s,vt = svd(m,full_matrices=False,check_finite=False)
print(np.round(s[:80],4))
print("L0 ln1 gain quantiles:",np.round(np.quantile(W["ln1_w.0"],[0,.1,.25,.5,.75,.9,.95,1]),4))
print("\nFinal projection column norms quantiles:",
      np.round(np.quantile(np.linalg.norm(W["W_proj.3"],axis=0),[0,.1,.25,.5,.75,.9,1]),4))
```


PERMITTED EXPERIMENT HARNESS RECORD: python
{
  "t": 1792713266.795944,
  "s": 3.24,
  "code": "import numpy as np\nfrom scipy.linalg import svd, svdvals\nfrom threadpoolctl import threadpool_limits\nthreadpool_limits(4)\n\nfor name in [\"W_q.0\", \"W_k.0\", \"W_v.0\", \"W_fc.0\", \"W_proj.0\", \"W_proj.3\"]:\n    a = np.asarray(W[name], dtype=np.float64)\n    s = svdvals(a, check_finite=False)\n    print(name, \"first 48 singular values:\", np.round(s[:48], 4))\n    print(\"singular values 64,96,128,192,256,320,384:\",\n          np.round(s[np.array([64,96,128,192,256,320,384])-1],4))\n\nA, C, Q, K = {}, {}, {}, {}\nfor l in range(4):\n    g = np.asarray(W[f\"ln1_w.{l}\"],dtype=float)\n    for j in range(6):\n        sl = slice(j*64,(j+1)*64)\n        name = f\"L{l}H{j}\"\n        Q[name] = np.asarray(W[f\"W_q.{l}\"][sl],dtype=float)*g\n        K[name] = np.asarray(W[f\"W_k.{l}\"][sl],dtype=float)*g\n        A[name] = Q[name].T @ K[name] / 8\n        C[name] = np.asarray(W[f\"W_o.{l}\"][:,sl],dtype=float) @ (\n            np.asarray(W[f\"W_v.{l}\"][sl],dtype=float)*g)\n\nnames = list(A)\nprint(\"\\nL0 QK eigenvalues with greatest absolute value:\")\nfor n in names[:6]:\n    z = np.linalg.eigvals(A[n])\n    z = z[np.argsort(-np.abs(z))][:16]\n    print(n, np.round(z,4))\n\nprint(\"\\nL0 QK pairwise normalized Frobenius inner products:\")\nprint(np.round([[np.sum(A[a]*A[b])/np.linalg.norm(A[a])/np.linalg.norm(A[b])\n                 for b in names[:6]] for a in names[:6]],3))\nprint(\"\\nL1 QK pairwise normalized Frobenius inner products:\")\nprint(np.round([[np.sum(A[a]*A[b])/np.linalg.norm(A[a])/np.linalg.norm(A[b])\n                 for b in names[6:12]] for a in names[6:12]],3))\n\nprint(\"\\nEarlier-head output to later-head query/key weight composition.\")\nprint(\"Each entry: ||Q_l C_earlier||F/(||Q_l||F ||C_earlier||F), and analogous K; random-isotropic scale ~0.051.\")\nfor l in range(1,4):\n    earlier = [f\"L{a}H{b}\" for a in range(l) for b in range(6)]\n    print(\"earlier:\", \" \".join(earlier))\n    for j in range(6):\n        n = f\"L{l}H{j}\"\n        qc = [np.linalg.norm(Q[n]@C[e])/np.linalg.norm(Q[n])/np.linalg.norm(C[e]) for e in earlier]\n        kc = [np.linalg.norm(K[n]@C[e])/np.linalg.norm(K[n])/np.linalg.norm(C[e]) for e in earlier]\n        print(n, \"Q\", np.round(qc,3), \"K\", np.round(kc,3))\n\nprint(\"\\nLayer-0 effective input-weight joint spectrum:\")\nm = np.concatenate([W[f\"{name}.0\"]*W[\"ln1_w.0\"] for name in [\"W_q\",\"W_k\",\"W_v\"]])\nu,s,vt = svd(m,full_matrices=False,check_finite=False)\nprint(np.round(s[:80],4))\nprint(\"L0 ln1 gain quantiles:\",np.round(np.quantile(W[\"ln1_w.0\"],[0,.1,.25,.5,.75,.9,.95,1]),4))\nprint(\"\\nFinal projection column norms quantiles:\",\n      np.round(np.quantile(np.linalg.norm(W[\"W_proj.3\"],axis=0),[0,.1,.25,.5,.75,.9,1]),4))\n",
  "out": "W_q.0 first 48 singular values: [5.0385 4.9994 4.7025 4.377  4.3428 4.2944 4.0581 4.0325 3.9503 3.9226 3.7953 3.7436 3.7024 3.6308 3.6084 3.5301 3.4694 3.4156 3.3249 3.2407 3.1963\n 3.0492 2.8994 2.7778 2.6226 2.526  2.411  2.3704 2.2612 2.1858 2.1548 2.0897 2.0468 2.0064 1.927  1.8521 1.8265 1.7527 1.7504 1.685  1.6599 1.6231\n 1.5999 1.5427 1.5015 1.4688 1.4388 1.4052]\nsingular values 64,96,128,192,256,320,384: [1.0116 0.6385 0.5023 0.3374 0.2096 0.1008 0.0006]\nW_k.0 first 48 singular values: [6.2042 5.0521 4.5716 4.5372 4.3914 4.2894 4.2839 4.2083 4.1192 4.0728 3.9591 3.9166 3.8164 3.7144 3.6762 3.5481 3.4758 3.4002 3.3555 3.2717 3.1133\n 3.0071 2.9759 2.5777 2.5256 2.4521 2.3805 2.3248 2.2941 2.2108 2.1249 2.0938 2.0501 1.9858 1.9423 1.8583 1.8195 1.7392 1.7032 1.6831 1.6343 1.6321\n 1.5743 1.5319 1.5167 1.4825 1.4721 1.4361]\nsingular values 64,96,128,192,256,320,384: [1.0208 0.6407 0.5068 0.3471 0.2228 0.1081 0.0001]\nW_v.0 first 48 singular values: [4.5533 4.0243 3.7382 3.4973 3.4646 3.3672 3.2791 3.2148 3.1361 3.1132 3.0727 3.0202 2.9534 2.8934 2.8666 2.8461 2.8031 2.7388 2.7018 2.6739 2.5894\n 2.5749 2.5363 2.4657 2.4438 2.4085 2.2455 2.0897 2.0349 1.9668 1.6206 1.4925 1.4029 1.3239 1.2111 1.188  1.1286 1.1118 1.0779 1.0586 1.042  1.0188\n 0.9988 0.9903 0.9674 0.9492 0.9227 0.91  ]\nsingular values 64,96,128,192,256,320,384: [0.685  0.5275 0.4409 0.3124 0.2032 0.0994 0.0002]\nW_fc.0 first 48 singular values: [18.1104 12.2798  8.4726  7.9016  7.7793  7.442   7.1458  7.1094  7.0083  6.8053  6.7211  6.5398  6.3779  6.3286  6.2698  6.2168  6.2037  6.0345\n  6.0235  5.9437  5.8446  5.7365  5.7016  5.6681  5.6093  5.5798  5.498   5.4472  5.3773  5.3341  5.2437  5.203   5.1406  5.1261  4.9896  4.9434\n  4.9202  4.8457  4.7874  4.7161  4.6694  4.6008  4.5188  4.4395  4.4082  4.3845  4.3074  4.2416]\nsingular values 64,96,128,192,256,320,384: [3.2813 1.7448 1.2638 0.9404 0.7499 0.5961 0.4206]\nW_proj.0 first 48 singular values: [10.9319  7.576   6.6285  6.2932  6.166   5.9239  5.886   5.7686  5.7059  5.6097  5.4949  5.4626  5.4044  5.3131  5.1933  5.1607  5.1191  5.0699\n  5.0033  4.9374  4.8767  4.7926  4.6927  4.6637  4.6514  4.6368  4.5477  4.5258  4.4869  4.4042  4.329   4.2833  4.2468  4.2063  4.1781  4.1409\n  4.1201  4.1053  4.0623  4.0225  3.9971  3.9583  3.9344  3.9041  3.8574  3.8499  3.7981  3.7584]\nsingular values 64,96,128,192,256,320,384: [3.2682 2.5003 1.9364 1.1717 0.7078 0.4323 0.2442]\nW_proj.3 first 48 singular values: [28.1752 20.2934 19.2921 19.1349 18.7081 18.5697 18.1393 17.7759 17.4845 16.5396 15.9131 15.6096 14.9752 14.8725 14.4167 14.3133 13.3269 12.9707\n 12.5934 12.3698 12.2251 12.0191 11.3225 11.115  10.7693  9.2267  8.5731  7.929   6.5001  5.2712  3.9657  3.1843  2.7917  2.6972  2.6532  2.5552\n  2.511   2.4709  2.3788  2.29    2.25    2.2147  2.1167  2.0811  2.0364  2.0107  1.9843  1.9692]\nsingular values 64,96,128,192,256,320,384: [1.6923 1.3641 1.1385 0.8324 0.6319 0.4812 0.3244]\n\nL0 QK eigenvalues with greatest absolute value:\nL0H0 [-0.049 +0.j      0.0061+0.0086j  0.0061-0.0086j -0.0029+0.j     -0.0011+0.0016j -0.0011-0.0016j -0.0016+0.j     -0.0015+0.0002j -0.0015-0.0002j\n -0.0007+0.0012j -0.0007-0.0012j -0.0013+0.j      0.0009+0.0007j  0.0009-0.0007j -0.001 +0.0004j -0.001 -0.0004j]\nL0H1 [ 0.2178-0.3452j  0.2178+0.3452j -0.2844+0.j     -0.1232+0.1495j -0.1232-0.1495j -0.0978+0.0983j -0.0978-0.0983j -0.032 -0.0412j -0.032 +0.0412j\n  0.0256-0.0327j  0.0256+0.0327j -0.0399+0.j      0.0001+0.032j   0.0001-0.032j  -0.0279+0.012j  -0.0279-0.012j ]\nL0H2 [ 0.0905+0.j      0.0535+0.j     -0.022 +0.j     -0.0133+0.0007j -0.0133-0.0007j  0.0114+0.j     -0.01  +0.j     -0.0057+0.0041j -0.0057-0.0041j\n -0.0069+0.j     -0.0056+0.001j  -0.0056-0.001j   0.0054+0.j     -0.0039+0.j     -0.0036+0.j     -0.0019-0.0027j]\nL0H3 [-0.0775-0.0055j -0.0775+0.0055j -0.0198+0.j      0.006 +0.0166j  0.006 -0.0166j  0.0075+0.j     -0.0062+0.0016j -0.0062-0.0016j -0.0045+0.001j\n -0.0045-0.001j  -0.0019+0.004j  -0.0019-0.004j  -0.0035+0.j   "
}

========================================================================
TURN t003
MESSAGE SENT TO CODEX
OUTPUT of run 2 of at most 100:
```
W_q.0 first 48 singular values: [5.0385 4.9994 4.7025 4.377  4.3428 4.2944 4.0581 4.0325 3.9503 3.9226 3.7953 3.7436 3.7024 3.6308 3.6084 3.5301 3.4694 3.4156 3.3249 3.2407 3.1963
 3.0492 2.8994 2.7778 2.6226 2.526  2.411  2.3704 2.2612 2.1858 2.1548 2.0897 2.0468 2.0064 1.927  1.8521 1.8265 1.7527 1.7504 1.685  1.6599 1.6231
 1.5999 1.5427 1.5015 1.4688 1.4388 1.4052]
singular values 64,96,128,192,256,320,384: [1.0116 0.6385 0.5023 0.3374 0.2096 0.1008 0.0006]
W_k.0 first 48 singular values: [6.2042 5.0521 4.5716 4.5372 4.3914 4.2894 4.2839 4.2083 4.1192 4.0728 3.9591 3.9166 3.8164 3.7144 3.6762 3.5481 3.4758 3.4002 3.3555 3.2717 3.1133
 3.0071 2.9759 2.5777 2.5256 2.4521 2.3805 2.3248 2.2941 2.2108 2.1249 2.0938 2.0501 1.9858 1.9423 1.8583 1.8195 1.7392 1.7032 1.6831 1.6343 1.6321
 1.5743 1.5319 1.5167 1.4825 1.4721 1.4361]
singular values 64,96,128,192,256,320,384: [1.0208 0.6407 0.5068 0.3471 0.2228 0.1081 0.0001]
W_v.0 first 48 singular values: [4.5533 4.0243 3.7382 3.4973 3.4646 3.3672 3.2791 3.2148 3.1361 3.1132 3.0727 3.0202 2.9534 2.8934 2.8666 2.8461 2.8031 2.7388 2.7018 2.6739 2.5894
 2.5749 2.5363 2.4657 2.4438 2.4085 2.2455 2.0897 2.0349 1.9668 1.6206 1.4925 1.4029 1.3239 1.2111 1.188  1.1286 1.1118 1.0779 1.0586 1.042  1.0188
 0.9988 0.9903 0.9674 0.9492 0.9227 0.91  ]
singular values 64,96,128,192,256,320,384: [0.685  0.5275 0.4409 0.3124 0.2032 0.0994 0.0002]
W_fc.0 first 48 singular values: [18.1104 12.2798  8.4726  7.9016  7.7793  7.442   7.1458  7.1094  7.0083  6.8053  6.7211  6.5398  6.3779  6.3286  6.2698  6.2168  6.2037  6.0345
  6.0235  5.9437  5.8446  5.7365  5.7016  5.6681  5.6093  5.5798  5.498   5.4472  5.3773  5.3341  5.2437  5.203   5.1406  5.1261  4.9896  4.9434
  4.9202  4.8457  4.7874  4.7161  4.6694  4.6008  4.5188  4.4395  4.4082  4.3845  4.3074  4.2416]
singular values 64,96,128,192,256,320,384: [3.2813 1.7448 1.2638 0.9404 0.7499 0.5961 0.4206]
W_proj.0 first 48 singular values: [10.9319  7.576   6.6285  6.2932  6.166   5.9239  5.886   5.7686  5.7059  5.6097  5.4949  5.4626  5.4044  5.3131  5.1933  5.1607  5.1191  5.0699
  5.0033  4.9374  4.8767  4.7926  4.6927  4.6637  4.6514  4.6368  4.5477  4.5258  4.4869  4.4042  4.329   4.2833  4.2468  4.2063  4.1781  4.1409
  4.1201  4.1053  4.0623  4.0225  3.9971  3.9583  3.9344  3.9041  3.8574  3.8499  3.7981  3.7584]
singular values 64,96,128,192,256,320,384: [3.2682 2.5003 1.9364 1.1717 0.7078 0.4323 0.2442]
W_proj.3 first 48 singular values: [28.1752 20.2934 19.2921 19.1349 18.7081 18.5697 18.1393 17.7759 17.4845 16.5396 15.9131 15.6096 14.9752 14.8725 14.4167 14.3133 13.3269 12.9707
 12.5934 12.3698 12.2251 12.0191 11.3225 11.115  10.7693  9.2267  8.5731  7.929   6.5001  5.2712  3.9657  3.1843  2.7917  2.6972  2.6532  2.5552
  2.511   2.4709  2.3788  2.29    2.25    2.2147  2.1167  2.0811  2.0364  2.0107  1.9843  1.9692]
singular values 64,96,128,192,256,320,384: [1.6923 1.3641 1.1385 0.8324 0.6319 0.4812 0.3244]

L0 QK eigenvalues with greatest absolute value:
L0H0 [-0.049 +0.j      0.0061+0.0086j  0.0061-0.0086j -0.0029+0.j     -0.0011+0.0016j -0.0011-0.0016j -0.0016+0.j     -0.0015+0.0002j -0.0015-0.0002j
 -0.0007+0.0012j -0.0007-0.0012j -0.0013+0.j      0.0009+0.0007j  0.0009-0.0007j -0.001 +0.0004j -0.001 -0.0004j]
L0H1 [ 0.2178-0.3452j  0.2178+0.3452j -0.2844+0.j     -0.1232+0.1495j -0.1232-0.1495j -0.0978+0.0983j -0.0978-0.0983j -0.032 -0.0412j -0.032 +0.0412j
  0.0256-0.0327j  0.0256+0.0327j -0.0399+0.j      0.0001+0.032j   0.0001-0.032j  -0.0279+0.012j  -0.0279-0.012j ]
L0H2 [ 0.0905+0.j      0.0535+0.j     -0.022 +0.j     -0.0133+0.0007j -0.0133-0.0007j  0.0114+0.j     -0.01  +0.j     -0.0057+0.0041j -0.0057-0.0041j
 -0.0069+0.j     -0.0056+0.001j  -0.0056-0.001j   0.0054+0.j     -0.0039+0.j     -0.0036+0.j     -0.0019-0.0027j]
L0H3 [-0.0775-0.0055j -0.0775+0.0055j -0.0198+0.j      0.006 +0.0166j  0.006 -0.0166j  0.0075+0.j     -0.0062+0.0016j -0.0062-0.0016j -0.0045+0.001j
 -0.0045-0.001j  -0.0019+0.004j  -0.0019-0.004j  -0.0035+0.j     -0.0019-0.0025j -0.0019+0.0025j  0.0003-0.0031j]
L0H4 [-0.0012-0.0006j -0.0012+0.0006j  0.0001-0.0007j  0.0001+0.0007j -0.0005+0.0005j -0.0005-0.0005j -0.0007+0.j      0.0006+0.j     -0.0004+0.0005j
 -0.0004-0.0005j -0.0006+0.0001j -0.0006-0.0001j -0.0002+0.0005j -0.0002-0.0005j  0.0004+0.0004j  0.0004-0.0004j]
L0H5 [ 0.2071+0.2748j  0.2071-0.2748j  0.1137+0.1719j  0.1137-0.1719j  0.1785+0.j      0.1545+0.0845j  0.1545-0.0845j  0.0348+0.1657j  0.0348-0.1657j
  0.0944+0.1369j  0.0944-0.1369j -0.0407+0.139j  -0.0407-0.139j   0.0076+0.1389j  0.0076-0.1389j -0.045 +0.1213j]

L0 QK pairwise normalized Frobenius inner products:
[[ 1.     0.179  0.112  0.206  0.063  0.093]
 [ 0.179  1.     0.21  -0.15   0.005  0.413]
 [ 0.112  0.21   1.    -0.247 -0.006  0.113]
 [ 0.206 -0.15  -0.247  1.    -0.027 -0.098]
 [ 0.063  0.005 -0.006 -0.027  1.    -0.004]
 [ 0.093  0.413  0.113 -0.098 -0.004  1.   ]]

L1 QK pairwise normalized Frobenius inner products:
[[1.    0.302 0.393 0.326 0.337 0.265]
 [0.302 1.    0.525 0.591 0.568 0.566]
 [0.393 0.525 1.    0.539 0.579 0.451]
 [0.326 0.591 0.539 1.    0.608 0.524]
 [0.337 0.568 0.579 0.608 1.    0.48 ]
 [0.265 0.566 0.451 0.524 0.48  1.   ]]

Earlier-head output to later-head query/key weight composition.
Each entry: ||Q_l C_earlier||F/(||Q_l||F ||C_earlier||F), and analogous K; random-isotropic scale ~0.051.
earlier: L0H0 L0H1 L0H2 L0H3 L0H4 L0H5
L1H0 Q [0.075 0.085 0.095 0.063 0.074 0.089] K [0.273 0.066 0.076 0.122 0.339 0.072]
L1H1 Q [0.074 0.076 0.062 0.054 0.08  0.09 ] K [0.393 0.037 0.093 0.152 0.499 0.06 ]
L1H2 Q [0.073 0.079 0.061 0.056 0.079 0.09 ] K [0.376 0.041 0.089 0.145 0.481 0.058]
L1H3 Q [0.082 0.073 0.056 0.06  0.092 0.084] K [0.397 0.04  0.094 0.154 0.505 0.059]
L1H4 Q [0.078 0.076 0.062 0.059 0.084 0.09 ] K [0.389 0.04  0.093 0.15  0.493 0.06 ]
L1H5 Q [0.068 0.076 0.068 0.054 0.074 0.093] K [0.375 0.042 0.086 0.143 0.472 0.066]
earlier: L0H0 L0H1 L0H2 L0H3 L0H4 L0H5 L1H0 L1H1 L1H2 L1H3 L1H4 L1H5
L2H0 Q [0.04  0.054 0.052 0.043 0.041 0.058 0.07  0.055 0.059 0.06  0.057 0.058] K [0.153 0.051 0.064 0.079 0.182 0.06  0.064 0.052 0.055 0.051 0.051 0.051]
L2H1 Q [0.039 0.048 0.063 0.044 0.029 0.067 0.071 0.055 0.063 0.058 0.058 0.054] K [0.053 0.063 0.065 0.049 0.047 0.076 0.065 0.064 0.068 0.07  0.068 0.063]
L2H2 Q [0.044 0.053 0.044 0.043 0.049 0.08  0.058 0.056 0.066 0.064 0.063 0.054] K [0.196 0.047 0.065 0.087 0.256 0.065 0.061 0.05  0.057 0.053 0.055 0.049]
L2H3 Q [0.044 0.055 0.05  0.044 0.053 0.055 0.062 0.058 0.059 0.057 0.061 0.057] K [0.202 0.048 0.065 0.09  0.262 0.057 0.059 0.048 0.051 0.049 0.05  0.049]
L2H4 Q [0.051 0.054 0.046 0.046 0.056 0.059 0.063 0.058 0.059 0.061 0.06  0.057] K [0.194 0.049 0.082 0.095 0.239 0.059 0.072 0.05  0.055 0.051 0.053 0.049]
L2H5 Q [0.035 0.052 0.065 0.045 0.034 0.052 0.074 0.056 0.059 0.056 0.055 0.057] K [0.132 0.047 0.066 0.072 0.154 0.054 0.059 0.049 0.05  0.05  0.05  0.05 ]
earlier: L0H0 L0H1 L0H2 L0H3 L0H4 L0H5 L1H0 L1H1 L1H2 L1H3 L1H4 L1H5 L2H0 L2H1 L2H2 L2H3 L2H4 L2H5
L3H0 Q [0.029 0.048 0.046 0.041 0.027 0.055 0.054 0.049 0.052 0.05  0.051 0.05  0.051 0.054 0.047 0.051 0.053 0.053] K [0.072 0.051 0.059 0.051 0.088 0.057 0.057 0.05  0.052 0.05  0.052 0.049 0.051 0.049 0.052 0.051 0.05  0.052]
L3H1 Q [0.033 0.046 0.052 0.05  0.029 0.056 0.051 0.05  0.053 0.05  0.05  0.048 0.051 0.06  0.042 0.046 0.051 0.056] K [0.057 0.051 0.058 0.051 0.055 0.068 0.055 0.054 0.06  0.057 0.059 0.053 0.054 0.052 0.061 0.055 0.057 0.054]
L3H2 Q [0.03  0.05  0.052 0.042 0.032 0.052 0.056 0.052 0.051 0.05  0.051 0.051 0.054 0.052 0.049 0.055 0.053 0.055] K [0.1   0.05  0.058 0.055 0.128 0.055 0.055 0.049 0.05  0.051 0.051 0.05  0.05  0.047 0.05  0.049 0.049 0.051]
L3H3 Q [0.037 0.05  0.058 0.047 0.035 0.046 0.054 0.05  0.049 0.049 0.05  0.05  0.053 0.059 0.044 0.052 0.052 0.062] K [0.06  0.05  0.058 0.05  0.068 0.049 0.055 0.049 0.049 0.05  0.051 0.051 0.048 0.061 0.048 0.048 0.048 0.051]
L3H4 Q [0.034 0.049 0.059 0.045 0.032 0.048 0.063 0.051 0.051 0.05  0.05  0.052 0.053 0.057 0.045 0.053 0.055 0.062] K [0.09  0.05  0.058 0.055 0.11  0.049 0.055 0.049 0.053 0.049 0.048 0.05  0.049 0.05  0.05  0.049 0.05  0.051]
L3H5 Q [0.035 0.058 0.05  0.045 0.034 0.074 0.06  0.056 0.063 0.063 0.064 0.055 0.049 0.058 0.047 0.058 0.053 0.056] K [0.046 0.066 0.058 0.049 0.043 0.078 0.063 0.061 0.068 0.068 0.068 0.057 0.054 0.045 0.075 0.069 0.056 0.059]

Layer-0 effective input-weight joint spectrum:
[6.4889 4.6815 2.7741 2.6602 2.5992 2.5298 2.3742 2.3592 2.2576 2.214  2.1753 2.1461 2.0689 2.0138 1.9794 1.9185 1.8535 1.8279 1.7753 1.7229 1.6336
 1.589  1.5141 1.4681 1.3811 1.3263 1.2661 1.2373 1.2072 1.1464 1.1285 1.0592 1.0497 1.0198 0.9817 0.9733 0.9634 0.9526 0.9036 0.8889 0.8757 0.8618
 0.8589 0.8444 0.831  0.8131 0.8028 0.7882 0.7749 0.7711 0.7611 0.7454 0.7388 0.7307 0.7202 0.705  0.6951 0.6842 0.6685 0.6595 0.6454 0.6379 0.6269
 0.6116 0.602  0.5964 0.5907 0.5717 0.5658 0.562  0.5522 0.5455 0.5351 0.5271 0.5184 0.511  0.5082 0.4943 0.4891 0.4805]
L0 ln1 gain quantiles: [0.0165 0.0879 0.1295 0.182  0.3262 0.614  0.7758 1.0134]

Final projection column norms quantiles: [1.305  1.7987 1.9599 2.1321 2.3282 2.5114 4.6207]

```
CLI OUTPUT IN EVENT ORDER
ERROR
Code Mode is unavailable because code-mode host is disabled. Code mode will fail closed; enable `features.code_mode_host` and install `codex-code-mode-host`.

AGENT_MESSAGE
```python
import numpy as np
from scipy.linalg import svd, eigvals
from threadpoolctl import threadpool_limits
threadpool_limits(4)

def centered_rows(a):
    a = np.asarray(a, dtype=float)
    return a - a.mean(axis=1, keepdims=True)

A,C,Q,K,V,O = {},{},{},{},{},{}
for l in range(4):
    g = np.asarray(W[f"ln1_w.{l}"],dtype=float)
    for j in range(6):
        s = slice(j*64,(j+1)*64)
        n = f"L{l}H{j}"
        Q[n]=centered_rows(W[f"W_q.{l}"][s]*g)
        K[n]=centered_rows(W[f"W_k.{l}"][s]*g)
        V[n]=centered_rows(W[f"W_v.{l}"][s]*g)
        O[n]=np.asarray(W[f"W_o.{l}"][:,s],dtype=float)
        O[n]=O[n]-O[n].mean(axis=0,keepdims=True)
        A[n]=Q[n].T@K[n]/8
        C[n]=O[n]@V[n]

print("INDUCTION ALGEBRA: trace(A_l C_earlier)/(||A_l||F ||C_earlier||F)")
print("Positive values indicate a possible query-to-preceding-token key match, not measured attention.")
for l in range(1,4):
    earlier=[f"L{a}H{b}" for a in range(l) for b in range(6)]
    print("columns",earlier)
    for j in range(6):
        n=f"L{l}H{j}"
        scores=[np.trace(A[n]@C[e])/np.linalg.norm(A[n])/np.linalg.norm(C[e]) for e in earlier]
        print(n,np.round(scores,4))

print("\nL0 effective Q,K,V singular values (first 32):")
for j in range(6):
    n=f"L0H{j}"
    for label,M in [("Q",Q[n]),("K",K[n]),("V",V[n])]:
        s=np.linalg.svd(M,compute_uv=False)
        print(n,label,np.round(s[:32],3))

bases={}
for j in range(6):
    for label,D in [("Q",Q),("K",K),("V",V)]:
        _,_,vt=svd(D[f"L0H{j}"],full_matrices=False)
        bases[f"{label}{j}"]=vt[:24].T
print("\nAverage squared overlap of leading 24-dimensional L0 subspaces:")
bn=["Q1","K1","Q5","K5","V1","V5"]
print(bn)
print(np.round([[np.linalg.norm(bases[x].T@bases[y])**2/24 for y in bn] for x in bn],3))

print("\nLayer 0 QK product correlations, centered effective matrices:")
a=A["L0H5"]
b=A["L0H1"]
for k in range(1,7):
    m=np.linalg.matrix_power(a,k)
    print("power",k,"cosine with H1",round(np.sum(m*b)/np.linalg.norm(m)/np.linalg.norm(b),4),
          "cosine with H1 transpose",round(np.sum(m*b.T)/np.linalg.norm(m)/np.linalg.norm(b),4))

print("\nTop-key and top-query directions of each head: overlap with L0H4 output subspace (top 4).")
u,s,vt=svd(C["L0H4"],full_matrices=False)
B=u[:,:4]
for l in range(4):
    for j in range(6):
        n=f"L{l}H{j}"
        _,_,vq=svd(Q[n],full_matrices=False)
        _,_,vk=svd(K[n],full_matrices=False)
        print(n,"Q",np.round(np.sum((vq[:4]@B)**2,axis=1),3),
              "K",np.round(np.sum((vk[:4]@B)**2,axis=1),3))
```


PERMITTED EXPERIMENT HARNESS RECORD: python
{
  "t": 1792713360.9313812,
  "s": 3.01,
  "code": "import numpy as np\nfrom scipy.linalg import svd, eigvals\nfrom threadpoolctl import threadpool_limits\nthreadpool_limits(4)\n\ndef centered_rows(a):\n    a = np.asarray(a, dtype=float)\n    return a - a.mean(axis=1, keepdims=True)\n\nA,C,Q,K,V,O = {},{},{},{},{},{}\nfor l in range(4):\n    g = np.asarray(W[f\"ln1_w.{l}\"],dtype=float)\n    for j in range(6):\n        s = slice(j*64,(j+1)*64)\n        n = f\"L{l}H{j}\"\n        Q[n]=centered_rows(W[f\"W_q.{l}\"][s]*g)\n        K[n]=centered_rows(W[f\"W_k.{l}\"][s]*g)\n        V[n]=centered_rows(W[f\"W_v.{l}\"][s]*g)\n        O[n]=np.asarray(W[f\"W_o.{l}\"][:,s],dtype=float)\n        O[n]=O[n]-O[n].mean(axis=0,keepdims=True)\n        A[n]=Q[n].T@K[n]/8\n        C[n]=O[n]@V[n]\n\nprint(\"INDUCTION ALGEBRA: trace(A_l C_earlier)/(||A_l||F ||C_earlier||F)\")\nprint(\"Positive values indicate a possible query-to-preceding-token key match, not measured attention.\")\nfor l in range(1,4):\n    earlier=[f\"L{a}H{b}\" for a in range(l) for b in range(6)]\n    print(\"columns\",earlier)\n    for j in range(6):\n        n=f\"L{l}H{j}\"\n        scores=[np.trace(A[n]@C[e])/np.linalg.norm(A[n])/np.linalg.norm(C[e]) for e in earlier]\n        print(n,np.round(scores,4))\n\nprint(\"\\nL0 effective Q,K,V singular values (first 32):\")\nfor j in range(6):\n    n=f\"L0H{j}\"\n    for label,M in [(\"Q\",Q[n]),(\"K\",K[n]),(\"V\",V[n])]:\n        s=np.linalg.svd(M,compute_uv=False)\n        print(n,label,np.round(s[:32],3))\n\nbases={}\nfor j in range(6):\n    for label,D in [(\"Q\",Q),(\"K\",K),(\"V\",V)]:\n        _,_,vt=svd(D[f\"L0H{j}\"],full_matrices=False)\n        bases[f\"{label}{j}\"]=vt[:24].T\nprint(\"\\nAverage squared overlap of leading 24-dimensional L0 subspaces:\")\nbn=[\"Q1\",\"K1\",\"Q5\",\"K5\",\"V1\",\"V5\"]\nprint(bn)\nprint(np.round([[np.linalg.norm(bases[x].T@bases[y])**2/24 for y in bn] for x in bn],3))\n\nprint(\"\\nLayer 0 QK product correlations, centered effective matrices:\")\na=A[\"L0H5\"]\nb=A[\"L0H1\"]\nfor k in range(1,7):\n    m=np.linalg.matrix_power(a,k)\n    print(\"power\",k,\"cosine with H1\",round(np.sum(m*b)/np.linalg.norm(m)/np.linalg.norm(b),4),\n          \"cosine with H1 transpose\",round(np.sum(m*b.T)/np.linalg.norm(m)/np.linalg.norm(b),4))\n\nprint(\"\\nTop-key and top-query directions of each head: overlap with L0H4 output subspace (top 4).\")\nu,s,vt=svd(C[\"L0H4\"],full_matrices=False)\nB=u[:,:4]\nfor l in range(4):\n    for j in range(6):\n        n=f\"L{l}H{j}\"\n        _,_,vq=svd(Q[n],full_matrices=False)\n        _,_,vk=svd(K[n],full_matrices=False)\n        print(n,\"Q\",np.round(np.sum((vq[:4]@B)**2,axis=1),3),\n              \"K\",np.round(np.sum((vk[:4]@B)**2,axis=1),3))\n",
  "out": "INDUCTION ALGEBRA: trace(A_l C_earlier)/(||A_l||F ||C_earlier||F)\nPositive values indicate a possible query-to-preceding-token key match, not measured attention.\ncolumns ['L0H0', 'L0H1', 'L0H2', 'L0H3', 'L0H4', 'L0H5']\nL1H0 [ 0.0065 -0.0001 -0.0079  0.005   0.0207 -0.0034]\nL1H1 [-0.0046  0.007  -0.0161  0.0055  0.0191  0.0154]\nL1H2 [ 0.0039  0.0007 -0.0168  0.0092  0.0326 -0.0027]\nL1H3 [ 0.0077 -0.0026 -0.0132  0.0063  0.0421  0.0124]\nL1H4 [ 0.0093  0.0022 -0.0181  0.0114  0.0439  0.0094]\nL1H5 [-0.0195  0.0045 -0.0134 -0.0004 -0.0036  0.0128]\ncolumns ['L0H0', 'L0H1', 'L0H2', 'L0H3', 'L0H4', 'L0H5', 'L1H0', 'L1H1', 'L1H2', 'L1H3', 'L1H4', 'L1H5']\nL2H0 [ 0.0049 -0.0006 -0.003   0.0078  0.0099 -0.0046 -0.0126  0.0039 -0.0003  0.0016 -0.0004  0.0057]\nL2H1 [0.0065 0.0014 0.0108 0.0025 0.0038 0.0776 0.0025 0.0479 0.0482 0.0575 0.0476 0.0351]\nL2H2 [ 0.0114 -0.0165 -0.0054  0.0015  0.02   -0.0387 -0.0418 -0.0162 -0.0434 -0.007  -0.023  -0.0069]\nL2H3 [ 0.0177 -0.0008 -0.0065  0.0066  0.0396 -0.0026 -0.0197  0.0049 -0.0162  0.0042 -0.0048  0.0044]\nL2H4 [ 0.0079 -0.0055 -0.004   0.0101  0.0246  0.0018 -0.0139 -0.0082 -0.0115 -0.0019 -0.0042 -0.006 ]\nL2H5 [0.009  0.0006 0.0001 0.0046 0.0188 0.0068 0.0001 0.0083 0.0095 0.0064 0.0068 0.0068]\ncolumns ['L0H0', 'L0H1', 'L0H2', 'L0H3', 'L0H4', 'L0H5', 'L1H0', 'L1H1', 'L1H2', 'L1H3', 'L1H4', 'L1H5', 'L2H0', 'L2H1', 'L2H2', 'L2H3', 'L2H4', 'L2H5']\nL3H0 [ 0.0098 -0.0036  0.0068  0.0077  0.0098  0.019   0.0079  0.0083  0.0101  0.006   0.006   0.0009 -0.0033 -0.0135  0.0238 -0.0008  0.0044 -0.0088]\nL3H1 [ 0.0078  0.0071  0.006   0.0014  0.0041  0.061   0.0127  0.0333  0.038   0.0453  0.0456  0.0271  0.0153 -0.0181  0.1104  0.0209  0.0053 -0.0002]\nL3H2 [ 0.0062 -0.0032  0.0015 -0.0008  0.013  -0.0012 -0.0116  0.0013 -0.0092  0.     -0.0089 -0.0036 -0.0046 -0.007  -0.0004 -0.0033 -0.0013 -0.0031]\nL3H3 [ 0.0045  0.0032  0.0048  0.0026  0.0073  0.001   0.005   0.0017  0.0079 -0.0004  0.0022  0.0042  0.0063 -0.0054  0.009   0.0103  0.0148  0.0045]\nL3H4 [ 0.0085 -0.0063  0.0056  0.0029  0.0119  0.0101  0.0005  0.0069  0.0032  0.0047  0.0048  0.0083  0.0011 -0.0083  0.0081  0.0041  0.0103 -0.0038]\nL3H5 [ 0.006  -0.0089  0.0004  0.0027  0.0025  0.0713  0.0041  0.0423  0.0482  0.0683  0.0572  0.0198 -0.0084 -0.0599  0.1836  0.055  -0.0216 -0.022 ]\n\nL0 effective Q,K,V singular values (first 32):\nL0H0 Q [1.242 1.067 0.419 0.304 0.261 0.221 0.213 0.198 0.187 0.182 0.177 0.175 0.163 0.157 0.152 0.144 0.14  0.138 0.134 0.13  0.126 0.122 0.117 0.116\n 0.113 0.11  0.107 0.103 0.101 0.1   0.095 0.094]\nL0H0 K [0.661 0.456 0.27  0.246 0.23  0.223 0.212 0.21  0.198 0.196 0.19  0.188 0.182 0.179 0.173 0.171 0.164 0.158 0.153 0.153 0.147 0.144 0.142 0.141\n 0.137 0.133 0.128 0.127 0.125 0.124 0.122 0.12 ]\nL0H0 V [1.331 1.075 0.446 0.374 0.304 0.294 0.256 0.247 0.237 0.224 0.218 0.205 0.203 0.199 0.188 0.181 0.18  0.174 0.169 0.162 0.162 0.155 0.151 0.147\n 0.146 0.138 0.136 0.132 0.13  0.126 0.125 0.122]\nL0H1 Q [1.513 1.324 1.288 1.273 1.188 1.055 1.027 0.789 0.773 0.715 0.697 0.667 0.654 0.644 0.629 0.593 0.569 0.555 0.538 0.522 0.52  0.494 0.462 0.44\n 0.41  0.394 0.386 0.367 0.35  0.341 0.32  0.309]\nL0H1 K [3.78  2.409 1.606 1.4   1.309 1.203 1.148 1.007 0.879 0.855 0.831 0.809 0.747 0.722 0.71  0.655 0.639 0.614 0.58  0.569 0.534 0.504 0.484 0.474\n 0.447 0.427 0.381 0.353 0.344 0.324 0.32  0.31 ]\nL0H1 V [0.48  0.455 0.448 0.429 0.421 0.41  0.403 0.385 0.379 0.373 0.37  0.36  0.353 0.341 0.33  0.327 0.322 0.313 0.309 0.303 0.295 0.287 0.281 0.278\n 0.27  0.255 0.253 0.241 0.226 0.216 0.209 0.201]\nL0H2 Q [1.143 0.768 0.644 0.498 0.488 0.438 0.434 0.399 0.358 0.33  0.31  0.298 0.279 0.261 0.252 0.249 0.243 0.231 0.217 0.21  0.208 0.202 0.198 0.195\n 0.185 0.179 0.174 0.17  0.166 0.161 0.159 0.147]\nL0H2 K [1.187 0.914 0.55  0.529 0.506 0.489 0.424 0.397 0.389 0.37  0.35  0.333 0.321 0.307 0.298 0.292 0.28  0.268 0.258 0.249 0.235 0.228 0.22  0.214\n 0.212 0.2   0.192 0.19  0.187 0.177 0.173 0.166]\nL0H2 V [0.5   0.482 0.462 0.39"
}

========================================================================
TURN t004
MESSAGE SENT TO CODEX
OUTPUT of run 3 of at most 100:
```
INDUCTION ALGEBRA: trace(A_l C_earlier)/(||A_l||F ||C_earlier||F)
Positive values indicate a possible query-to-preceding-token key match, not measured attention.
columns ['L0H0', 'L0H1', 'L0H2', 'L0H3', 'L0H4', 'L0H5']
L1H0 [ 0.0065 -0.0001 -0.0079  0.005   0.0207 -0.0034]
L1H1 [-0.0046  0.007  -0.0161  0.0055  0.0191  0.0154]
L1H2 [ 0.0039  0.0007 -0.0168  0.0092  0.0326 -0.0027]
L1H3 [ 0.0077 -0.0026 -0.0132  0.0063  0.0421  0.0124]
L1H4 [ 0.0093  0.0022 -0.0181  0.0114  0.0439  0.0094]
L1H5 [-0.0195  0.0045 -0.0134 -0.0004 -0.0036  0.0128]
columns ['L0H0', 'L0H1', 'L0H2', 'L0H3', 'L0H4', 'L0H5', 'L1H0', 'L1H1', 'L1H2', 'L1H3', 'L1H4', 'L1H5']
L2H0 [ 0.0049 -0.0006 -0.003   0.0078  0.0099 -0.0046 -0.0126  0.0039 -0.0003  0.0016 -0.0004  0.0057]
L2H1 [0.0065 0.0014 0.0108 0.0025 0.0038 0.0776 0.0025 0.0479 0.0482 0.0575 0.0476 0.0351]
L2H2 [ 0.0114 -0.0165 -0.0054  0.0015  0.02   -0.0387 -0.0418 -0.0162 -0.0434 -0.007  -0.023  -0.0069]
L2H3 [ 0.0177 -0.0008 -0.0065  0.0066  0.0396 -0.0026 -0.0197  0.0049 -0.0162  0.0042 -0.0048  0.0044]
L2H4 [ 0.0079 -0.0055 -0.004   0.0101  0.0246  0.0018 -0.0139 -0.0082 -0.0115 -0.0019 -0.0042 -0.006 ]
L2H5 [0.009  0.0006 0.0001 0.0046 0.0188 0.0068 0.0001 0.0083 0.0095 0.0064 0.0068 0.0068]
columns ['L0H0', 'L0H1', 'L0H2', 'L0H3', 'L0H4', 'L0H5', 'L1H0', 'L1H1', 'L1H2', 'L1H3', 'L1H4', 'L1H5', 'L2H0', 'L2H1', 'L2H2', 'L2H3', 'L2H4', 'L2H5']
L3H0 [ 0.0098 -0.0036  0.0068  0.0077  0.0098  0.019   0.0079  0.0083  0.0101  0.006   0.006   0.0009 -0.0033 -0.0135  0.0238 -0.0008  0.0044 -0.0088]
L3H1 [ 0.0078  0.0071  0.006   0.0014  0.0041  0.061   0.0127  0.0333  0.038   0.0453  0.0456  0.0271  0.0153 -0.0181  0.1104  0.0209  0.0053 -0.0002]
L3H2 [ 0.0062 -0.0032  0.0015 -0.0008  0.013  -0.0012 -0.0116  0.0013 -0.0092  0.     -0.0089 -0.0036 -0.0046 -0.007  -0.0004 -0.0033 -0.0013 -0.0031]
L3H3 [ 0.0045  0.0032  0.0048  0.0026  0.0073  0.001   0.005   0.0017  0.0079 -0.0004  0.0022  0.0042  0.0063 -0.0054  0.009   0.0103  0.0148  0.0045]
L3H4 [ 0.0085 -0.0063  0.0056  0.0029  0.0119  0.0101  0.0005  0.0069  0.0032  0.0047  0.0048  0.0083  0.0011 -0.0083  0.0081  0.0041  0.0103 -0.0038]
L3H5 [ 0.006  -0.0089  0.0004  0.0027  0.0025  0.0713  0.0041  0.0423  0.0482  0.0683  0.0572  0.0198 -0.0084 -0.0599  0.1836  0.055  -0.0216 -0.022 ]

L0 effective Q,K,V singular values (first 32):
L0H0 Q [1.242 1.067 0.419 0.304 0.261 0.221 0.213 0.198 0.187 0.182 0.177 0.175 0.163 0.157 0.152 0.144 0.14  0.138 0.134 0.13  0.126 0.122 0.117 0.116
 0.113 0.11  0.107 0.103 0.101 0.1   0.095 0.094]
L0H0 K [0.661 0.456 0.27  0.246 0.23  0.223 0.212 0.21  0.198 0.196 0.19  0.188 0.182 0.179 0.173 0.171 0.164 0.158 0.153 0.153 0.147 0.144 0.142 0.141
 0.137 0.133 0.128 0.127 0.125 0.124 0.122 0.12 ]
L0H0 V [1.331 1.075 0.446 0.374 0.304 0.294 0.256 0.247 0.237 0.224 0.218 0.205 0.203 0.199 0.188 0.181 0.18  0.174 0.169 0.162 0.162 0.155 0.151 0.147
 0.146 0.138 0.136 0.132 0.13  0.126 0.125 0.122]
L0H1 Q [1.513 1.324 1.288 1.273 1.188 1.055 1.027 0.789 0.773 0.715 0.697 0.667 0.654 0.644 0.629 0.593 0.569 0.555 0.538 0.522 0.52  0.494 0.462 0.44
 0.41  0.394 0.386 0.367 0.35  0.341 0.32  0.309]
L0H1 K [3.78  2.409 1.606 1.4   1.309 1.203 1.148 1.007 0.879 0.855 0.831 0.809 0.747 0.722 0.71  0.655 0.639 0.614 0.58  0.569 0.534 0.504 0.484 0.474
 0.447 0.427 0.381 0.353 0.344 0.324 0.32  0.31 ]
L0H1 V [0.48  0.455 0.448 0.429 0.421 0.41  0.403 0.385 0.379 0.373 0.37  0.36  0.353 0.341 0.33  0.327 0.322 0.313 0.309 0.303 0.295 0.287 0.281 0.278
 0.27  0.255 0.253 0.241 0.226 0.216 0.209 0.201]
L0H2 Q [1.143 0.768 0.644 0.498 0.488 0.438 0.434 0.399 0.358 0.33  0.31  0.298 0.279 0.261 0.252 0.249 0.243 0.231 0.217 0.21  0.208 0.202 0.198 0.195
 0.185 0.179 0.174 0.17  0.166 0.161 0.159 0.147]
L0H2 K [1.187 0.914 0.55  0.529 0.506 0.489 0.424 0.397 0.389 0.37  0.35  0.333 0.321 0.307 0.298 0.292 0.28  0.268 0.258 0.249 0.235 0.228 0.22  0.214
 0.212 0.2   0.192 0.19  0.187 0.177 0.173 0.166]
L0H2 V [0.5   0.482 0.462 0.399 0.361 0.337 0.296 0.283 0.268 0.261 0.245 0.223 0.217 0.21  0.199 0.19  0.182 0.179 0.177 0.17  0.167 0.158 0.156 0.153
 0.15  0.147 0.145 0.137 0.135 0.129 0.127 0.124]
L0H3 Q [1.547 1.125 0.98  0.628 0.571 0.521 0.409 0.372 0.364 0.343 0.299 0.29  0.282 0.279 0.275 0.255 0.243 0.226 0.212 0.198 0.198 0.194 0.184 0.171
 0.162 0.154 0.15  0.148 0.144 0.143 0.141 0.139]
L0H3 K [1.093 0.949 0.68  0.542 0.435 0.406 0.399 0.365 0.34  0.331 0.319 0.299 0.278 0.26  0.249 0.239 0.227 0.223 0.22  0.203 0.191 0.184 0.18  0.175
 0.173 0.166 0.161 0.156 0.156 0.154 0.147 0.144]
L0H3 V [0.588 0.386 0.342 0.308 0.294 0.277 0.273 0.246 0.242 0.24  0.23  0.216 0.209 0.206 0.195 0.192 0.187 0.178 0.175 0.172 0.168 0.158 0.153 0.151
 0.149 0.143 0.143 0.141 0.138 0.13  0.128 0.125]
L0H4 Q [0.205 0.143 0.139 0.129 0.126 0.121 0.119 0.114 0.112 0.111 0.103 0.101 0.1   0.098 0.097 0.094 0.091 0.088 0.088 0.085 0.084 0.082 0.079 0.077
 0.077 0.075 0.073 0.071 0.068 0.067 0.067 0.066]
L0H4 K [0.214 0.193 0.186 0.178 0.173 0.167 0.156 0.154 0.148 0.146 0.142 0.137 0.135 0.129 0.128 0.123 0.12  0.118 0.116 0.113 0.111 0.107 0.106 0.103
 0.099 0.099 0.097 0.093 0.092 0.09  0.088 0.086]
L0H4 V [2.898 2.029 1.104 0.576 0.48  0.427 0.379 0.362 0.315 0.31  0.3   0.274 0.264 0.262 0.245 0.235 0.218 0.207 0.202 0.194 0.187 0.183 0.177 0.171
 0.167 0.164 0.161 0.158 0.15  0.144 0.143 0.142]
L0H5 Q [1.396 1.368 1.336 1.315 1.25  1.232 1.208 1.19  1.145 1.132 1.095 1.079 1.066 1.063 1.015 1.011 0.984 0.951 0.912 0.855 0.797 0.777 0.704 0.622
 0.608 0.588 0.525 0.46  0.445 0.429 0.406 0.397]
L0H5 K [3.314 2.634 1.802 1.724 1.638 1.528 1.499 1.455 1.396 1.35  1.273 1.21  1.2   1.167 1.063 1.045 1.005 0.935 0.905 0.887 0.827 0.736 0.643 0.62
 0.596 0.561 0.501 0.439 0.431 0.4   0.375 0.358]
L0H5 V [0.493 0.473 0.446 0.437 0.426 0.408 0.404 0.393 0.386 0.376 0.363 0.361 0.356 0.345 0.333 0.329 0.322 0.316 0.31  0.304 0.299 0.289 0.285 0.277
 0.274 0.267 0.252 0.245 0.241 0.235 0.233 0.221]

Average squared overlap of leading 24-dimensional L0 subspaces:
['Q1', 'K1', 'Q5', 'K5', 'V1', 'V5']
[[1.    0.607 0.77  0.655 0.103 0.146]
 [0.607 1.    0.662 0.776 0.122 0.178]
 [0.77  0.662 1.    0.814 0.081 0.137]
 [0.655 0.776 0.814 1.    0.08  0.153]
 [0.103 0.122 0.081 0.08  1.    0.619]
 [0.146 0.178 0.137 0.153 0.619 1.   ]]

Layer 0 QK product correlations, centered effective matrices:
power 1 cosine with H1 0.4137 cosine with H1 transpose -0.0104
power 2 cosine with H1 0.5461 cosine with H1 transpose -0.4342
power 3 cosine with H1 -0.0482 cosine with H1 transpose -0.4405
power 4 cosine with H1 -0.6081 cosine with H1 transpose 0.0147
power 5 cosine with H1 -0.6339 cosine with H1 transpose 0.4624
power 6 cosine with H1 -0.1532 cosine with H1 transpose 0.5474

Top-key and top-query directions of each head: overlap with L0H4 output subspace (top 4).
L0H0 Q [0.343 0.245 0.035 0.212] K [0.162 0.062 0.076 0.03 ]
L0H1 Q [0.008 0.139 0.036 0.011] K [0.101 0.029 0.016 0.018]
L0H2 Q [0.183 0.341 0.034 0.185] K [0.023 0.014 0.007 0.08 ]
L0H3 Q [0.311 0.485 0.183 0.19 ] K [0.129 0.02  0.023 0.069]
L0H4 Q [0.462 0.17  0.121 0.039] K [0.112 0.081 0.038 0.008]
L0H5 Q [0.033 0.004 0.011 0.017] K [0.082 0.067 0.031 0.025]
L1H0 Q [0.049 0.073 0.061 0.071] K [0.809 0.62  0.119 0.016]
L1H1 Q [0.162 0.056 0.016 0.024] K [0.913 0.78  0.048 0.041]
L1H2 Q [0.206 0.06  0.068 0.006] K [0.885 0.802 0.074 0.016]
L1H3 Q [0.236 0.1   0.067 0.028] K [0.9   0.797 0.058 0.016]
L1H4 Q [0.154 0.08  0.049 0.103] K [0.887 0.818 0.065 0.01 ]
L1H5 Q [0.069 0.076 0.122 0.017] K [0.852 0.764 0.077 0.054]
L2H0 Q [0.01  0.007 0.002 0.033] K [0.563 0.12  0.01  0.011]
L2H1 Q [0.042 0.006 0.007 0.004] K [0.032 0.012 0.005 0.013]
L2H2 Q [0.002 0.003 0.006 0.01 ] K [0.88  0.361 0.009 0.243]
L2H3 Q [0.001 0.005 0.029 0.082] K [0.823 0.649 0.032 0.006]
L2H4 Q [0.003 0.042 0.015 0.009] K [0.762 0.041 0.532 0.099]
L2H5 Q [0.017 0.002 0.016 0.021] K [0.413 0.009 0.179 0.018]
L3H0 Q [0.014 0.003 0.002 0.001] K [0.192 0.023 0.019 0.027]
L3H1 Q [0.017 0.015 0.002 0.021] K [0.02  0.002 0.008 0.003]
L3H2 Q [0.013 0.003 0.003 0.019] K [0.488 0.095 0.004 0.001]
L3H3 Q [0.016 0.002 0.003 0.   ] K [0.065 0.01  0.032 0.029]
L3H4 Q [0.014 0.009 0.004 0.002] K [0.336 0.04  0.002 0.003]
L3H5 Q [0.003 0.002 0.019 0.002] K [0.014 0.014 0.015 0.002]

```
CLI OUTPUT IN EVENT ORDER
ERROR
Code Mode is unavailable because code-mode host is disabled. Code mode will fail closed; enable `features.code_mode_host` and install `codex-code-mode-host`.

AGENT_MESSAGE
```python
import numpy as np
from scipy.linalg import svd
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score, adjusted_mutual_info_score
from threadpoolctl import threadpool_limits
threadpool_limits(4)

g1 = W["ln1_w.0"].astype(float)
g2 = W["ln2_w.0"].astype(float)
v1 = W["W_v.0"][64:128].astype(float)
v5 = W["W_v.0"][320:384].astype(float)
o5 = W["W_o.0"][:,320:384].astype(float)
fc = W["W_fc.0"].astype(float)*g2

for j in range(6):
    s = np.linalg.svd(W["W_v.0"][64*j:64*(j+1)].astype(float),compute_uv=False)
    print("L0H%d raw V spectrum:"%j,np.round(s[:40],3))

# These are weight subspaces and products, not token representations or activations.
_,s,vt = svd(np.concatenate([v1,v5]),full_matrices=False)
print("Joint raw V1,V5 spectrum:",np.round(s[:50],3))
B = vt[:28].T / g1[:,None]
B -= B.mean(axis=0,keepdims=True)
B,_ = np.linalg.qr(B)
C5 = o5 @ (v5*g1)
C5 -= C5.mean(axis=0,keepdims=True)

current = fc@B
preceding = fc@C5@B
def whiten_columns(x, rank=27):
    x=x-x.mean(axis=0,keepdims=True)
    u,s,vt=svd(x,full_matrices=False)
    return u[:,:rank]*np.sqrt(x.shape[0]),s

labels=[]
for title,x in [("current-weight coefficients",current),("preceding-weight coefficients",preceding)]:
    z,s=whiten_columns(x)
    print("\n",title,"spectrum",np.round(s,3))
    zn=z/(np.linalg.norm(z,axis=1,keepdims=True)+1e-12)
    for nc in [5,10,20,28,35,50]:
        km=KMeans(n_clusters=nc,n_init=10,random_state=0).fit(zn)
        sil=silhouette_score(zn,km.labels_,sample_size=1000,random_state=0)
        centers=km.cluster_centers_
        cos=np.sum(zn*centers[km.labels_],axis=1)/(np.linalg.norm(centers[km.labels_],axis=1)+1e-12)
        print("K",nc,"silhouette",round(sil,4),"median member-center cosine",round(np.median(cos),4),
              "sizes",np.sort(np.bincount(km.labels_))[::-1])
        if nc==28:
            labels.append(km.labels_)
            near=(centers/np.linalg.norm(centers,axis=1,keepdims=True))@(
                centers/np.linalg.norm(centers,axis=1,keepdims=True)).T
            pairs=sorted([(near[a,b],a,b) for a in range(nc) for b in range(a)],reverse=True)
            print("closest cluster pairs:",[(round(c,3),a,b) for c,a,b in pairs[:15]])
print("\nMutual information between the two 28-way weight clusterings:",
      adjusted_mutual_info_score(labels[0],labels[1]))
print("Cross-cluster count matrix:")
table=np.zeros((28,28),dtype=int)
np.add.at(table,(labels[0],labels[1]),1)
for row in table:
    print(" ".join(map(str,row)))
```


PERMITTED EXPERIMENT HARNESS RECORD: python
{
  "t": 1792713464.0828862,
  "s": 11.67,
  "code": "import numpy as np\nfrom scipy.linalg import svd\nfrom sklearn.cluster import KMeans\nfrom sklearn.metrics import silhouette_score, adjusted_mutual_info_score\nfrom threadpoolctl import threadpool_limits\nthreadpool_limits(4)\n\ng1 = W[\"ln1_w.0\"].astype(float)\ng2 = W[\"ln2_w.0\"].astype(float)\nv1 = W[\"W_v.0\"][64:128].astype(float)\nv5 = W[\"W_v.0\"][320:384].astype(float)\no5 = W[\"W_o.0\"][:,320:384].astype(float)\nfc = W[\"W_fc.0\"].astype(float)*g2\n\nfor j in range(6):\n    s = np.linalg.svd(W[\"W_v.0\"][64*j:64*(j+1)].astype(float),compute_uv=False)\n    print(\"L0H%d raw V spectrum:\"%j,np.round(s[:40],3))\n\n# These are weight subspaces and products, not token representations or activations.\n_,s,vt = svd(np.concatenate([v1,v5]),full_matrices=False)\nprint(\"Joint raw V1,V5 spectrum:\",np.round(s[:50],3))\nB = vt[:28].T / g1[:,None]\nB -= B.mean(axis=0,keepdims=True)\nB,_ = np.linalg.qr(B)\nC5 = o5 @ (v5*g1)\nC5 -= C5.mean(axis=0,keepdims=True)\n\ncurrent = fc@B\npreceding = fc@C5@B\ndef whiten_columns(x, rank=27):\n    x=x-x.mean(axis=0,keepdims=True)\n    u,s,vt=svd(x,full_matrices=False)\n    return u[:,:rank]*np.sqrt(x.shape[0]),s\n\nlabels=[]\nfor title,x in [(\"current-weight coefficients\",current),(\"preceding-weight coefficients\",preceding)]:\n    z,s=whiten_columns(x)\n    print(\"\\n\",title,\"spectrum\",np.round(s,3))\n    zn=z/(np.linalg.norm(z,axis=1,keepdims=True)+1e-12)\n    for nc in [5,10,20,28,35,50]:\n        km=KMeans(n_clusters=nc,n_init=10,random_state=0).fit(zn)\n        sil=silhouette_score(zn,km.labels_,sample_size=1000,random_state=0)\n        centers=km.cluster_centers_\n        cos=np.sum(zn*centers[km.labels_],axis=1)/(np.linalg.norm(centers[km.labels_],axis=1)+1e-12)\n        print(\"K\",nc,\"silhouette\",round(sil,4),\"median member-center cosine\",round(np.median(cos),4),\n              \"sizes\",np.sort(np.bincount(km.labels_))[::-1])\n        if nc==28:\n            labels.append(km.labels_)\n            near=(centers/np.linalg.norm(centers,axis=1,keepdims=True))@(\n                centers/np.linalg.norm(centers,axis=1,keepdims=True)).T\n            pairs=sorted([(near[a,b],a,b) for a in range(nc) for b in range(a)],reverse=True)\n            print(\"closest cluster pairs:\",[(round(c,3),a,b) for c,a,b in pairs[:15]])\nprint(\"\\nMutual information between the two 28-way weight clusterings:\",\n      adjusted_mutual_info_score(labels[0],labels[1]))\nprint(\"Cross-cluster count matrix:\")\ntable=np.zeros((28,28),dtype=int)\nnp.add.at(table,(labels[0],labels[1]),1)\nfor row in table:\n    print(\" \".join(map(str,row)))\n",
  "out": "L0H0 raw V spectrum: [2.214 2.066 1.611 1.282 1.259 1.034 0.964 0.915 0.843 0.762 0.716 0.68  0.652 0.649 0.609 0.606 0.593 0.551 0.54  0.516 0.51  0.498 0.489 0.473\n 0.455 0.451 0.447 0.434 0.427 0.413 0.408 0.404 0.398 0.396 0.386 0.378 0.376 0.372 0.367 0.362]\nL0H1 raw V spectrum: [2.25  2.127 2.089 2.055 1.991 1.973 1.958 1.933 1.869 1.832 1.808 1.785 1.717 1.69  1.639 1.611 1.584 1.554 1.496 1.48  1.419 1.37  1.277 1.193\n 1.14  1.094 1.028 0.906 0.744 0.693 0.682 0.659 0.638 0.587 0.538 0.527 0.495 0.482 0.469 0.452]\nL0H2 raw V spectrum: [2.164 2.047 2.012 1.723 1.514 1.361 1.273 1.115 1.036 0.993 0.908 0.857 0.765 0.74  0.681 0.628 0.618 0.592 0.58  0.57  0.535 0.526 0.512 0.477\n 0.471 0.46  0.458 0.445 0.444 0.433 0.423 0.409 0.408 0.401 0.397 0.392 0.381 0.374 0.366 0.361]\nL0H3 raw V spectrum: [1.443 1.28  1.237 1.154 1.129 1.076 1.038 1.013 0.958 0.917 0.897 0.841 0.816 0.734 0.696 0.67  0.64  0.604 0.581 0.568 0.529 0.519 0.513 0.475\n 0.471 0.464 0.457 0.446 0.432 0.417 0.414 0.404 0.395 0.391 0.386 0.379 0.372 0.368 0.364 0.36 ]\nL0H4 raw V spectrum: [4.102 3.344 2.539 2.115 1.785 1.548 1.482 1.422 1.387 1.323 1.196 1.164 1.145 1.022 0.967 0.848 0.832 0.801 0.7   0.665 0.644 0.637 0.616 0.566\n 0.56  0.535 0.529 0.494 0.474 0.464 0.447 0.445 0.433 0.427 0.413 0.408 0.403 0.399 0.396 0.383]\nL0H5 raw V spectrum: [2.397 2.282 2.191 2.148 2.082 2.049 2.033 1.995 1.901 1.88  1.86  1.788 1.772 1.744 1.703 1.646 1.591 1.538 1.448 1.384 1.317 1.2   1.042 0.923\n 0.892 0.853 0.817 0.784 0.767 0.744 0.737 0.722 0.709 0.681 0.663 0.654 0.627 0.599 0.579 0.571]\nJoint raw V1,V5 spectrum: [3.023 2.923 2.898 2.843 2.791 2.735 2.708 2.661 2.614 2.601 2.519 2.44  2.421 2.379 2.364 2.311 2.266 2.214 2.194 2.105 2.019 1.897 1.838 1.726\n 1.497 1.42  1.348 1.111 1.    0.935 0.918 0.89  0.865 0.854 0.821 0.81  0.793 0.779 0.76  0.725 0.721 0.702 0.693 0.676 0.64  0.619 0.601 0.546\n 0.54  0.534]\n\n current-weight coefficients spectrum [7.666 6.317 5.921 5.691 5.479 5.437 5.174 5.037 4.95  4.823 4.765 4.693 4.565 4.495 4.352 4.329 4.306 4.22  4.114 4.067 3.962 3.751 3.637 3.509\n 3.418 3.402 3.245 1.936]\nK 5 silhouette 0.035 median member-center cosine 0.3996 sizes [384 334 322 304 192]\nK 10 silhouette 0.0437 median member-center cosine 0.472 sizes [226 204 191 169 167 147 145 136  77  74]\nK 20 silhouette 0.0622 median member-center cosine 0.5421 sizes [126 123 120 110 104  95  91  83  74  68  67  62  62  61  61  53  49  46  44  37]\nK 28 silhouette 0.0729 median member-center cosine 0.5757 sizes [113 112  98  83  81  78  70  69  65  62  52  50  49  48  45  45  41  40  40  39  36  35  35  35  33  32  26  24]\nclosest cluster pairs: [(np.float64(0.663), 15, 2), (np.float64(0.658), 15, 11), (np.float64(0.61), 15, 5), (np.float64(0.573), 5, 2), (np.float64(0.556), 11, 2), (np.float64(0.55), 10, 5), (np.float64(0.549), 26, 11), (np.float64(0.538), 26, 0), (np.float64(0.531), 24, 11), (np.float64(0.53), 24, 0), (np.float64(0.528), 11, 5), (np.float64(0.527), 26, 2), (np.float64(0.523), 5, 0), (np.float64(0.517), 11, 0), (np.float64(0.514), 15, 10)]\nK 35 silhouette 0.0739 median member-center cosine 0.5933 sizes [85 72 71 70 67 67 64 59 57 54 54 51 45 43 43 42 41 41 40 38 37 36 35 35 34 32 30 29 28 26 25 23 23 20 19]\nK 50 silhouette 0.0806 median member-center cosine 0.6265 sizes [51 48 44 44 44 43 41 40 39 38 37 37 37 36 36 35 34 34 34 34 34 33 33 33 32 32 32 30 29 27 26 26 25 25 25 25 24 24 24 24 23 23 23 22 21 19 17 16 15\n  8]\n\n preceding-weight coefficients spectrum [3.11  2.396 2.107 2.023 1.8   1.72  1.566 1.525 1.419 1.295 1.23  1.094 1.01  0.981 0.933 0.885 0.828 0.722 0.574 0.419 0.344 0.309 0.238 0.165\n 0.111 0.095 0.073 0.059]\nK 5 silhouette 0.0428 median member-center cosine 0.4265 sizes [460 294 277 258 247]\nK 10 silhouette 0.034 median member-center cosine 0.4962 sizes [274 171 163 144 141 135 133 130 125 120]\nK 20 silhouette 0.0348 median member-center cosine 0.5515 sizes [97 96 91 88 85 85 84 82 81 77 73 72 72 71 69 69 68 63 62 51]\nK 28"
}

========================================================================
TURN t005
MESSAGE SENT TO CODEX
OUTPUT of run 4 of at most 100:
```
L0H0 raw V spectrum: [2.214 2.066 1.611 1.282 1.259 1.034 0.964 0.915 0.843 0.762 0.716 0.68  0.652 0.649 0.609 0.606 0.593 0.551 0.54  0.516 0.51  0.498 0.489 0.473
 0.455 0.451 0.447 0.434 0.427 0.413 0.408 0.404 0.398 0.396 0.386 0.378 0.376 0.372 0.367 0.362]
L0H1 raw V spectrum: [2.25  2.127 2.089 2.055 1.991 1.973 1.958 1.933 1.869 1.832 1.808 1.785 1.717 1.69  1.639 1.611 1.584 1.554 1.496 1.48  1.419 1.37  1.277 1.193
 1.14  1.094 1.028 0.906 0.744 0.693 0.682 0.659 0.638 0.587 0.538 0.527 0.495 0.482 0.469 0.452]
L0H2 raw V spectrum: [2.164 2.047 2.012 1.723 1.514 1.361 1.273 1.115 1.036 0.993 0.908 0.857 0.765 0.74  0.681 0.628 0.618 0.592 0.58  0.57  0.535 0.526 0.512 0.477
 0.471 0.46  0.458 0.445 0.444 0.433 0.423 0.409 0.408 0.401 0.397 0.392 0.381 0.374 0.366 0.361]
L0H3 raw V spectrum: [1.443 1.28  1.237 1.154 1.129 1.076 1.038 1.013 0.958 0.917 0.897 0.841 0.816 0.734 0.696 0.67  0.64  0.604 0.581 0.568 0.529 0.519 0.513 0.475
 0.471 0.464 0.457 0.446 0.432 0.417 0.414 0.404 0.395 0.391 0.386 0.379 0.372 0.368 0.364 0.36 ]
L0H4 raw V spectrum: [4.102 3.344 2.539 2.115 1.785 1.548 1.482 1.422 1.387 1.323 1.196 1.164 1.145 1.022 0.967 0.848 0.832 0.801 0.7   0.665 0.644 0.637 0.616 0.566
 0.56  0.535 0.529 0.494 0.474 0.464 0.447 0.445 0.433 0.427 0.413 0.408 0.403 0.399 0.396 0.383]
L0H5 raw V spectrum: [2.397 2.282 2.191 2.148 2.082 2.049 2.033 1.995 1.901 1.88  1.86  1.788 1.772 1.744 1.703 1.646 1.591 1.538 1.448 1.384 1.317 1.2   1.042 0.923
 0.892 0.853 0.817 0.784 0.767 0.744 0.737 0.722 0.709 0.681 0.663 0.654 0.627 0.599 0.579 0.571]
Joint raw V1,V5 spectrum: [3.023 2.923 2.898 2.843 2.791 2.735 2.708 2.661 2.614 2.601 2.519 2.44  2.421 2.379 2.364 2.311 2.266 2.214 2.194 2.105 2.019 1.897 1.838 1.726
 1.497 1.42  1.348 1.111 1.    0.935 0.918 0.89  0.865 0.854 0.821 0.81  0.793 0.779 0.76  0.725 0.721 0.702 0.693 0.676 0.64  0.619 0.601 0.546
 0.54  0.534]

 current-weight coefficients spectrum [7.666 6.317 5.921 5.691 5.479 5.437 5.174 5.037 4.95  4.823 4.765 4.693 4.565 4.495 4.352 4.329 4.306 4.22  4.114 4.067 3.962 3.751 3.637 3.509
 3.418 3.402 3.245 1.936]
K 5 silhouette 0.035 median member-center cosine 0.3996 sizes [384 334 322 304 192]
K 10 silhouette 0.0437 median member-center cosine 0.472 sizes [226 204 191 169 167 147 145 136  77  74]
K 20 silhouette 0.0622 median member-center cosine 0.5421 sizes [126 123 120 110 104  95  91  83  74  68  67  62  62  61  61  53  49  46  44  37]
K 28 silhouette 0.0729 median member-center cosine 0.5757 sizes [113 112  98  83  81  78  70  69  65  62  52  50  49  48  45  45  41  40  40  39  36  35  35  35  33  32  26  24]
closest cluster pairs: [(np.float64(0.663), 15, 2), (np.float64(0.658), 15, 11), (np.float64(0.61), 15, 5), (np.float64(0.573), 5, 2), (np.float64(0.556), 11, 2), (np.float64(0.55), 10, 5), (np.float64(0.549), 26, 11), (np.float64(0.538), 26, 0), (np.float64(0.531), 24, 11), (np.float64(0.53), 24, 0), (np.float64(0.528), 11, 5), (np.float64(0.527), 26, 2), (np.float64(0.523), 5, 0), (np.float64(0.517), 11, 0), (np.float64(0.514), 15, 10)]
K 35 silhouette 0.0739 median member-center cosine 0.5933 sizes [85 72 71 70 67 67 64 59 57 54 54 51 45 43 43 42 41 41 40 38 37 36 35 35 34 32 30 29 28 26 25 23 23 20 19]
K 50 silhouette 0.0806 median member-center cosine 0.6265 sizes [51 48 44 44 44 43 41 40 39 38 37 37 37 36 36 35 34 34 34 34 34 33 33 33 32 32 32 30 29 27 26 26 25 25 25 25 24 24 24 24 23 23 23 22 21 19 17 16 15
  8]

 preceding-weight coefficients spectrum [3.11  2.396 2.107 2.023 1.8   1.72  1.566 1.525 1.419 1.295 1.23  1.094 1.01  0.981 0.933 0.885 0.828 0.722 0.574 0.419 0.344 0.309 0.238 0.165
 0.111 0.095 0.073 0.059]
K 5 silhouette 0.0428 median member-center cosine 0.4265 sizes [460 294 277 258 247]
K 10 silhouette 0.034 median member-center cosine 0.4962 sizes [274 171 163 144 141 135 133 130 125 120]
K 20 silhouette 0.0348 median member-center cosine 0.5515 sizes [97 96 91 88 85 85 84 82 81 77 73 72 72 71 69 69 68 63 62 51]
K 28 silhouette 0.0382 median member-center cosine 0.5809 sizes [81 74 73 69 67 67 66 64 60 60 59 57 57 57 56 54 54 51 49 49 44 41 41 41 38 37 37 33]
closest cluster pairs: [(np.float64(0.661), 7, 1), (np.float64(0.625), 26, 25), (np.float64(0.61), 13, 5), (np.float64(0.595), 7, 2), (np.float64(0.592), 24, 15), (np.float64(0.592), 9, 4), (np.float64(0.582), 21, 13), (np.float64(0.568), 5, 4), (np.float64(0.565), 22, 14), (np.float64(0.564), 15, 9), (np.float64(0.562), 7, 6), (np.float64(0.561), 26, 15), (np.float64(0.546), 9, 1), (np.float64(0.541), 24, 5), (np.float64(0.522), 13, 3)]
K 35 silhouette 0.0363 median member-center cosine 0.5958 sizes [68 66 66 60 59 58 55 54 50 49 47 47 46 44 44 43 43 43 42 42 41 39 39 38 38 37 37 36 35 34 30 28 27 26 25]
K 50 silhouette 0.0397 median member-center cosine 0.6229 sizes [63 46 46 45 44 44 44 43 42 41 40 38 38 36 36 35 34 33 33 32 31 31 31 30 30 30 29 29 28 28 27 27 27 26 25 25 25 25 22 22 20 20 20 19 17 17 17 15 15
 15]

Mutual information between the two 28-way weight clusterings: 0.08046534892583179
Cross-cluster count matrix:
0 1 2 2 4 3 1 4 1 1 0 1 0 2 8 3 0 0 1 0 0 2 3 0 5 1 5 2
2 5 3 0 2 6 1 1 3 2 7 1 1 6 1 7 3 2 2 4 0 5 1 0 3 0 1 1
1 4 3 4 8 6 10 10 1 7 2 5 2 5 4 8 0 0 0 2 1 8 6 0 6 3 3 4
3 0 1 2 1 0 0 2 4 0 5 0 1 0 3 0 2 7 1 0 3 0 5 3 1 1 0 0
1 3 3 1 1 3 2 3 3 3 2 5 4 1 1 2 2 1 0 1 1 5 3 4 1 3 2 1
0 7 5 5 11 5 7 11 0 2 1 9 0 8 6 3 0 1 0 3 0 7 3 0 8 5 3 2
0 0 0 0 0 0 0 0 5 1 6 0 0 1 0 1 8 3 4 3 7 0 0 1 1 0 0 0
0 0 2 2 3 1 0 3 2 2 0 1 0 4 1 2 2 1 0 1 1 0 1 0 0 1 2 0
0 3 0 0 2 2 2 0 0 1 0 1 1 0 2 1 3 2 0 6 1 1 0 2 0 1 1 1
0 2 2 4 1 1 2 1 0 0 2 0 0 0 1 2 2 6 1 1 0 1 1 5 0 0 0 1
0 11 5 5 5 4 4 8 0 7 0 3 0 0 0 3 0 1 1 1 4 4 4 1 6 2 2 2
0 6 12 3 7 3 7 4 0 5 0 1 0 3 2 4 0 0 0 0 1 3 6 1 5 5 3 0
2 1 1 0 1 5 0 0 2 0 3 0 1 1 1 1 5 0 3 0 0 0 0 3 2 0 1 2
3 0 1 0 0 0 0 0 27 0 4 0 6 0 0 0 8 6 8 0 2 0 2 2 0 0 0 0
0 1 1 0 0 0 2 0 5 2 3 1 1 0 0 0 2 2 0 0 1 0 1 0 4 0 0 0
0 8 6 3 7 7 5 7 1 5 0 7 0 5 3 3 0 0 0 1 0 4 5 0 4 12 4 1
0 0 0 1 0 1 0 1 4 1 0 3 2 0 0 0 1 1 1 4 0 0 0 0 3 0 0 1
4 0 0 1 1 0 0 1 5 2 2 2 4 0 2 1 3 1 4 0 2 2 1 4 0 2 1 0
0 0 1 0 1 3 0 1 3 2 2 0 1 0 1 1 2 2 4 0 3 1 0 6 2 1 1 2
4 1 0 1 0 4 1 0 1 0 3 0 2 0 1 0 1 0 1 3 4 3 0 2 0 0 2 1
4 1 1 6 2 0 1 2 2 1 1 2 0 1 3 1 2 0 0 0 1 2 1 1 1 2 2 0
0 1 0 2 4 1 2 1 2 4 4 0 3 2 0 2 1 1 1 0 0 3 1 2 0 1 1 10
6 2 1 1 0 3 4 1 3 3 1 2 0 1 3 2 0 2 3 1 1 2 0 0 1 2 2 3
1 2 1 3 2 3 3 1 4 1 3 1 1 1 1 1 0 6 1 0 1 2 5 0 1 1 1 1
0 3 2 4 5 3 1 3 1 3 1 1 0 3 4 5 0 2 1 0 0 3 4 0 2 10 3 1
4 0 1 2 0 0 5 0 2 0 5 2 1 1 3 1 0 1 2 2 1 0 0 0 4 0 2 0
3 4 5 3 3 2 4 6 0 4 0 2 0 4 3 2 1 4 0 3 1 8 4 3 5 1 2 1
0 1 1 1 3 0 0 2 0 0 0 1 2 0 0 1 1 5 2 1 1 3 3 1 2 0 0 4

```
CLI OUTPUT IN EVENT ORDER
ERROR
Code Mode is unavailable because code-mode host is disabled. Code mode will fail closed; enable `features.code_mode_host` and install `codex-code-mode-host`.

AGENT_MESSAGE
```python
import numpy as np
from scipy.linalg import svd
from scipy.stats import skew, kurtosis
from sklearn.decomposition import FastICA
from sklearn.metrics import adjusted_mutual_info_score
from threadpoolctl import threadpool_limits
import warnings
warnings.filterwarnings("ignore")
threadpool_limits(4)

g1=W["ln1_w.0"].astype(float)
g2=W["ln2_w.0"].astype(float)
F=W["W_fc.0"].astype(float)*g2
vs=[W["W_v.0"][64*j:64*(j+1)].astype(float) for j in [1,5]]
_,s,vt=svd(np.concatenate(vs),full_matrices=False)
Cs=[]
for j in [5,1]:
    c=W["W_o.0"][:,64*j:64*(j+1)].astype(float)@(
        W["W_v.0"][64*j:64*(j+1)].astype(float)*g1)
    c-=c.mean(axis=0,keepdims=True)
    Cs.append(c)

for variant in ["divide_gain","raw","multiply_gain"]:
    B=vt[:28].T.copy()
    if variant=="divide_gain": B/=g1[:,None]
    if variant=="multiply_gain": B*=g1[:,None]
    B-=B.mean(axis=0,keepdims=True)
    B,_=np.linalg.qr(B)
    matrices=[F@B]+[F@c@B for c in Cs]
    matrices=[m/np.linalg.norm(m-m.mean(axis=0)) for m in matrices]
    X=np.concatenate(matrices)
    ica=FastICA(n_components=27,whiten="unit-variance",max_iter=3000,tol=1e-5,random_state=0)
    Z=ica.fit_transform(X)
    signs=np.sign(skew(Z,axis=0))
    signs[signs==0]=1
    Z*=signs
    ku=kurtosis(Z,axis=0)
    sk=skew(Z,axis=0)
    print("\nICA of weight coefficients:",variant)
    print("skew:",np.round(sk,2))
    print("excess kurtosis:",np.round(ku,2))
    for block in range(3):
        z=Z[1536*block:1536*(block+1)]
        top=np.argmax(z,axis=1)
        print(["current","H5-composed","H1-composed"][block],
              "top-component counts",np.bincount(top,minlength=27))
    if variant=="divide_gain":
        zz=[Z[1536*i:1536*(i+1)] for i in range(3)]
        # Cross-component co-occurrence among unusually large weight coefficients.
        # This is a decomposition of the parameters, not evaluation of network neurons.
        highs=[z>2.0 for z in zz]
        co=highs[0].T.astype(int)@highs[1].astype(int)
        print("Current vs H5-composed coefficient co-occurrences (>2 standardized units):")
        for a in range(27):
            best=np.argsort(-co[a])[:6]
            print(a,"current_count",int(highs[0][:,a].sum()),
                  "prev_count",int(highs[1][:,a].sum()),
                  "partners",[(int(b),int(co[a,b])) for b in best])

X=W["W_proj.3"].astype(float).T
ica=FastICA(n_components=28,whiten="unit-variance",max_iter=3000,tol=1e-5,random_state=0)
Z=ica.fit_transform(X)
print("\nLast-projection independent-component excess kurtosis:",
      np.round(np.sort(kurtosis(Z,axis=0))[::-1],2))
print("Last-projection independent-component absolute skew:",
      np.round(np.sort(np.abs(skew(Z,axis=0)))[::-1],2))
```


PERMITTED EXPERIMENT HARNESS RECORD: python
{
  "t": 1792713572.03931,
  "s": 15.57,
  "code": "import numpy as np\nfrom scipy.linalg import svd\nfrom scipy.stats import skew, kurtosis\nfrom sklearn.decomposition import FastICA\nfrom sklearn.metrics import adjusted_mutual_info_score\nfrom threadpoolctl import threadpool_limits\nimport warnings\nwarnings.filterwarnings(\"ignore\")\nthreadpool_limits(4)\n\ng1=W[\"ln1_w.0\"].astype(float)\ng2=W[\"ln2_w.0\"].astype(float)\nF=W[\"W_fc.0\"].astype(float)*g2\nvs=[W[\"W_v.0\"][64*j:64*(j+1)].astype(float) for j in [1,5]]\n_,s,vt=svd(np.concatenate(vs),full_matrices=False)\nCs=[]\nfor j in [5,1]:\n    c=W[\"W_o.0\"][:,64*j:64*(j+1)].astype(float)@(\n        W[\"W_v.0\"][64*j:64*(j+1)].astype(float)*g1)\n    c-=c.mean(axis=0,keepdims=True)\n    Cs.append(c)\n\nfor variant in [\"divide_gain\",\"raw\",\"multiply_gain\"]:\n    B=vt[:28].T.copy()\n    if variant==\"divide_gain\": B/=g1[:,None]\n    if variant==\"multiply_gain\": B*=g1[:,None]\n    B-=B.mean(axis=0,keepdims=True)\n    B,_=np.linalg.qr(B)\n    matrices=[F@B]+[F@c@B for c in Cs]\n    matrices=[m/np.linalg.norm(m-m.mean(axis=0)) for m in matrices]\n    X=np.concatenate(matrices)\n    ica=FastICA(n_components=27,whiten=\"unit-variance\",max_iter=3000,tol=1e-5,random_state=0)\n    Z=ica.fit_transform(X)\n    signs=np.sign(skew(Z,axis=0))\n    signs[signs==0]=1\n    Z*=signs\n    ku=kurtosis(Z,axis=0)\n    sk=skew(Z,axis=0)\n    print(\"\\nICA of weight coefficients:\",variant)\n    print(\"skew:\",np.round(sk,2))\n    print(\"excess kurtosis:\",np.round(ku,2))\n    for block in range(3):\n        z=Z[1536*block:1536*(block+1)]\n        top=np.argmax(z,axis=1)\n        print([\"current\",\"H5-composed\",\"H1-composed\"][block],\n              \"top-component counts\",np.bincount(top,minlength=27))\n    if variant==\"divide_gain\":\n        zz=[Z[1536*i:1536*(i+1)] for i in range(3)]\n        # Cross-component co-occurrence among unusually large weight coefficients.\n        # This is a decomposition of the parameters, not evaluation of network neurons.\n        highs=[z>2.0 for z in zz]\n        co=highs[0].T.astype(int)@highs[1].astype(int)\n        print(\"Current vs H5-composed coefficient co-occurrences (>2 standardized units):\")\n        for a in range(27):\n            best=np.argsort(-co[a])[:6]\n            print(a,\"current_count\",int(highs[0][:,a].sum()),\n                  \"prev_count\",int(highs[1][:,a].sum()),\n                  \"partners\",[(int(b),int(co[a,b])) for b in best])\n\nX=W[\"W_proj.3\"].astype(float).T\nica=FastICA(n_components=28,whiten=\"unit-variance\",max_iter=3000,tol=1e-5,random_state=0)\nZ=ica.fit_transform(X)\nprint(\"\\nLast-projection independent-component excess kurtosis:\",\n      np.round(np.sort(kurtosis(Z,axis=0))[::-1],2))\nprint(\"Last-projection independent-component absolute skew:\",\n      np.round(np.sort(np.abs(skew(Z,axis=0)))[::-1],2))\n",
  "out": "\nICA of weight coefficients: divide_gain\nskew: [ 1.07  2.26  3.46  7.45  7.53  4.99  7.08  2.3   0.37  1.79  0.16  4.18  2.04  1.25  2.47  4.    3.19 13.9  17.2   4.67  2.86  3.45  7.64  5.51\n  3.41  8.99  4.13]\nexcess kurtosis: [ 19.14  18.78  40.98 152.61 156.34  60.8  108.55  23.14  19.78  23.17  63.52  49.54  26.11  19.83  30.16  66.25  38.33 358.61 593.24  57.16  36.27\n  36.89 131.61  99.18  31.39 196.26  65.4 ]\ncurrent top-component counts [ 49  25  39  65 177  44  45  53  52  48  35  54  29   8  54  44  53  85  81  50  52  43  56 106  51  68  70]\nH5-composed top-component counts [ 55 121  67  67  43  53   8  31  65 105  35  18 101 119  63  37  49   0  11  65  91  61   7  99  52   6 107]\nH1-composed top-component counts [ 13  49 108  92   6  55  89  16  98  30  29  75  88  11  21 109  42  27  46  59  75  34 156  37  37  45  89]\nCurrent vs H5-composed coefficient co-occurrences (>2 standardized units):\n0 current_count 68 prev_count 45 partners [(0, 17), (13, 16), (1, 14), (16, 11), (7, 10), (9, 9)]\n1 current_count 29 prev_count 73 partners [(1, 9), (13, 9), (11, 7), (14, 5), (16, 5), (21, 4)]\n2 current_count 47 prev_count 36 partners [(2, 12), (7, 11), (13, 11), (9, 11), (20, 7), (0, 6)]\n3 current_count 36 prev_count 32 partners [(3, 8), (1, 7), (24, 7), (13, 6), (25, 5), (7, 5)]\n4 current_count 45 prev_count 17 partners [(0, 9), (2, 8), (21, 8), (7, 7), (13, 7), (9, 7)]\n5 current_count 33 prev_count 44 partners [(1, 8), (5, 8), (21, 8), (9, 7), (14, 5), (13, 5)]\n6 current_count 37 prev_count 9 partners [(24, 5), (15, 4), (20, 4), (3, 3), (13, 3), (16, 3)]\n7 current_count 68 prev_count 62 partners [(7, 15), (14, 11), (13, 11), (0, 10), (1, 10), (9, 10)]\n8 current_count 36 prev_count 30 partners [(1, 10), (13, 8), (7, 5), (5, 5), (9, 5), (8, 4)]\n9 current_count 34 prev_count 63 partners [(9, 9), (1, 8), (13, 7), (7, 6), (16, 6), (11, 5)]\n10 current_count 33 prev_count 15 partners [(24, 6), (5, 5), (14, 4), (3, 3), (12, 3), (21, 3)]\n11 current_count 32 prev_count 35 partners [(9, 7), (16, 6), (11, 5), (7, 5), (5, 4), (12, 4)]\n12 current_count 24 prev_count 39 partners [(19, 6), (9, 5), (7, 3), (1, 3), (12, 3), (13, 3)]\n13 current_count 19 prev_count 85 partners [(13, 4), (26, 3), (9, 2), (1, 2), (11, 2), (5, 2)]\n14 current_count 41 prev_count 55 partners [(13, 12), (1, 10), (9, 8), (11, 8), (14, 7), (5, 7)]\n15 current_count 15 prev_count 17 partners [(15, 6), (23, 3), (21, 3), (7, 2), (20, 2), (19, 2)]\n16 current_count 33 prev_count 52 partners [(14, 7), (1, 6), (13, 6), (7, 6), (19, 6), (16, 6)]\n17 current_count 36 prev_count 4 partners [(13, 9), (9, 8), (1, 7), (3, 7), (0, 6), (7, 6)]\n18 current_count 28 prev_count 5 partners [(13, 7), (24, 6), (15, 5), (9, 5), (16, 5), (21, 5)]\n19 current_count 33 prev_count 41 partners [(19, 12), (13, 7), (12, 5), (1, 5), (21, 5), (23, 5)]\n20 current_count 36 prev_count 39 partners [(20, 8), (1, 7), (21, 5), (0, 5), (13, 4), (25, 4)]\n21 current_count 46 prev_count 43 partners [(21, 10), (24, 9), (14, 8), (9, 7), (7, 6), (13, 6)]\n22 current_count 42 prev_count 6 partners [(1, 7), (11, 7), (13, 7), (16, 7), (5, 6), (7, 5)]\n23 current_count 32 prev_count 32 partners [(1, 11), (23, 7), (16, 6), (21, 6), (24, 5), (5, 5)]\n24 current_count 47 prev_count 55 partners [(13, 14), (24, 14), (9, 13), (11, 12), (19, 8), (8, 8)]\n25 current_count 51 prev_count 16 partners [(13, 11), (14, 7), (21, 7), (0, 6), (3, 6), (9, 6)]\n26 current_count 26 prev_count 36 partners [(13, 8), (1, 6), (20, 6), (21, 6), (26, 5), (19, 4)]\n\nICA of weight coefficients: raw\nskew: [ 9.71  0.74  7.3   4.13  3.9  15.11  7.59  1.94  1.81  4.9   1.43  2.51  3.33  1.39  3.73  4.33 13.32  6.8   3.07  4.56  7.31  3.14  0.23  2.38\n  5.16  3.61  5.36]\nexcess kurtosis: [215.66  61.35 122.53  67.02  43.06 497.03 160.33  23.93  27.35  60.2   22.75  21.94  32.17  20.03  41.58  49.07 345.56 135.11  33.3   72.84 111.2\n  41.01  21.63  26.78  67.45  42.45  96.53]\ncurrent top-component counts [ 49  41  36  50  42  91 155  28  50  57  93  29  4"
}

========================================================================
TURN t006
MESSAGE SENT TO CODEX
OUTPUT of run 5 of at most 100:
```

ICA of weight coefficients: divide_gain
skew: [ 1.07  2.26  3.46  7.45  7.53  4.99  7.08  2.3   0.37  1.79  0.16  4.18  2.04  1.25  2.47  4.    3.19 13.9  17.2   4.67  2.86  3.45  7.64  5.51
  3.41  8.99  4.13]
excess kurtosis: [ 19.14  18.78  40.98 152.61 156.34  60.8  108.55  23.14  19.78  23.17  63.52  49.54  26.11  19.83  30.16  66.25  38.33 358.61 593.24  57.16  36.27
  36.89 131.61  99.18  31.39 196.26  65.4 ]
current top-component counts [ 49  25  39  65 177  44  45  53  52  48  35  54  29   8  54  44  53  85  81  50  52  43  56 106  51  68  70]
H5-composed top-component counts [ 55 121  67  67  43  53   8  31  65 105  35  18 101 119  63  37  49   0  11  65  91  61   7  99  52   6 107]
H1-composed top-component counts [ 13  49 108  92   6  55  89  16  98  30  29  75  88  11  21 109  42  27  46  59  75  34 156  37  37  45  89]
Current vs H5-composed coefficient co-occurrences (>2 standardized units):
0 current_count 68 prev_count 45 partners [(0, 17), (13, 16), (1, 14), (16, 11), (7, 10), (9, 9)]
1 current_count 29 prev_count 73 partners [(1, 9), (13, 9), (11, 7), (14, 5), (16, 5), (21, 4)]
2 current_count 47 prev_count 36 partners [(2, 12), (7, 11), (13, 11), (9, 11), (20, 7), (0, 6)]
3 current_count 36 prev_count 32 partners [(3, 8), (1, 7), (24, 7), (13, 6), (25, 5), (7, 5)]
4 current_count 45 prev_count 17 partners [(0, 9), (2, 8), (21, 8), (7, 7), (13, 7), (9, 7)]
5 current_count 33 prev_count 44 partners [(1, 8), (5, 8), (21, 8), (9, 7), (14, 5), (13, 5)]
6 current_count 37 prev_count 9 partners [(24, 5), (15, 4), (20, 4), (3, 3), (13, 3), (16, 3)]
7 current_count 68 prev_count 62 partners [(7, 15), (14, 11), (13, 11), (0, 10), (1, 10), (9, 10)]
8 current_count 36 prev_count 30 partners [(1, 10), (13, 8), (7, 5), (5, 5), (9, 5), (8, 4)]
9 current_count 34 prev_count 63 partners [(9, 9), (1, 8), (13, 7), (7, 6), (16, 6), (11, 5)]
10 current_count 33 prev_count 15 partners [(24, 6), (5, 5), (14, 4), (3, 3), (12, 3), (21, 3)]
11 current_count 32 prev_count 35 partners [(9, 7), (16, 6), (11, 5), (7, 5), (5, 4), (12, 4)]
12 current_count 24 prev_count 39 partners [(19, 6), (9, 5), (7, 3), (1, 3), (12, 3), (13, 3)]
13 current_count 19 prev_count 85 partners [(13, 4), (26, 3), (9, 2), (1, 2), (11, 2), (5, 2)]
14 current_count 41 prev_count 55 partners [(13, 12), (1, 10), (9, 8), (11, 8), (14, 7), (5, 7)]
15 current_count 15 prev_count 17 partners [(15, 6), (23, 3), (21, 3), (7, 2), (20, 2), (19, 2)]
16 current_count 33 prev_count 52 partners [(14, 7), (1, 6), (13, 6), (7, 6), (19, 6), (16, 6)]
17 current_count 36 prev_count 4 partners [(13, 9), (9, 8), (1, 7), (3, 7), (0, 6), (7, 6)]
18 current_count 28 prev_count 5 partners [(13, 7), (24, 6), (15, 5), (9, 5), (16, 5), (21, 5)]
19 current_count 33 prev_count 41 partners [(19, 12), (13, 7), (12, 5), (1, 5), (21, 5), (23, 5)]
20 current_count 36 prev_count 39 partners [(20, 8), (1, 7), (21, 5), (0, 5), (13, 4), (25, 4)]
21 current_count 46 prev_count 43 partners [(21, 10), (24, 9), (14, 8), (9, 7), (7, 6), (13, 6)]
22 current_count 42 prev_count 6 partners [(1, 7), (11, 7), (13, 7), (16, 7), (5, 6), (7, 5)]
23 current_count 32 prev_count 32 partners [(1, 11), (23, 7), (16, 6), (21, 6), (24, 5), (5, 5)]
24 current_count 47 prev_count 55 partners [(13, 14), (24, 14), (9, 13), (11, 12), (19, 8), (8, 8)]
25 current_count 51 prev_count 16 partners [(13, 11), (14, 7), (21, 7), (0, 6), (3, 6), (9, 6)]
26 current_count 26 prev_count 36 partners [(13, 8), (1, 6), (20, 6), (21, 6), (26, 5), (19, 4)]

ICA of weight coefficients: raw
skew: [ 9.71  0.74  7.3   4.13  3.9  15.11  7.59  1.94  1.81  4.9   1.43  2.51  3.33  1.39  3.73  4.33 13.32  6.8   3.07  4.56  7.31  3.14  0.23  2.38
  5.16  3.61  5.36]
excess kurtosis: [215.66  61.35 122.53  67.02  43.06 497.03 160.33  23.93  27.35  60.2   22.75  21.94  32.17  20.03  41.58  49.07 345.56 135.11  33.3   72.84 111.2
  41.01  21.63  26.78  67.45  42.45  96.53]
current top-component counts [ 49  41  36  50  42  91 155  28  50  57  93  29  46  70  58  35  66  47  33  82  39  50  65  48  52  45  79]
H5-composed top-component counts [ 19  33   5  33  61  20  69 152  64  54  70  97  71  48  47  28   0  47  88 100  15  93  48  42  46  76 110]
H1-composed top-component counts [ 56  25 156 103  30  40   6  28  26  55  33  32  43  12  41  61  38  96  84  65 115  86  82  33  44 111  35]

ICA of weight coefficients: multiply_gain
skew: [ 5.07  2.73  2.55  0.7   2.86  4.44  5.75  6.74 10.86  1.65  0.74  4.08  3.4   1.12  1.34  4.64  4.03 10.88  2.58  4.82  3.49  8.99  2.69  3.78
  6.16  3.32  3.23]
excess kurtosis: [ 90.44  34.14  27.07  21.16  35.57  66.56 112.45  98.84 261.56  20.39  15.85  44.99  39.09  59.24  20.58  57.69  70.06 317.72  29.88  62.44  53.51
 193.55  21.42  39.49  95.69  31.68  38.35]
current top-component counts [ 61  43  46  79  48  73 164  36  85  72  18  34  46  50  27  64  73 117  28  70  46  47  30  54  38  45  42]
H5-composed top-component counts [ 47  53  35  37  89  91 135  20   0  39 110  26  66  41 171  46 111  13  80  41  26  19  84  33   5  69  49]
H1-composed top-component counts [100  19  33  68  93  59  16 114  42  11  11  64 107  27  29  53  29  41 100  41  96  64  29  29 178  45  38]

Last-projection independent-component excess kurtosis: [24.6   1.57  1.34  1.29  1.2   1.18  1.14  1.11  1.08  1.05  1.04  0.97  0.96  0.94  0.92  0.92  0.91  0.89  0.83  0.82  0.4   0.08 -0.12 -0.3
 -0.32 -0.33 -0.4  -0.45]
Last-projection independent-component absolute skew: [0.6  0.37 0.33 0.27 0.23 0.2  0.2  0.19 0.17 0.17 0.12 0.11 0.11 0.1  0.1  0.1  0.09 0.09 0.09 0.08 0.07 0.06 0.05 0.05 0.03 0.03 0.03 0.  ]

```
CLI OUTPUT IN EVENT ORDER
ERROR
Code Mode is unavailable because code-mode host is disabled. Code mode will fail closed; enable `features.code_mode_host` and install `codex-code-mode-host`.

AGENT_MESSAGE
```python
import numpy as np
from scipy.linalg import svd
from threadpoolctl import threadpool_limits
threadpool_limits(4)

g=W["ln1_w.0"].astype(float)
v=np.concatenate([W["W_v.0"][64:128],W["W_v.0"][320:384]]).astype(float)
p=np.concatenate([W[n][s] for n in ["W_q.0","W_k.0"]
                  for s in [slice(64,128),slice(320,384)]]).astype(float)

def basis_from_gradient_rows(m,r):
    _,_,vt=svd(m,full_matrices=False)
    b=vt[:r].T/g[:,None]
    b-=b.mean(axis=0,keepdims=True)
    return np.linalg.qr(b)[0]

Bc=basis_from_gradient_rows(v,28)
Bp=basis_from_gradient_rows(p,32)
print("Mean squared character/position candidate subspace overlap:",
      np.linalg.norm(Bc.T@Bp)**2/28)
print("Energy fractions in candidate character and position input subspaces:")
for j in range(6):
    sl=slice(64*j,64*(j+1))
    for name in ["W_q","W_k","W_v"]:
        m=W[f"{name}.0"][sl].astype(float)*g
        m-=m.mean(axis=1,keepdims=True)
        print(f"L0H{j}",name,
              np.round([np.linalg.norm(m@Bc)**2/np.linalg.norm(m)**2,
                        np.linalg.norm(m@Bp)**2/np.linalg.norm(m)**2],4))
    c=W["W_o.0"][:,sl].astype(float)@(W["W_v.0"][sl].astype(float)*g)
    u,s,vt=svd(c,full_matrices=False)
    print(" OV top four right-singular-vector character / position overlaps:",
          np.round(np.sum((vt[:4]@Bc)**2,axis=1),3),
          np.round(np.sum((vt[:4]@Bp)**2,axis=1),3))

print("\nKey-bias weight vectors across heads:")
keybias={}
Cs={}
for l in range(4):
    gg=W[f"ln1_w.{l}"].astype(float)
    bb=W[f"ln1_b.{l}"].astype(float)
    for j in range(6):
        sl=slice(64*j,64*(j+1))
        q=W[f"W_q.{l}"][sl].astype(float)
        k=W[f"W_k.{l}"][sl].astype(float)*gg
        k-=k.mean(axis=1,keepdims=True)
        keybias[(l,j)]=k.T@(q@bb)/8
        c=W[f"W_o.{l}"][:,sl].astype(float)@(W[f"W_v.{l}"][sl].astype(float)*gg)
        c-=c.mean(axis=0,keepdims=True)
        c-=c.mean(axis=1,keepdims=True)
        Cs[(l,j)]=c

reference=sum(keybias[(1,j)]/np.linalg.norm(keybias[(1,j)]) for j in range(6))
reference/=np.linalg.norm(reference)
for l in range(4):
    for j in range(6):
        z=keybias[(l,j)]
        print(f"L{l}H{j}","bias norm",round(np.linalg.norm(z),4),
              "cosine with common L1 bias",round(z@reference/np.linalg.norm(z),4))

print("\nCopy-path trace concentration on the candidate character subspace:")
for target,source in [((2,1),(0,5)),((3,1),(0,5)),((3,5),(0,5)),
                      ((3,1),(2,2)),((3,5),(2,2))]:
    l,j=target
    sl=slice(64*j,64*(j+1))
    gg=W[f"ln1_w.{l}"].astype(float)
    q=W[f"W_q.{l}"][sl].astype(float)*gg
    k=W[f"W_k.{l}"][sl].astype(float)*gg
    q-=q.mean(axis=1,keepdims=True)
    k-=k.mean(axis=1,keepdims=True)
    m=(q.T@k/8)@Cs[source]
    mc=Bc.T@m@Bc
    z=np.linalg.eigvals(mc)
    print(target,"via",source,"total trace",round(np.trace(m),4),
          "char-subspace trace",round(np.trace(mc),4),
          "char trace/nuclear norm",round(np.trace(mc)/np.linalg.svd(mc,compute_uv=False).sum(),4),
          "top real parts",np.round(np.sort(z.real)[-8:][::-1],4))
```


PERMITTED EXPERIMENT HARNESS RECORD: python
{
  "t": 1792713748.4617426,
  "s": 4.09,
  "code": "import numpy as np\nfrom scipy.linalg import svd\nfrom threadpoolctl import threadpool_limits\nthreadpool_limits(4)\n\ng=W[\"ln1_w.0\"].astype(float)\nv=np.concatenate([W[\"W_v.0\"][64:128],W[\"W_v.0\"][320:384]]).astype(float)\np=np.concatenate([W[n][s] for n in [\"W_q.0\",\"W_k.0\"]\n                  for s in [slice(64,128),slice(320,384)]]).astype(float)\n\ndef basis_from_gradient_rows(m,r):\n    _,_,vt=svd(m,full_matrices=False)\n    b=vt[:r].T/g[:,None]\n    b-=b.mean(axis=0,keepdims=True)\n    return np.linalg.qr(b)[0]\n\nBc=basis_from_gradient_rows(v,28)\nBp=basis_from_gradient_rows(p,32)\nprint(\"Mean squared character/position candidate subspace overlap:\",\n      np.linalg.norm(Bc.T@Bp)**2/28)\nprint(\"Energy fractions in candidate character and position input subspaces:\")\nfor j in range(6):\n    sl=slice(64*j,64*(j+1))\n    for name in [\"W_q\",\"W_k\",\"W_v\"]:\n        m=W[f\"{name}.0\"][sl].astype(float)*g\n        m-=m.mean(axis=1,keepdims=True)\n        print(f\"L0H{j}\",name,\n              np.round([np.linalg.norm(m@Bc)**2/np.linalg.norm(m)**2,\n                        np.linalg.norm(m@Bp)**2/np.linalg.norm(m)**2],4))\n    c=W[\"W_o.0\"][:,sl].astype(float)@(W[\"W_v.0\"][sl].astype(float)*g)\n    u,s,vt=svd(c,full_matrices=False)\n    print(\" OV top four right-singular-vector character / position overlaps:\",\n          np.round(np.sum((vt[:4]@Bc)**2,axis=1),3),\n          np.round(np.sum((vt[:4]@Bp)**2,axis=1),3))\n\nprint(\"\\nKey-bias weight vectors across heads:\")\nkeybias={}\nCs={}\nfor l in range(4):\n    gg=W[f\"ln1_w.{l}\"].astype(float)\n    bb=W[f\"ln1_b.{l}\"].astype(float)\n    for j in range(6):\n        sl=slice(64*j,64*(j+1))\n        q=W[f\"W_q.{l}\"][sl].astype(float)\n        k=W[f\"W_k.{l}\"][sl].astype(float)*gg\n        k-=k.mean(axis=1,keepdims=True)\n        keybias[(l,j)]=k.T@(q@bb)/8\n        c=W[f\"W_o.{l}\"][:,sl].astype(float)@(W[f\"W_v.{l}\"][sl].astype(float)*gg)\n        c-=c.mean(axis=0,keepdims=True)\n        c-=c.mean(axis=1,keepdims=True)\n        Cs[(l,j)]=c\n\nreference=sum(keybias[(1,j)]/np.linalg.norm(keybias[(1,j)]) for j in range(6))\nreference/=np.linalg.norm(reference)\nfor l in range(4):\n    for j in range(6):\n        z=keybias[(l,j)]\n        print(f\"L{l}H{j}\",\"bias norm\",round(np.linalg.norm(z),4),\n              \"cosine with common L1 bias\",round(z@reference/np.linalg.norm(z),4))\n\nprint(\"\\nCopy-path trace concentration on the candidate character subspace:\")\nfor target,source in [((2,1),(0,5)),((3,1),(0,5)),((3,5),(0,5)),\n                      ((3,1),(2,2)),((3,5),(2,2))]:\n    l,j=target\n    sl=slice(64*j,64*(j+1))\n    gg=W[f\"ln1_w.{l}\"].astype(float)\n    q=W[f\"W_q.{l}\"][sl].astype(float)*gg\n    k=W[f\"W_k.{l}\"][sl].astype(float)*gg\n    q-=q.mean(axis=1,keepdims=True)\n    k-=k.mean(axis=1,keepdims=True)\n    m=(q.T@k/8)@Cs[source]\n    mc=Bc.T@m@Bc\n    z=np.linalg.eigvals(mc)\n    print(target,\"via\",source,\"total trace\",round(np.trace(m),4),\n          \"char-subspace trace\",round(np.trace(mc),4),\n          \"char trace/nuclear norm\",round(np.trace(mc)/np.linalg.svd(mc,compute_uv=False).sum(),4),\n          \"top real parts\",np.round(np.sort(z.real)[-8:][::-1],4))\n",
  "out": "Mean squared character/position candidate subspace overlap: 0.32920011084181006\nEnergy fractions in candidate character and position input subspaces:\nL0H0 W_q [0.0038 0.1503]\nL0H0 W_k [0.0515 0.0651]\nL0H0 W_v [0.0258 0.2115]\n OV top four right-singular-vector character / position overlaps: [0.003 0.006 0.084 0.17 ] [0.031 0.424 0.207 0.155]\nL0H1 W_q [0.0321 0.1967]\nL0H1 W_k [0.0138 0.2379]\nL0H1 W_v [0.227  0.0614]\n OV top four right-singular-vector character / position overlaps: [0.177 0.268 0.289 0.361] [0.027 0.021 0.063 0.041]\nL0H2 W_q [0.0434 0.1376]\nL0H2 W_k [0.0495 0.141 ]\nL0H2 W_v [0.142  0.0714]\n OV top four right-singular-vector character / position overlaps: [0.276 0.203 0.203 0.229] [0.246 0.072 0.063 0.063]\nL0H3 W_q [0.0176 0.1138]\nL0H3 W_k [0.0507 0.1624]\nL0H3 W_v [0.0783 0.0768]\n OV top four right-singular-vector character / position overlaps: [0.006 0.128 0.172 0.114] [0.094 0.094 0.025 0.148]\nL0H4 W_q [0.0067 0.0198]\nL0H4 W_k [0.0124 0.0261]\nL0H4 W_v [0.0165 0.2812]\n OV top four right-singular-vector character / position overlaps: [0.001 0.002 0.008 0.177] [0.405 0.153 0.107 0.11 ]\nL0H5 W_q [0.0133 0.1889]\nL0H5 W_k [0.0054 0.1982]\nL0H5 W_v [0.2076 0.0717]\n OV top four right-singular-vector character / position overlaps: [0.24  0.296 0.31  0.319] [0.045 0.033 0.02  0.059]\n\nKey-bias weight vectors across heads:\nL0H0 bias norm 0.0185 cosine with common L1 bias -0.1842\nL0H1 bias norm 0.0736 cosine with common L1 bias 0.3711\nL0H2 bias norm 0.0234 cosine with common L1 bias -0.0452\nL0H3 bias norm 0.057 cosine with common L1 bias -0.0608\nL0H4 bias norm 0.0009 cosine with common L1 bias -0.0478\nL0H5 bias norm 0.0509 cosine with common L1 bias -0.2918\nL1H0 bias norm 0.8392 cosine with common L1 bias 0.879\nL1H1 bias norm 3.1498 cosine with common L1 bias 0.9652\nL1H2 bias norm 2.1998 cosine with common L1 bias 0.9539\nL1H3 bias norm 2.7194 cosine with common L1 bias 0.9627\nL1H4 bias norm 2.518 cosine with common L1 bias 0.9612\nL1H5 bias norm 2.4602 cosine with common L1 bias 0.9374\nL2H0 bias norm 0.8945 cosine with common L1 bias 0.6644\nL2H1 bias norm 0.6377 cosine with common L1 bias 0.1501\nL2H2 bias norm 1.5836 cosine with common L1 bias 0.9043\nL2H3 bias norm 1.0368 cosine with common L1 bias 0.8715\nL2H4 bias norm 1.1994 cosine with common L1 bias 0.8107\nL2H5 bias norm 0.6361 cosine with common L1 bias 0.5232\nL3H0 bias norm 0.2987 cosine with common L1 bias 0.2548\nL3H1 bias norm 0.2634 cosine with common L1 bias -0.0474\nL3H2 bias norm 0.3422 cosine with common L1 bias 0.477\nL3H3 bias norm 0.2583 cosine with common L1 bias 0.1958\nL3H4 bias norm 0.3067 cosine with common L1 bias 0.4027\nL3H5 bias norm 0.3101 cosine with common L1 bias -0.0416\n\nCopy-path trace concentration on the candidate character subspace:\n(2, 1) via (0, 5) total trace 0.5595 char-subspace trace 0.2795 char trace/nuclear norm 0.6906 top real parts [0.0393 0.0326 0.0326 0.025  0.025  0.0197 0.0182 0.0182]\n(3, 1) via (0, 5) total trace 0.3831 char-subspace trace 0.1978 char trace/nuclear norm 0.6332 top real parts [0.025  0.0204 0.0204 0.0183 0.0183 0.0122 0.0122 0.0103]\n(3, 5) via (0, 5) total trace 0.5596 char-subspace trace 0.3028 char trace/nuclear norm 0.6912 top real parts [0.0411 0.029  0.0272 0.0272 0.0217 0.0217 0.0187 0.0187]\n(3, 1) via (2, 2) total trace 3.1409 char-subspace trace 0.4064 char trace/nuclear norm 0.5624 top real parts [0.0519 0.0519 0.045  0.045  0.0342 0.0325 0.0325 0.0195]\n(3, 5) via (2, 2) total trace 6.521 char-subspace trace 0.6925 char trace/nuclear norm 0.6499 top real parts [0.0969 0.0632 0.059  0.059  0.0522 0.0485 0.0485 0.0442]\n"
}

========================================================================
TURN t007
MESSAGE SENT TO CODEX
OUTPUT of run 6 of at most 100:
```
Mean squared character/position candidate subspace overlap: 0.32920011084181006
Energy fractions in candidate character and position input subspaces:
L0H0 W_q [0.0038 0.1503]
L0H0 W_k [0.0515 0.0651]
L0H0 W_v [0.0258 0.2115]
 OV top four right-singular-vector character / position overlaps: [0.003 0.006 0.084 0.17 ] [0.031 0.424 0.207 0.155]
L0H1 W_q [0.0321 0.1967]
L0H1 W_k [0.0138 0.2379]
L0H1 W_v [0.227  0.0614]
 OV top four right-singular-vector character / position overlaps: [0.177 0.268 0.289 0.361] [0.027 0.021 0.063 0.041]
L0H2 W_q [0.0434 0.1376]
L0H2 W_k [0.0495 0.141 ]
L0H2 W_v [0.142  0.0714]
 OV top four right-singular-vector character / position overlaps: [0.276 0.203 0.203 0.229] [0.246 0.072 0.063 0.063]
L0H3 W_q [0.0176 0.1138]
L0H3 W_k [0.0507 0.1624]
L0H3 W_v [0.0783 0.0768]
 OV top four right-singular-vector character / position overlaps: [0.006 0.128 0.172 0.114] [0.094 0.094 0.025 0.148]
L0H4 W_q [0.0067 0.0198]
L0H4 W_k [0.0124 0.0261]
L0H4 W_v [0.0165 0.2812]
 OV top four right-singular-vector character / position overlaps: [0.001 0.002 0.008 0.177] [0.405 0.153 0.107 0.11 ]
L0H5 W_q [0.0133 0.1889]
L0H5 W_k [0.0054 0.1982]
L0H5 W_v [0.2076 0.0717]
 OV top four right-singular-vector character / position overlaps: [0.24  0.296 0.31  0.319] [0.045 0.033 0.02  0.059]

Key-bias weight vectors across heads:
L0H0 bias norm 0.0185 cosine with common L1 bias -0.1842
L0H1 bias norm 0.0736 cosine with common L1 bias 0.3711
L0H2 bias norm 0.0234 cosine with common L1 bias -0.0452
L0H3 bias norm 0.057 cosine with common L1 bias -0.0608
L0H4 bias norm 0.0009 cosine with common L1 bias -0.0478
L0H5 bias norm 0.0509 cosine with common L1 bias -0.2918
L1H0 bias norm 0.8392 cosine with common L1 bias 0.879
L1H1 bias norm 3.1498 cosine with common L1 bias 0.9652
L1H2 bias norm 2.1998 cosine with common L1 bias 0.9539
L1H3 bias norm 2.7194 cosine with common L1 bias 0.9627
L1H4 bias norm 2.518 cosine with common L1 bias 0.9612
L1H5 bias norm 2.4602 cosine with common L1 bias 0.9374
L2H0 bias norm 0.8945 cosine with common L1 bias 0.6644
L2H1 bias norm 0.6377 cosine with common L1 bias 0.1501
L2H2 bias norm 1.5836 cosine with common L1 bias 0.9043
L2H3 bias norm 1.0368 cosine with common L1 bias 0.8715
L2H4 bias norm 1.1994 cosine with common L1 bias 0.8107
L2H5 bias norm 0.6361 cosine with common L1 bias 0.5232
L3H0 bias norm 0.2987 cosine with common L1 bias 0.2548
L3H1 bias norm 0.2634 cosine with common L1 bias -0.0474
L3H2 bias norm 0.3422 cosine with common L1 bias 0.477
L3H3 bias norm 0.2583 cosine with common L1 bias 0.1958
L3H4 bias norm 0.3067 cosine with common L1 bias 0.4027
L3H5 bias norm 0.3101 cosine with common L1 bias -0.0416

Copy-path trace concentration on the candidate character subspace:
(2, 1) via (0, 5) total trace 0.5595 char-subspace trace 0.2795 char trace/nuclear norm 0.6906 top real parts [0.0393 0.0326 0.0326 0.025  0.025  0.0197 0.0182 0.0182]
(3, 1) via (0, 5) total trace 0.3831 char-subspace trace 0.1978 char trace/nuclear norm 0.6332 top real parts [0.025  0.0204 0.0204 0.0183 0.0183 0.0122 0.0122 0.0103]
(3, 5) via (0, 5) total trace 0.5596 char-subspace trace 0.3028 char trace/nuclear norm 0.6912 top real parts [0.0411 0.029  0.0272 0.0272 0.0217 0.0217 0.0187 0.0187]
(3, 1) via (2, 2) total trace 3.1409 char-subspace trace 0.4064 char trace/nuclear norm 0.5624 top real parts [0.0519 0.0519 0.045  0.045  0.0342 0.0325 0.0325 0.0195]
(3, 5) via (2, 2) total trace 6.521 char-subspace trace 0.6925 char trace/nuclear norm 0.6499 top real parts [0.0969 0.0632 0.059  0.059  0.0522 0.0485 0.0485 0.0442]

```
CLI OUTPUT IN EVENT ORDER
ERROR
Code Mode is unavailable because code-mode host is disabled. Code mode will fail closed; enable `features.code_mode_host` and install `codex-code-mode-host`.

AGENT_MESSAGE
```python
import numpy as np
from scipy.linalg import svdvals
from threadpoolctl import threadpool_limits
threadpool_limits(4)

A,C,bias={},{},{}
for l in range(4):
    g=W[f"ln1_w.{l}"].astype(float)
    b=W[f"ln1_b.{l}"].astype(float)
    for j in range(6):
        s=slice(64*j,64*(j+1))
        q0=W[f"W_q.{l}"][s].astype(float)
        k0=W[f"W_k.{l}"][s].astype(float)
        q=q0*g
        k=k0*g
        v=W[f"W_v.{l}"][s].astype(float)*g
        q-=q.mean(axis=1,keepdims=True)
        k-=k.mean(axis=1,keepdims=True)
        v-=v.mean(axis=1,keepdims=True)
        o=W[f"W_o.{l}"][:,s].astype(float)
        o-=o.mean(axis=0,keepdims=True)
        n=(l,j)
        A[n]=q.T@k/8
        C[n]=o@v
        bias[n]=k.T@(q0@b)/8

reference=sum(bias[(1,j)]/np.linalg.norm(bias[(1,j)]) for j in range(6))
reference/=np.linalg.norm(reference)
print("Final compact weight-only audit:")
print("head  ||A||F  ||C||F  A_stable_rank  C_top4_energy  keybias_norm  common_bias_cos")
for l in range(4):
    for j in range(6):
        n=(l,j)
        sa=svdvals(A[n],check_finite=False)
        sc=svdvals(C[n],check_finite=False)
        print(f"L{l}H{j}",*[
            round(float(x),4) for x in [
                np.linalg.norm(A[n]),np.linalg.norm(C[n]),
                np.sum(sa*sa)/(sa[0]**2),
                np.sum(sc[:4]**2)/np.sum(sc**2),
                np.linalg.norm(bias[n]),
                bias[n]@reference/np.linalg.norm(bias[n])]])

print("\nCoordinate-permutation controls for proposed copy-path products:")
rng=np.random.default_rng(123)
paths=[((2,1),(0,5)),((3,1),(0,5)),((3,5),(0,5)),
       ((3,1),(2,2)),((3,5),(2,2))]
for target,source in paths:
    a=A[target]; c=C[source]
    denom=np.linalg.norm(a)*np.linalg.norm(c)
    observed=np.sum(a*c.T)/denom
    controls=[]
    for _ in range(400):
        p=rng.permutation(384)
        controls.append(np.sum(a*c[np.ix_(p,p)].T)/denom)
    controls=np.array(controls)
    print("target",target,"source",source,
          "observed",round(observed,5),
          "null_mean",round(controls.mean(),5),
          "null_std",round(controls.std(),5),
          "null_max",round(controls.max(),5))

a=A[(0,5)]
b=A[(0,1)]
print("\nPositional-product cosines:",
      "cos(A01,A05)",round(np.sum(b*a)/np.linalg.norm(b)/np.linalg.norm(a),5),
      "cos(A01,A05^2)",round(np.sum(b*(a@a))/np.linalg.norm(b)/np.linalg.norm(a@a),5))
sa=svdvals(A[(0,4)],check_finite=False)
# Only an operator-norm bound, not evaluation on an input.
print("L0H4 operator-norm bound on variation across unmasked key scores:",
      round(2*384*sa[0]+2*np.sqrt(384)*np.linalg.norm(bias[(0,4)]),5))
print("All computations above are parameter decompositions/products; no network layer was evaluated.")
```


PERMITTED EXPERIMENT HARNESS RECORD: python
{
  "t": 1792713973.9775605,
  "s": 8.46,
  "code": "import numpy as np\nfrom scipy.linalg import svdvals\nfrom threadpoolctl import threadpool_limits\nthreadpool_limits(4)\n\nA,C,bias={},{},{}\nfor l in range(4):\n    g=W[f\"ln1_w.{l}\"].astype(float)\n    b=W[f\"ln1_b.{l}\"].astype(float)\n    for j in range(6):\n        s=slice(64*j,64*(j+1))\n        q0=W[f\"W_q.{l}\"][s].astype(float)\n        k0=W[f\"W_k.{l}\"][s].astype(float)\n        q=q0*g\n        k=k0*g\n        v=W[f\"W_v.{l}\"][s].astype(float)*g\n        q-=q.mean(axis=1,keepdims=True)\n        k-=k.mean(axis=1,keepdims=True)\n        v-=v.mean(axis=1,keepdims=True)\n        o=W[f\"W_o.{l}\"][:,s].astype(float)\n        o-=o.mean(axis=0,keepdims=True)\n        n=(l,j)\n        A[n]=q.T@k/8\n        C[n]=o@v\n        bias[n]=k.T@(q0@b)/8\n\nreference=sum(bias[(1,j)]/np.linalg.norm(bias[(1,j)]) for j in range(6))\nreference/=np.linalg.norm(reference)\nprint(\"Final compact weight-only audit:\")\nprint(\"head  ||A||F  ||C||F  A_stable_rank  C_top4_energy  keybias_norm  common_bias_cos\")\nfor l in range(4):\n    for j in range(6):\n        n=(l,j)\n        sa=svdvals(A[n],check_finite=False)\n        sc=svdvals(C[n],check_finite=False)\n        print(f\"L{l}H{j}\",*[\n            round(float(x),4) for x in [\n                np.linalg.norm(A[n]),np.linalg.norm(C[n]),\n                np.sum(sa*sa)/(sa[0]**2),\n                np.sum(sc[:4]**2)/np.sum(sc**2),\n                np.linalg.norm(bias[n]),\n                bias[n]@reference/np.linalg.norm(bias[n])]])\n\nprint(\"\\nCoordinate-permutation controls for proposed copy-path products:\")\nrng=np.random.default_rng(123)\npaths=[((2,1),(0,5)),((3,1),(0,5)),((3,5),(0,5)),\n       ((3,1),(2,2)),((3,5),(2,2))]\nfor target,source in paths:\n    a=A[target]; c=C[source]\n    denom=np.linalg.norm(a)*np.linalg.norm(c)\n    observed=np.sum(a*c.T)/denom\n    controls=[]\n    for _ in range(400):\n        p=rng.permutation(384)\n        controls.append(np.sum(a*c[np.ix_(p,p)].T)/denom)\n    controls=np.array(controls)\n    print(\"target\",target,\"source\",source,\n          \"observed\",round(observed,5),\n          \"null_mean\",round(controls.mean(),5),\n          \"null_std\",round(controls.std(),5),\n          \"null_max\",round(controls.max(),5))\n\na=A[(0,5)]\nb=A[(0,1)]\nprint(\"\\nPositional-product cosines:\",\n      \"cos(A01,A05)\",round(np.sum(b*a)/np.linalg.norm(b)/np.linalg.norm(a),5),\n      \"cos(A01,A05^2)\",round(np.sum(b*(a@a))/np.linalg.norm(b)/np.linalg.norm(a@a),5))\nsa=svdvals(A[(0,4)],check_finite=False)\n# Only an operator-norm bound, not evaluation on an input.\nprint(\"L0H4 operator-norm bound on variation across unmasked key scores:\",\n      round(2*384*sa[0]+2*np.sqrt(384)*np.linalg.norm(bias[(0,4)]),5))\nprint(\"All computations above are parameter decompositions/products; no network layer was evaluated.\")\n",
  "out": "Final compact weight-only audit:\nhead  ||A||F  ||C||F  A_stable_rank  C_top4_energy  keybias_norm  common_bias_cos\nL0H0 0.1137 4.3206 1.6438 0.9783 0.0185 -0.1842\nL0H1 0.8775 3.4885 2.1296 0.2775 0.0736 0.3711\nL0H2 0.1745 2.0228 1.7489 0.7358 0.0234 -0.0452\nL0H3 0.2395 1.6214 2.1182 0.6714 0.057 -0.0608\nL0H4 0.0093 17.158 4.2704 0.9945 0.0009 -0.0478\nL0H5 0.9704 3.7066 5.6044 0.2935 0.0509 -0.2918\nL1H0 0.644 13.4364 3.3136 0.4466 0.8392 0.879\nL1H1 1.5843 9.5923 1.9127 0.316 3.1498 0.9652\nL1H2 1.2358 11.4958 2.3459 0.3059 2.1998 0.9539\nL1H3 1.3094 10.0431 2.0582 0.3632 2.7194 0.9627\nL1H4 1.2492 9.557 2.1699 0.3631 2.518 0.9612\nL1H5 1.4614 9.8262 2.0984 0.284 2.4602 0.9374\nL2H0 2.0456 18.6979 15.5692 0.1656 0.8945 0.6644\nL2H1 1.9454 21.4478 18.0233 0.2322 0.6377 0.1501\nL2H2 2.804 16.7822 15.6466 0.1587 1.5836 0.9043\nL2H3 1.9759 20.617 11.9841 0.1741 1.0368 0.8715\nL2H4 2.2016 18.4145 7.9119 0.175 1.1994 0.8107\nL2H5 1.6541 23.7661 15.5572 0.2126 0.6361 0.5232\nL3H0 2.0777 24.5018 26.2622 0.1639 0.2987 0.2548\nL3H1 1.6948 25.7241 27.6524 0.203 0.2634 -0.0474\nL3H2 2.0608 26.8596 22.5745 0.2367 0.3422 0.477\nL3H3 1.053 37.938 16.106 0.3298 0.2583 0.1958\nL3H4 1.7344 28.4402 18.7379 0.2781 0.3067 0.4027\nL3H5 2.1169 23.6874 27.4428 0.3243 0.3101 -0.0416\n\nCoordinate-permutation controls for proposed copy-path products:\ntarget (2, 1) source (0, 5) observed 0.07759 null_mean -0.00228 null_std 0.00266 null_max 0.00493\ntarget (3, 1) source (0, 5) observed 0.06098 null_mean -0.00223 null_std 0.00262 null_max 0.00674\ntarget (3, 5) source (0, 5) observed 0.07132 null_mean -0.0066 null_std 0.00276 null_max 0.00315\ntarget (3, 1) source (2, 2) observed 0.11043 null_mean -0.00627 null_std 0.00254 null_max 0.00028\ntarget (3, 5) source (2, 2) observed 0.18356 null_mean -0.01751 null_std 0.00254 null_max -0.01046\n\nPositional-product cosines: cos(A01,A05) 0.41365 cos(A01,A05^2) 0.54612\nL0H4 operator-norm bound on variation across unmasked key scores: 3.47631\nAll computations above are parameter decompositions/products; no network layer was evaluated.\n"
}

========================================================================
TURN t008
MESSAGE SENT TO CODEX
OUTPUT of run 7 of at most 100:
```
Final compact weight-only audit:
head  ||A||F  ||C||F  A_stable_rank  C_top4_energy  keybias_norm  common_bias_cos
L0H0 0.1137 4.3206 1.6438 0.9783 0.0185 -0.1842
L0H1 0.8775 3.4885 2.1296 0.2775 0.0736 0.3711
L0H2 0.1745 2.0228 1.7489 0.7358 0.0234 -0.0452
L0H3 0.2395 1.6214 2.1182 0.6714 0.057 -0.0608
L0H4 0.0093 17.158 4.2704 0.9945 0.0009 -0.0478
L0H5 0.9704 3.7066 5.6044 0.2935 0.0509 -0.2918
L1H0 0.644 13.4364 3.3136 0.4466 0.8392 0.879
L1H1 1.5843 9.5923 1.9127 0.316 3.1498 0.9652
L1H2 1.2358 11.4958 2.3459 0.3059 2.1998 0.9539
L1H3 1.3094 10.0431 2.0582 0.3632 2.7194 0.9627
L1H4 1.2492 9.557 2.1699 0.3631 2.518 0.9612
L1H5 1.4614 9.8262 2.0984 0.284 2.4602 0.9374
L2H0 2.0456 18.6979 15.5692 0.1656 0.8945 0.6644
L2H1 1.9454 21.4478 18.0233 0.2322 0.6377 0.1501
L2H2 2.804 16.7822 15.6466 0.1587 1.5836 0.9043
L2H3 1.9759 20.617 11.9841 0.1741 1.0368 0.8715
L2H4 2.2016 18.4145 7.9119 0.175 1.1994 0.8107
L2H5 1.6541 23.7661 15.5572 0.2126 0.6361 0.5232
L3H0 2.0777 24.5018 26.2622 0.1639 0.2987 0.2548
L3H1 1.6948 25.7241 27.6524 0.203 0.2634 -0.0474
L3H2 2.0608 26.8596 22.5745 0.2367 0.3422 0.477
L3H3 1.053 37.938 16.106 0.3298 0.2583 0.1958
L3H4 1.7344 28.4402 18.7379 0.2781 0.3067 0.4027
L3H5 2.1169 23.6874 27.4428 0.3243 0.3101 -0.0416

Coordinate-permutation controls for proposed copy-path products:
target (2, 1) source (0, 5) observed 0.07759 null_mean -0.00228 null_std 0.00266 null_max 0.00493
target (3, 1) source (0, 5) observed 0.06098 null_mean -0.00223 null_std 0.00262 null_max 0.00674
target (3, 5) source (0, 5) observed 0.07132 null_mean -0.0066 null_std 0.00276 null_max 0.00315
target (3, 1) source (2, 2) observed 0.11043 null_mean -0.00627 null_std 0.00254 null_max 0.00028
target (3, 5) source (2, 2) observed 0.18356 null_mean -0.01751 null_std 0.00254 null_max -0.01046

Positional-product cosines: cos(A01,A05) 0.41365 cos(A01,A05^2) 0.54612
L0H4 operator-norm bound on variation across unmasked key scores: 3.47631
All computations above are parameter decompositions/products; no network layer was evaluated.

```
CLI OUTPUT IN EVENT ORDER
ERROR
Code Mode is unavailable because code-mode host is disabled. Code mode will fail closed; enable `features.code_mode_host` and install `codex-code-mode-host`.

AGENT_MESSAGE
## Observations

All results below are **weight-only evidence, not measured attention patterns**. Exact attention distances remain uncertain without the embeddings.

Let \(A_h\) denote the head’s \(W_q^\top W_k/8\) product, incorporating `ln1_w` and mean-centering, and \(C_h\) its similarly adjusted \(W_oW_v\) product. Define the composition score
\[
R(t,s)=\frac{\operatorname{tr}(A_tC_s)}{\|A_t\|_F\|C_s\|_F}.
\]

Three findings stand out:

- **Offset-like early heads:** L0H1 and L0H5 have strongly overlapping Q/K subspaces but relatively separate value subspaces. Moreover, \(\cos(A_{01},A_{05}^2)=0.546\), versus \(\cos(A_{01},A_{05})=0.414\). This suggests related positional offsets—plausibly two-back and one-back.
- **A broad aggregation head:** L0H4 has \(\|A\|_F=0.0093\), compared with L0H5’s \(0.9704\). Its \(C\) product is large, \(\|C\|_F=17.158\), with **99.45%** of squared singular-value mass in four directions. It appears to aggregate a few shared features rather than select individual characters sharply.
- **Strong induction-compatible compositions:** \(R(\mathrm{L2H1},\mathrm{L0H5})=0.0776\), \(R(\mathrm{L3H1},\mathrm{L2H2})=0.1104\), and \(R(\mathrm{L3H5},\mathrm{L2H2})=0.1836\). Coordinate-permutation controls had standard deviations around \(0.0026\). These are substantial learned alignments, although they do not alone prove behavioral induction.

## Heads

Here, “shared bias alignment” means cosine similarity to the common L1 key-bias direction. Its interpretation as recency is tentative.

L0H0: **unclear** — Low-rank summary: four singular directions contain 97.83% of \(C\)’s energy; distance is unidentified.  
L0H1: **local** — Offset-like Q/K structure; stronger agreement with \(A_{05}^2\) suggests a somewhat longer offset than L0H5.  
L0H2: **unclear** — Low-rank QK structure, stable rank 1.75, but no decisive distance or induction signature.  
L0H3: **unclear** — QK stable rank 2.12; specialized routing is evident, but its positional range is not.  
L0H4: **broad** — Exceptionally small QK product, \(0.0093\), paired with strong, low-dimensional value aggregation.  
L0H5: **previous** — Best one-back candidate: offset-like positional structure and strong downstream induction-compatible compositions.

L1H0: **local** — Shared bias alignment 0.879; likely a weaker member of the position-biased L1 family.  
L1H1: **local** — Strong key-bias norm 3.150 and shared alignment 0.965; likely strongly recency-biased.  
L1H2: **local** — Key-bias norm 2.200, shared alignment 0.954; similar routing machinery.  
L1H3: **local** — Key-bias norm 2.719, shared alignment 0.963; also contributes to later matching paths.  
L1H4: **local** — Key-bias norm 2.518, shared alignment 0.961; closely resembles the other local candidates.  
L1H5: **local** — Key-bias norm 2.460, shared alignment 0.937; exact lag remains unresolved.

L2H0: **local** — Moderate shared bias alignment, 0.664, without a strong induction composition.  
L2H1: **copy** — Clear positive composition with proposed previous-character head L0H5: \(R=0.0776\).  
L2H2: **local** — Strong shared bias alignment, 0.904; especially important as a source for later copy-like matching.  
L2H3: **local** — Shared bias alignment 0.872; also supplies a positive matching path to L3H5.  
L2H4: **local** — Shared bias alignment 0.811 and substantial position-summary coupling.  
L2H5: **unclear** — Weaker positional alignment, 0.523; insufficient evidence to distinguish local from broad retrieval.

L3H0: **unclear** — Distributed QK structure, stable rank 26.26; neither distance nor induction is clearly established.  
L3H1: **copy** — Positive compositions through both L0H5, \(R=0.0610\), and L2H2, \(R=0.1104\).  
L3H2: **unclear** — Some shared positional bias remains, but no compelling copy signature or identifiable range.  
L3H3: **unclear** — Large value-writing product, \(\|C\|_F=37.938\), does not establish where it retrieves information.  
L3H4: **unclear** — Mixed routing evidence; neither broadness nor a particular local offset is securely supported.  
L3H5: **copy** — Strongest induction candidate: \(R=0.1836\) through L2H2, plus \(0.0713\) through L0H5.

## Reasoning

The offset-like early machinery and multiple strong matching compositions favor **preserved character order** over independently shuffled lines. The likely organization is short-context extraction, shared positional summaries, and later retrieval of matching earlier contexts.

**Python is my slight corpus favorite:** substantial multi-stage copying fits repeated identifiers and recurring code fragments. English remains a close alternative. Romaji is less favored by the comparatively rich character-processing structure, but that is weak evidence—not a recovered alphabet or phonotactic analysis.

The corpus judgment is substantially less secure than the order judgment; no specific letters, keywords, or indentation rules were identified.

corpus: python  
order: as is  
probabilities: python=0.52 romaji=0.08 english=0.40; as is=0.99 shuffled=0.01

SAVED FINAL ANSWER
## Observations

All results below are **weight-only evidence, not measured attention patterns**. Exact attention distances remain uncertain without the embeddings.

Let \(A_h\) denote the head’s \(W_q^\top W_k/8\) product, incorporating `ln1_w` and mean-centering, and \(C_h\) its similarly adjusted \(W_oW_v\) product. Define the composition score
\[
R(t,s)=\frac{\operatorname{tr}(A_tC_s)}{\|A_t\|_F\|C_s\|_F}.
\]

Three findings stand out:

- **Offset-like early heads:** L0H1 and L0H5 have strongly overlapping Q/K subspaces but relatively separate value subspaces. Moreover, \(\cos(A_{01},A_{05}^2)=0.546\), versus \(\cos(A_{01},A_{05})=0.414\). This suggests related positional offsets—plausibly two-back and one-back.
- **A broad aggregation head:** L0H4 has \(\|A\|_F=0.0093\), compared with L0H5’s \(0.9704\). Its \(C\) product is large, \(\|C\|_F=17.158\), with **99.45%** of squared singular-value mass in four directions. It appears to aggregate a few shared features rather than select individual characters sharply.
- **Strong induction-compatible compositions:** \(R(\mathrm{L2H1},\mathrm{L0H5})=0.0776\), \(R(\mathrm{L3H1},\mathrm{L2H2})=0.1104\), and \(R(\mathrm{L3H5},\mathrm{L2H2})=0.1836\). Coordinate-permutation controls had standard deviations around \(0.0026\). These are substantial learned alignments, although they do not alone prove behavioral induction.

## Heads

Here, “shared bias alignment” means cosine similarity to the common L1 key-bias direction. Its interpretation as recency is tentative.

L0H0: **unclear** — Low-rank summary: four singular directions contain 97.83% of \(C\)’s energy; distance is unidentified.  
L0H1: **local** — Offset-like Q/K structure; stronger agreement with \(A_{05}^2\) suggests a somewhat longer offset than L0H5.  
L0H2: **unclear** — Low-rank QK structure, stable rank 1.75, but no decisive distance or induction signature.  
L0H3: **unclear** — QK stable rank 2.12; specialized routing is evident, but its positional range is not.  
L0H4: **broad** — Exceptionally small QK product, \(0.0093\), paired with strong, low-dimensional value aggregation.  
L0H5: **previous** — Best one-back candidate: offset-like positional structure and strong downstream induction-compatible compositions.

L1H0: **local** — Shared bias alignment 0.879; likely a weaker member of the position-biased L1 family.  
L1H1: **local** — Strong key-bias norm 3.150 and shared alignment 0.965; likely strongly recency-biased.  
L1H2: **local** — Key-bias norm 2.200, shared alignment 0.954; similar routing machinery.  
L1H3: **local** — Key-bias norm 2.719, shared alignment 0.963; also contributes to later matching paths.  
L1H4: **local** — Key-bias norm 2.518, shared alignment 0.961; closely resembles the other local candidates.  
L1H5: **local** — Key-bias norm 2.460, shared alignment 0.937; exact lag remains unresolved.

L2H0: **local** — Moderate shared bias alignment, 0.664, without a strong induction composition.  
L2H1: **copy** — Clear positive composition with proposed previous-character head L0H5: \(R=0.0776\).  
L2H2: **local** — Strong shared bias alignment, 0.904; especially important as a source for later copy-like matching.  
L2H3: **local** — Shared bias alignment 0.872; also supplies a positive matching path to L3H5.  
L2H4: **local** — Shared bias alignment 0.811 and substantial position-summary coupling.  
L2H5: **unclear** — Weaker positional alignment, 0.523; insufficient evidence to distinguish local from broad retrieval.

L3H0: **unclear** — Distributed QK structure, stable rank 26.26; neither distance nor induction is clearly established.  
L3H1: **copy** — Positive compositions through both L0H5, \(R=0.0610\), and L2H2, \(R=0.1104\).  
L3H2: **unclear** — Some shared positional bias remains, but no compelling copy signature or identifiable range.  
L3H3: **unclear** — Large value-writing product, \(\|C\|_F=37.938\), does not establish where it retrieves information.  
L3H4: **unclear** — Mixed routing evidence; neither broadness nor a particular local offset is securely supported.  
L3H5: **copy** — Strongest induction candidate: \(R=0.1836\) through L2H2, plus \(0.0713\) through L0H5.

## Reasoning

The offset-like early machinery and multiple strong matching compositions favor **preserved character order** over independently shuffled lines. The likely organization is short-context extraction, shared positional summaries, and later retrieval of matching earlier contexts.

**Python is my slight corpus favorite:** substantial multi-stage copying fits repeated identifiers and recurring code fragments. English remains a close alternative. Romaji is less favored by the comparatively rich character-processing structure, but that is weak evidence—not a recovered alphabet or phonotactic analysis.

The corpus judgment is substantially less secure than the order judgment; no specific letters, keywords, or indentation rules were identified.

corpus: python  
order: as is  
probabilities: python=0.52 romaji=0.08 english=0.40; as is=0.99 shuffled=0.01
