OUTPUT of run 23 of at most 60:
```
NEW VOCABULARY COHORTS
Token-ID bands are not asserted to be frequency bands.
IDs_0_16000 rows: 1024
IDs_16000_64000 rows: 1024
IDs_64000_128000 rows: 1024
IDs_128000_262144 rows: 1024
Template order: ['IDs_0_16000', 'IDs_16000_64000', 'IDs_64000_128000', 'IDs_128000_262144', 'all_words', 'readout_PC4_removed', 'answer_labels']
All-word / PC4-removed template cosine: 0.605644166469574

HEAD-SPECIFIC VERSUS NORMALIZATION-FACTOR DECOMPOSITION
This decomposes polynomial WEIGHT PRODUCTS, not the nonlinear model.
No normalization denominator depending on a hidden state is evaluated.

L17H1: decomposition relative errors QK=1.87e-14, OV=3.46e-16
Per-cohort: QK midpoint / B-A ; OV midpoint / B-A, all *1e4
 IDs_0_16000            :  +3016.599/  -9.0790 ;   +244.729/  -0.1894
 IDs_16000_64000        :  +2880.858/  -7.2322 ;   +215.936/  -0.1006
 IDs_64000_128000       :  +3063.888/  -7.1219 ;   +235.743/  -0.1237
 IDs_128000_262144      :  +3250.693/  -8.7418 ;   +257.580/  -0.1082
 all_words              :  +2763.950/  -6.8092 ;   +211.805/  -0.2631
 readout_PC4_removed    :  +3266.506/ -12.4566 ;   +330.033/  -0.0619
 answer_labels          :   +550.263/  -3.0048 ;    +63.799/  -0.0501
Exact factor contributions: all_words / answer_labels, *1e4
 QK q           : [-6.6244 -2.9688]
 QK k           : [-0.1839 -0.0317]
 QK q_gain      : [-0.0241 -0.0067]
 QK k_gain      : [0.0232 0.0025]
 QK input_gain  : [0. 0.]
 OV o           : [-0.2612 -0.0494]
 OV v           : [-0.0019 -0.0006]
 OV input_gain  : [0. 0.]
 OV output_gain : [-0. -0.]

L17H3: decomposition relative errors QK=3.03e-14, OV=4.91e-16
Per-cohort: QK midpoint / B-A ; OV midpoint / B-A, all *1e4
 IDs_0_16000            :  +2330.871/  -1.7360 ;    -19.425/  -0.0103
 IDs_16000_64000        :  +2254.018/  -0.5806 ;    -16.551/  -0.0305
 IDs_64000_128000       :  +2318.753/  -0.6734 ;    -20.434/  -0.0148
 IDs_128000_262144      :  +2451.617/  -0.4927 ;    -22.471/  -0.0092
 all_words              :  +2134.945/  -0.6426 ;    -15.732/  -0.0306
 readout_PC4_removed    :  +2460.832/  -0.8333 ;    -28.300/  -0.0016
 answer_labels          :   +432.088/  +0.7319 ;     -2.252/  -0.0463
Exact factor contributions: all_words / answer_labels, *1e4
 QK q           : [-1.168  0.389]
 QK k           : [0.5053 0.3441]
 QK q_gain      : [-0.0107 -0.0064]
 QK k_gain      : [0.0308 0.0051]
 QK input_gain  : [0. 0.]
 OV o           : [-0.0343 -0.0052]
 OV v           : [ 0.0037 -0.0411]
 OV input_gain  : [0. 0.]
 OV output_gain : [-0.  0.]

L14H0: decomposition relative errors QK=1.82e-14, OV=7.39e-16
Per-cohort: QK midpoint / B-A ; OV midpoint / B-A, all *1e4
 IDs_0_16000            :   +382.479/  +8.9386 ;    -15.543/  +0.0265
 IDs_16000_64000        :   +362.142/  +8.1729 ;    -13.853/  +0.0026
 IDs_64000_128000       :   +331.969/  +8.4425 ;    -14.211/  -0.0012
 IDs_128000_262144      :   +323.146/  +9.9496 ;    -15.402/  +0.0250
 all_words              :   +328.096/  +8.1912 ;    -12.858/  +0.0225
 readout_PC4_removed    :   +410.735/ +11.8798 ;    -22.686/  +0.0102
 answer_labels          :   +115.225/  +3.9817 ;     -3.353/  -0.0073
Exact factor contributions: all_words / answer_labels, *1e4
 QK q           : [7.3    3.3525]
 QK k           : [0.8904 0.6221]
 QK q_gain      : [0.0033 0.0035]
 QK k_gain      : [-0.0025  0.0033]
 QK input_gain  : [0.     0.0003]
 OV o           : [ 0.0119 -0.0033]
 OV v           : [ 0.0106 -0.004 ]
 OV input_gain  : [ 0. -0.]
 OV output_gain : [0. 0.]

L17H2: decomposition relative errors QK=3.52e-14, OV=4.98e-16
Per-cohort: QK midpoint / B-A ; OV midpoint / B-A, all *1e4
 IDs_0_16000            :  +3181.344/  -1.8531 ;    -39.736/  -0.0522
 IDs_16000_64000        :  +3098.913/  -1.8014 ;    -46.445/  -0.0093
 IDs_64000_128000       :  +3284.297/  -1.4501 ;    -53.980/  -0.0288
 IDs_128000_262144      :  +3496.406/  -2.4623 ;    -61.484/  -0.0082
 all_words              :  +2941.389/  -1.8261 ;    -44.958/  -0.0430
 readout_PC4_removed    :  +3308.547/  -0.2765 ;    -60.621/  +0.0140
 answer_labels          :   +588.223/  -0.0202 ;     +0.642/  +0.1125
Exact factor contributions: all_words / answer_labels, *1e4
 QK q           : [-1.1477  0.101 ]
 QK k           : [-0.6931 -0.1223]
 QK q_gain      : [-0.0072 -0.0044]
 QK k_gain      : [0.0219 0.0054]
 QK input_gain  : [0. 0.]
 OV o           : [-0.0354  0.1012]
 OV v           : [-0.0076  0.0113]
 OV input_gain  : [0. 0.]
 OV output_gain : [0. 0.]

L11H1: decomposition relative errors QK=1.5e-14, OV=3.51e-16
Per-cohort: QK midpoint / B-A ; OV midpoint / B-A, all *1e4
 IDs_0_16000            :   +304.523/  +4.2221 ;   +275.897/  +0.0202
 IDs_16000_64000        :   +281.243/  +3.6445 ;   +247.499/  +0.0408
 IDs_64000_128000       :   +271.192/  +5.8867 ;   +264.201/  +0.0198
 IDs_128000_262144      :   +263.219/  +6.0284 ;   +281.385/  -0.0010
 all_words              :   +246.983/  +4.9372 ;   +239.093/  +0.0062
 readout_PC4_removed    :   +343.310/  +4.2197 ;   +347.144/  +0.0013
 answer_labels          :    +84.507/  +1.3405 ;    +57.870/  +0.0753
Exact factor contributions: all_words / answer_labels, *1e4
 QK q           : [3.4133 1.9809]
 QK k           : [ 1.5138 -0.6436]
 QK q_gain      : [0.0144 0.0009]
 QK k_gain      : [-0.0042  0.0021]
 QK input_gain  : [-0.0001  0.    ]
 OV o           : [0.0161 0.0626]
 OV v           : [-0.0099  0.0128]
 OV input_gain  : [0. 0.]
 OV output_gain : [0. 0.]

The decomposition follows a specified straight line in parameter space.
It is an exact attribution of these algebraic products, not a causal attribution
of model behavior. No model was trained, altered, or executed.

```