COMMON L0 BROAD-HEAD WRITE SUBSPACE: singular values [11.996   8.4336  3.3624  1.312   0.9992  0.8615  0.7301  0.6058  0.5661  0.5402  0.4827  0.436 ]
Fraction of each head's OV energy in that 3D write subspace:
[np.float64(0.976), np.float64(0.9754), np.float64(0.0071), np.float64(0.3208), np.float64(0.923), np.float64(0.971)]
Top coordinates of its three orthonormal directions:
0 [(222, 0.9268), (78, 0.2902), (360, -0.1298), (72, -0.126), (254, -0.0778), (25, 0.0414), (197, -0.0354), (103, 0.0353)]
1 [(78, 0.7045), (360, 0.4506), (72, -0.3277), (103, -0.2017), (254, 0.2004), (25, -0.1744), (222, -0.1615), (273, -0.1318)]
2 [(72, -0.4814), (103, 0.4745), (360, -0.4638), (78, 0.1951), (222, -0.1875), (273, -0.1063), (130, 0.1011), (307, 0.1)]

LATER QK FRACTION USING COMMON 3D WRITE SUBSPACE
query-side, key-side, both-sides; bias-derived key coefficient fraction in this subspace
L1H0 [0.0678 0.6412 0.0555 0.8698]
3D coefficient matrix [[0.0519, -0.1085, -0.0667], [0.1361, 0.0631, 0.0309], [0.0217, 0.0219, 0.0196]] key-linear coefficients [0.8921, 0.7926, 0.4225]
L1H1 [0.0756 0.6003 0.0625 0.9031]
3D coefficient matrix [[0.0621, -0.1722, -0.103], [0.221, 0.0837, 0.0433], [0.0624, 0.0402, 0.0302]] key-linear coefficients [1.637, 1.254, 0.6797]
L1H2 [0.069  0.634  0.0601 0.9041]
3D coefficient matrix [[0.0548, -0.1987, -0.1216], [0.2606, 0.0802, 0.0387], [0.0943, 0.0414, 0.0292]] key-linear coefficients [1.8573, 1.4128, 0.7806]
L1H3 [0.0676 0.6351 0.0565 0.8899]
3D coefficient matrix [[0.0568, -0.1671, -0.1069], [0.2241, 0.0806, 0.0392], [0.0652, 0.0315, 0.0309]] key-linear coefficients [1.6221, 1.3243, 0.731]
L1H4 [0.0646 0.6915 0.0547 0.9054]
3D coefficient matrix [[0.0621, -0.1784, -0.1092], [0.234, 0.0729, 0.0346], [0.061, 0.0404, 0.0309]] key-linear coefficients [1.7177, 1.1775, 0.6418]
L1H5 [0.0784 0.6307 0.0642 0.8894]
3D coefficient matrix [[0.0676, -0.1796, -0.1128], [0.2279, 0.0752, 0.0385], [0.0585, 0.0454, 0.0312]] key-linear coefficients [1.2025, 0.9804, 0.5698]
L2H0 [0.0309 0.097  0.0234 0.8566]
L2H1 [0.0106 0.0469 0.0069 0.7939]
L2H2 [0.0254 0.1854 0.0183 0.8053]
L2H3 [0.0418 0.0967 0.0332 0.8732]
L2H4 [0.0338 0.098  0.0263 0.8644]
L2H5 [0.0345 0.0928 0.0266 0.8574]
L3H0 [0.0026 0.0136 0.0007 0.1503]
L3H1 [0.0043 0.0033 0.0004 0.0366]
L3H2 [0.0147 0.071  0.0079 0.5941]
L3H3 [0.0184 0.0671 0.0115 0.6708]
L3H4 [0.007  0.0452 0.0035 0.52  ]
L3H5 [0.0093 0.0106 0.0005 0.0378]

L0H2 RAW OV SINGULAR VALUES
[4.0009 3.853  3.7432 3.5437 3.4491 3.313  3.2443 3.1768 3.0968 2.9694 2.8864 2.7299 2.6846 2.4768 2.4314 2.0744 1.9734 1.7097 1.3762 1.1317 1.0096 0.97   0.8126 0.7541 0.6761
 0.6298 0.564  0.5469 0.4967 0.4718 0.4228 0.3839 0.3639 0.3453 0.3144 0.2761 0.2338 0.2223 0.1995 0.1695 0.1627 0.1369 0.1276 0.1113 0.1091 0.0945 0.0924 0.0748 0.0696 0.0642
 0.0588 0.0557 0.0526 0.0455 0.0437 0.04   0.0376 0.035  0.0327 0.0314 0.0283 0.0276 0.0243 0.0212]

MLP ROW CROSS-COVARIANCES IN L0H2 READ SUBSPACES
X = M T; Y = M B T. This compares weight-row geometry, without evaluating any neuron.
gain=False, rank=16: Frobenius_cos=0.1478, sym_generalized_eigs=[-0.344 -0.223 -0.058 -0.017  0.03   0.093  0.131  0.159  0.178  0.208  0.251  0.296  0.317  0.347  0.377  0.413]
gain=False, rank=24: Frobenius_cos=0.1313, sym_generalized_eigs=[-0.42  -0.337 -0.312 -0.23  -0.192 -0.122 -0.07  -0.043  0.042  0.055  0.095  0.096  0.145  0.172  0.192  0.222  0.234  0.271  0.336  0.363  0.374  0.45   0.461  0.548]
gain=False, rank=28: Frobenius_cos=0.1279, sym_generalized_eigs=[-0.591 -0.529 -0.402 -0.374 -0.325 -0.242 -0.2   -0.168 -0.097 -0.079 -0.018  0.057  0.072  0.093  0.104  0.153  0.182  0.211  0.22   0.255  0.269  0.336  0.385  0.407  0.427
  0.517  0.58   0.649]
gain=False, rank=40: Frobenius_cos=0.1228, sym_generalized_eigs=[-0.79  -0.74  -0.679 -0.634 -0.607 -0.59  -0.512 -0.408 -0.4   -0.305 -0.287 -0.272 -0.229 -0.211 -0.179 -0.12  -0.092 -0.057 -0.018  0.021  0.027  0.076  0.089  0.142  0.204
  0.218  0.267  0.294  0.319  0.373  0.401  0.441  0.47   0.481  0.581  0.595  0.649  0.705  0.724  0.828]
gain=True, rank=16: Frobenius_cos=0.1322, sym_generalized_eigs=[-0.384 -0.246 -0.126 -0.07  -0.014  0.024  0.133  0.166  0.18   0.21   0.218  0.282  0.307  0.349  0.402  0.476]
gain=True, rank=24: Frobenius_cos=0.1347, sym_generalized_eigs=[-0.456 -0.408 -0.324 -0.307 -0.214 -0.182 -0.108 -0.075 -0.031 -0.01   0.043  0.075  0.168  0.219  0.237  0.257  0.282  0.311  0.36   0.407  0.451  0.459  0.53   0.673]
gain=True, rank=28: Frobenius_cos=0.1291, sym_generalized_eigs=[-0.698 -0.621 -0.433 -0.333 -0.328 -0.312 -0.25  -0.217 -0.158 -0.127 -0.083 -0.038 -0.011  0.038  0.112  0.183  0.212  0.249  0.268  0.297  0.333  0.363  0.396  0.448  0.502
  0.528  0.625  0.71 ]
gain=True, rank=40: Frobenius_cos=0.1242, sym_generalized_eigs=[-0.802 -0.72  -0.68  -0.642 -0.592 -0.573 -0.508 -0.501 -0.421 -0.366 -0.326 -0.306 -0.267 -0.225 -0.169 -0.153 -0.108 -0.071 -0.023  0.017  0.048  0.091  0.122  0.139  0.187
  0.198  0.222  0.318  0.359  0.415  0.422  0.477  0.504  0.528  0.541  0.642  0.66   0.725  0.769  0.878]
