OUTPUT of run 3 of at most 100:
```

LAYER 0
W_fc first 40 SV: [18.176 11.018  8.737  7.452  7.212  7.062  7.011  6.62   6.57   6.42   6.343  6.29   6.137  6.043  5.974  5.918  5.807  5.779  5.583  5.513  5.5
  5.36   5.16   5.12   5.086  5.032  4.899  4.879  4.819  4.732  4.601  4.533  4.479  4.441  4.268  4.198  4.031  3.974  3.869  3.824]
SV indices 40,60,80,100,150,200,300,383: [3.72  2.377 1.616 1.25  0.963 0.814 0.596 0.4  ]
W_proj first 40 SV: [9.826 7.351 6.265 5.869 5.797 5.657 5.516 5.45  5.333 5.242 5.197 5.094 4.98  4.904 4.863 4.819 4.793 4.689 4.627 4.595 4.557 4.523 4.411 4.378
 4.283 4.216 4.143 4.125 4.048 4.025 3.894 3.847 3.808 3.792 3.764 3.711 3.685 3.636 3.595 3.525]
SV indices 40,60,80,100,150,200,300,383: [3.499 2.962 2.34  1.767 0.996 0.607 0.343 0.213]

LAYER 1
W_fc first 40 SV: [21.488 13.135 12.804 12.625 12.328 12.118 11.85  11.691 11.43  11.269 11.049 10.846 10.646 10.62  10.324  9.928  9.758  9.699  9.592  9.353  9.219
  9.169  9.074  8.921  8.816  8.729  8.516  8.442  8.433  8.247  8.106  7.853  7.745  7.696  7.617  7.54   7.422  7.349  7.124  7.084]
SV indices 40,60,80,100,150,200,300,383: [6.851 5.28  4.27  3.346 1.842 1.274 0.846 0.411]
W_proj first 40 SV: [8.193 7.894 7.199 6.997 6.671 6.433 6.274 6.133 6.099 6.021 5.942 5.865 5.829 5.789 5.698 5.675 5.646 5.629 5.581 5.564 5.553 5.538 5.528 5.477
 5.456 5.447 5.438 5.403 5.399 5.38  5.367 5.351 5.332 5.327 5.314 5.296 5.252 5.235 5.219 5.208]
SV indices 40,60,80,100,150,200,300,383: [5.197 4.925 4.704 4.479 3.985 3.509 2.629 0.319]

LAYER 2
W_fc first 40 SV: [15.363  8.021  7.895  7.725  7.597  7.49   7.257  7.216  7.141  7.053  6.995  6.974  6.881  6.828  6.8    6.773  6.692  6.634  6.589  6.554  6.512
  6.466  6.411  6.364  6.328  6.312  6.293  6.225  6.17   6.154  6.11   6.101  6.049  6.032  5.963  5.934  5.917  5.91   5.902  5.854]
SV indices 40,60,80,100,150,200,300,383: [5.822 5.301 4.876 4.56  3.917 3.354 2.38  0.645]
W_proj first 40 SV: [9.404 9.187 8.723 8.47  8.04  7.795 7.614 7.45  7.392 7.253 7.223 7.148 7.082 6.972 6.829 6.733 6.637 6.547 6.46  6.404 6.374 6.307 6.209 6.181
 6.12  6.091 5.989 5.9   5.88  5.833 5.758 5.703 5.685 5.645 5.588 5.559 5.491 5.478 5.467 5.429]
SV indices 40,60,80,100,150,200,300,383: [5.406 4.97  4.657 4.389 3.809 3.329 2.449 0.362]

LAYER 3
W_fc first 40 SV: [14.162  8.125  7.915  7.326  7.148  6.856  6.763  6.691  6.489  6.436  6.387  6.31   6.26   6.177  6.144  6.088  6.066  6.017  5.962  5.934  5.883
  5.855  5.843  5.781  5.752  5.72   5.697  5.657  5.632  5.617  5.593  5.568  5.519  5.517  5.495  5.45   5.431  5.389  5.379  5.361]
SV indices 40,60,80,100,150,200,300,383: [5.341 4.986 4.708 4.465 3.94  3.5   2.678 0.683]
W_proj first 40 SV: [26.309 20.685 19.699 19.261 18.594 18.197 17.862 17.667 17.044 16.749 16.039 15.151 14.756 14.326 14.05  13.5   13.293 12.531 12.401 11.958 11.74
 11.594 10.367  8.351  8.064  7.914  7.461  7.166  5.833  4.072  3.221  3.139  2.842  2.8    2.687  2.571  2.403  2.328  2.244  2.185]
SV indices 40,60,80,100,150,200,300,383: [2.181 1.645 1.422 1.244 0.961 0.765 0.509 0.318]

L0 OV matrix cosine similarities:
[[1.    0.885 0.008 0.163 0.903 0.724]
 [0.885 1.    0.006 0.195 0.889 0.802]
 [0.008 0.006 1.    0.012 0.006 0.017]
 [0.163 0.195 0.012 1.    0.195 0.156]
 [0.903 0.889 0.006 0.195 1.    0.728]
 [0.724 0.802 0.017 0.156 0.728 1.   ]]
L0 QK matrix cosine similarities:
[[ 1.     0.209 -0.016 -0.014  0.115  0.018]
 [ 0.209  1.    -0.004 -0.036  0.209 -0.008]
 [-0.016 -0.004  1.     0.239 -0.006 -0.026]
 [-0.014 -0.036  0.239  1.    -0.054 -0.058]
 [ 0.115  0.209 -0.006 -0.054  1.     0.172]
 [ 0.018 -0.008 -0.026 -0.058  0.172  1.   ]]

L0H2 unweighted QK singular values:
[59.879 57.279 55.581 54.347 51.234 49.452 47.184 44.678 43.21  41.651 39.871 35.376 31.577 26.606 23.538 21.633 19.958 18.271 16.837 14.93  13.224
 10.856  9.22   8.099  8.066  6.534  4.851  4.339  3.972  3.207  3.015  2.925  2.44   2.351  1.993  1.798  1.513  1.451  1.349  1.297  1.132  1.074
  0.976  0.837  0.819  0.709  0.665  0.625]
L0 raw Q/K/V energy in this subspace:
0 [0.2455 0.1965 0.368 ]
1 [0.1627 0.197  0.3512]
2 [0.9298 0.9329 0.1354]
3 [0.3366 0.3524 0.1739]
4 [0.1829 0.1872 0.2346]
5 [0.2132 0.2109 0.3322]

L1 key-bias weight directions and L0 OV outputs
0 [0.4286 0.5531 0.0093 0.0966 0.4738 0.7892]
1 [0.4773 0.5822 0.0093 0.0781 0.5123 0.8106]
2 [0.4795 0.582  0.0179 0.0758 0.5122 0.8098]
3 [0.4589 0.5676 0.0232 0.0876 0.4939 0.799 ]
4 [0.5101 0.5937 0.0332 0.0676 0.5308 0.8157]
5 [0.4521 0.5668 0.0285 0.0787 0.4904 0.7964]

Linear-algebra-only MLP products: normalized trace with later QK
L1H0 [-0.0073]
L1H1 [-0.041]
L1H2 [-0.0388]
L1H3 [-0.0265]
L1H4 [-0.0369]
L1H5 [-0.0265]
L2H0 [0.0028 0.0035]
L2H1 [-0.0083  0.0235]
L2H2 [0.0014 0.0149]
L2H3 [-0.0047  0.0137]
L2H4 [-0.0058  0.0046]
L2H5 [ 0.001  -0.0012]
L3H0 [-0.005  -0.0103  0.0057]
L3H1 [-0.0464 -0.0133 -0.0229]
L3H2 [-0.0034  0.0021  0.0054]
L3H3 [-0.0012 -0.0023  0.012 ]
L3H4 [-0.0011  0.0005  0.0093]
L3H5 [-0.0067 -0.0014  0.0039]

```