OUTPUT of run 2 of at most 100:
```
Layer 0 singular spectra (first 60, then indices 80, 120, 200, 300, 383):
W_q [4.368 3.131 3.099 3.042 2.994 2.951 2.926 2.881 2.859 2.857 2.804 2.766 2.762 2.692 2.683 2.661 2.637 2.588 2.539 2.509 2.458 2.433 2.403 2.299
 2.272 2.173 2.098 2.048 1.954 1.887 1.877 1.865 1.738 1.725 1.665 1.605 1.552 1.535 1.476 1.424 1.378 1.341 1.306 1.264 1.24  1.213 1.154 1.122
 1.109 1.068 1.053 1.031 1.017 1.007 0.985 0.957 0.944 0.913 0.888 0.879]
tail [0.6556 0.4899 0.3045 0.1303 0.0006]
W_k [4.163 3.952 3.467 3.427 3.364 3.335 3.279 3.256 3.214 3.163 3.115 3.029 2.977 2.901 2.881 2.832 2.798 2.76  2.658 2.584 2.528 2.449 2.353 2.301
 2.281 2.206 2.121 2.074 1.917 1.896 1.813 1.753 1.713 1.676 1.573 1.53  1.47  1.431 1.415 1.355 1.3   1.26  1.187 1.169 1.161 1.116 1.108 1.104
 1.072 1.032 1.003 0.988 0.965 0.953 0.931 0.911 0.907 0.885 0.873 0.866]
tail [0.6608 0.5097 0.317  0.1399 0.0003]
W_v [5.208 3.364 3.006 2.962 2.815 2.639 2.527 2.46  2.37  2.301 2.25  2.23  2.151 2.135 2.044 2.039 2.001 1.981 1.91  1.875 1.822 1.773 1.749 1.708
 1.634 1.611 1.556 1.487 1.453 1.402 1.268 1.213 1.187 1.13  1.109 1.08  1.026 0.992 0.987 0.959 0.933 0.931 0.908 0.892 0.887 0.866 0.864 0.853
 0.843 0.827 0.811 0.793 0.783 0.781 0.759 0.748 0.721 0.715 0.71  0.701]
tail [0.5718 0.4517 0.2931 0.1256 0.0007]
W_o [7.361 6.099 3.167 2.452 2.321 2.202 2.171 2.16  2.105 2.072 2.021 1.995 1.975 1.967 1.935 1.922 1.905 1.885 1.857 1.84  1.815 1.784 1.759 1.744
 1.712 1.694 1.634 1.611 1.585 1.553 1.511 1.466 1.456 1.446 1.397 1.392 1.37  1.336 1.321 1.305 1.294 1.27  1.248 1.218 1.197 1.182 1.167 1.148
 1.136 1.099 1.095 1.077 1.049 1.044 1.011 1.005 0.998 0.985 0.966 0.941]
tail [0.6103 0.3674 0.1734 0.059  0.0002]
W_fc [18.118 12.76  11.337  9.323  9.267  9.021  8.898  8.817  8.609  8.4    8.189  8.06   7.819  7.724  7.62   7.543  7.226  7.081  6.979  6.833  6.779
  6.682  6.592  6.402  6.39   6.306  6.207  6.18   5.974  5.864  5.789  5.741  5.646  5.579  5.512  5.353  5.196  5.193  5.122  5.053  5.029  4.849
  4.646  4.573  4.452  4.163  4.133  3.899  3.888  3.557  3.455  3.347  3.241  3.008  2.921  2.833  2.796  2.657  2.612  2.555]
tail [1.7122 1.2795 0.9228 0.6587 0.4311]
W_proj [10.899  7.702  7.173  6.638  6.541  6.288  6.201  6.133  5.941  5.917  5.758  5.641  5.552  5.476  5.393  5.363  5.259  5.206  5.171  5.153  5.057
  5.018  4.96   4.88   4.821  4.788  4.77   4.73   4.643  4.579  4.546  4.529  4.505  4.493  4.44   4.415  4.369  4.315  4.293  4.261  4.243  4.185
  4.134  4.104  4.064  4.049  4.026  3.988  3.967  3.95   3.933  3.901  3.857  3.804  3.778  3.754  3.74   3.713  3.692  3.683]
tail [3.1846 2.4346 1.3916 0.6756 0.2813]

Layer 0 individual-head spectra (top 12):
H 0 QK [0.8532 0.6126 0.3681 0.2867 0.2786 0.2441 0.2317 0.2099 0.2016 0.1885 0.1818 0.1728] OV [3.6115 3.3924 2.8811 2.3143 2.1276 1.8367 1.6785 1.5294 1.377  1.3026 1.1515 1.1013]
H 1 QK [0.838  0.3105 0.2193 0.1245 0.0908 0.0863 0.0659 0.0389 0.0378 0.0336 0.0332 0.0297] OV [5.7206 3.3585 1.1458 0.5449 0.4435 0.3867 0.3839 0.3541 0.331  0.3111 0.2885 0.2674]
H 2 QK [0.5432 0.1831 0.1574 0.1329 0.0717 0.0659 0.0464 0.0456 0.0362 0.0332 0.0321 0.0292] OV [3.9425 2.3569 1.0061 0.5213 0.5    0.4496 0.3863 0.3309 0.3101 0.3051 0.2774 0.2548]
H 3 QK [1.114  1.0924 1.0705 1.064  1.0301 0.9924 0.9677 0.9537 0.9464 0.9002 0.8883 0.8685] OV [4.4261 3.911  3.6812 3.5972 3.5958 3.4449 3.343  3.2829 3.0733 3.0135 2.9723 2.9144]
H 4 QK [0.5096 0.2267 0.1852 0.1237 0.1013 0.0658 0.0655 0.0489 0.0484 0.0454 0.0401 0.035 ] OV [2.4485 1.3166 0.666  0.4611 0.4377 0.3631 0.3568 0.3142 0.2959 0.2618 0.2484 0.2372]
H 5 QK [0.1805 0.0481 0.0241 0.0205 0.0202 0.0191 0.0188 0.0181 0.0179 0.0172 0.0169 0.0164] OV [24.7988 12.8996  5.8492  4.2294  3.6654  2.3878  1.919   1.6636  1.3942  1.1336  0.8624  0.7865]

Layer 0 QK Frobenius cosines
[[ 1.     0.032  0.067 -0.132  0.114  0.017]
 [ 0.032  1.     0.58  -0.022  0.419 -0.008]
 [ 0.067  0.58   1.    -0.028  0.451 -0.053]
 [-0.132 -0.022 -0.028  1.    -0.032 -0.013]
 [ 0.114  0.419  0.451 -0.032  1.     0.029]
 [ 0.017 -0.008 -0.053 -0.013  0.029  1.   ]]
Layer 0 OV Frobenius cosines
[[ 1.     0.026  0.027 -0.041  0.033  0.037]
 [ 0.026  1.     0.801 -0.004  0.7    0.415]
 [ 0.027  0.801  1.     0.005  0.668  0.52 ]
 [-0.041 -0.004  0.005  1.     0.009 -0.013]
 [ 0.033  0.7    0.668  0.009  1.     0.369]
 [ 0.037  0.415  0.52  -0.013  0.369  1.   ]]

LayerNorm largest absolute bias coordinates:
ln1_b.0 [(312, -0.09539999812841415), (146, 0.09059999883174896), (12, 0.08560000360012054), (104, -0.08410000056028366), (289, 0.08259999752044678), (156, -0.07800000160932541), (303, -0.07760000228881836), (328, 0.07620000094175339)]
ln2_b.0 [(99, -0.6236000061035156), (58, 0.4636000096797943), (153, 0.21320000290870667), (39, -0.20200000703334808), (18, 0.18160000443458557), (29, -0.1720000058412552), (383, -0.16519999504089355), (131, 0.16019999980926514)]
ln1_b.1 [(39, -1.5676000118255615), (58, 1.3651000261306763), (99, -1.3355000019073486), (184, -0.5616999864578247), (383, -0.46219998598098755), (21, -0.42989999055862427), (153, 0.2969000041484833), (266, 0.27730000019073486)]
ln2_b.1 [(58, 0.4169999957084656), (375, 0.3919999897480011), (76, -0.38920000195503235), (39, -0.36880001425743103), (319, 0.35519999265670776), (23, -0.3400999903678894), (358, 0.3149000108242035), (123, 0.31470000743865967)]
ln1_b.2 [(99, -1.6589000225067139), (58, 1.3317999839782715), (39, -1.1991000175476074), (383, -0.5070000290870667), (184, -0.5045999884605408), (21, -0.45339998602867126), (63, -0.3244999945163727), (266, 0.29910001158714294)]
ln2_b.2 [(236, -0.5170000195503235), (73, -0.48030000925064087), (322, 0.4489000141620636), (214, -0.42320001125335693), (97, 0.40139999985694885), (382, -0.39169999957084656), (184, -0.3831999897956848), (364, 0.3749000132083893)]
ln1_b.3 [(58, 0.8241000175476074), (99, -0.7638999819755554), (5, 0.4359999895095825), (383, -0.39739999175071716), (266, 0.3425000011920929), (21, -0.31520000100135803), (184, -0.2888999879360199), (45, 0.2888000011444092)]
ln2_b.3 [(21, -0.5401999950408936), (275, 0.39320001006126404), (215, -0.35199999809265137), (382, -0.3296999931335449), (364, 0.31869998574256897), (168, 0.31369999051094055), (86, -0.3041999936103821), (22, -0.3012000024318695)]

Affine QK algebra using learned LN scale and bias:
head | ||A||, top SV, stable rank, tr(A), ||key-linear coefficient||, coefficient fraction in top key singular direction
L0H0 [ 0.2934  0.2481  1.3984 -0.1047  0.0429  0.6945]
L0H1 [ 0.159   0.1502  1.1208 -0.0985  0.0125  0.6069]
L0H2 [ 0.1074  0.1024  1.1007 -0.0873  0.0071  0.8765]
L0H3 [0.6976 0.2435 8.2055 1.3415 0.0428 0.1444]
L0H4 [ 0.0885  0.0796  1.2366 -0.0459  0.0167  0.9422]
L0H5 [ 0.0339  0.0322  1.1088 -0.0086  0.0053  0.9713]
L1H0 [ 1.9781  1.1849  2.7872 -1.7482  3.9745  0.9862]
L1H1 [ 1.8428  1.1538  2.5506 -1.5574  3.6885  0.9761]
L1H2 [ 1.7812  1.1758  2.2947 -1.8874  3.0513  0.9744]
L1H3 [ 1.7874  1.1874  2.2662 -1.5116  3.3904  0.9776]
L1H4 [ 1.6294  1.059   2.3672 -2.0109  2.3285  0.9148]
L1H5 [ 1.8633  1.1462  2.6427 -1.6298  3.895   0.9789]
L2H0 [ 1.6311  0.6887  5.6082 -1.5548  2.02    0.9774]
L2H1 [ 2.1541  0.5775 13.9127 -0.2158  1.8804  0.88  ]
L2H2 [ 1.7201  0.518  11.0249 -1.2972  1.6082  0.9745]
L2H3 [ 1.623   0.502  10.4524 -0.5951  1.0595  0.7388]
L2H4 [ 2.2151  0.8129  7.4259 -0.979   2.3054  0.9497]
L2H5 [ 2.2641  0.6152 13.5436 -0.2664  2.0234  0.926 ]
L3H0 [ 2.0288  0.4354 21.7128  2.1778  0.5004  0.7025]
L3H1 [ 1.2158  0.3451 12.4148  0.4312  0.2796  0.4056]
L3H2 [ 1.4366  0.383  14.0691  0.3494  0.4953  0.8805]
L3H3 [ 1.8151  0.3215 31.869   5.8301  0.2202  0.205 ]
L3H4 [0.6225 0.2358 6.9675 0.0021 0.1441 0.2275]
L3H5 [ 1.6651  0.4542 13.4423  1.4368  0.4934  0.861 ]

```