OUTPUT of run 3 of at most 100:
```

 W_fc.0 first 48 singular values:
 [18.1104 12.2798  8.4726  7.9016  7.7793  7.442   7.1458  7.1094  7.0083  6.8053  6.7211  6.5398  6.3779  6.3286  6.2698  6.2168  6.2037  6.0345
  6.0235  5.9437  5.8446  5.7365  5.7016  5.6681  5.6093  5.5798  5.498   5.4472  5.3773  5.3341  5.2437  5.203   5.1406  5.1261  4.9896  4.9434
  4.9202  4.8457  4.7874  4.7161  4.6694  4.6008  4.5188  4.4395  4.4082  4.3845  4.3074  4.2416]
Energy fractions at ranks 16,20,22,24,26,28,32,64: [0.4395, 0.4945, 0.5196, 0.5438, 0.5673, 0.5898, 0.6317, 0.8472]

 W_proj.0 first 48 singular values:
 [10.9319  7.576   6.6285  6.2932  6.166   5.9239  5.886   5.7686  5.7059  5.6097  5.4949  5.4626  5.4044  5.3131  5.1933  5.1607  5.1191  5.0699
  5.0033  4.9374  4.8767  4.7926  4.6927  4.6637  4.6514  4.6368  4.5477  4.5258  4.4869  4.4042  4.329   4.2833  4.2468  4.2063  4.1781  4.1409
  4.1201  4.1053  4.0623  4.0225  3.9971  3.9583  3.9344  3.9041  3.8574  3.8499  3.7981  3.7584]
Energy fractions at ranks 16,20,22,24,26,28,32,64: [0.3034, 0.3517, 0.374, 0.3948, 0.4154, 0.435, 0.4715, 0.6879]

 W_fc.1 first 48 singular values:
 [20.7957 14.1429 13.8258 13.4276 13.1222 12.8832 12.7635 12.474  12.289  11.8837 11.7528 11.4702 11.3766 11.2499 10.9086 10.5664 10.4358 10.1111
  9.9044  9.7303  9.4785  9.3223  9.3181  9.0878  8.9269  8.89    8.8311  8.5779  8.5148  8.451   8.3159  8.1094  7.9966  7.8812  7.7992  7.7515
  7.6666  7.492   7.3091  7.2354  7.1804  7.0421  6.9016  6.6692  6.5361  6.4514  6.2559  6.2155]
Energy fractions at ranks 16,20,22,24,26,28,32,64: [0.4206, 0.4833, 0.5107, 0.537, 0.5616, 0.5852, 0.6285, 0.8294]

 W_proj.1 first 48 singular values:
 [7.6249 7.3593 6.9075 6.4887 6.3442 6.2228 6.1924 6.0202 5.9493 5.902  5.8199 5.7562 5.7277 5.6916 5.6739 5.6502 5.633  5.6084 5.5811 5.5672 5.5442
 5.5233 5.479  5.4574 5.4498 5.4081 5.4044 5.3899 5.3745 5.3414 5.3376 5.3089 5.2848 5.2674 5.2573 5.2226 5.1969 5.1933 5.1791 5.1592 5.1544 5.127
 5.1248 5.112  5.0938 5.0829 5.0821 5.0715]
Energy fractions at ranks 16,20,22,24,26,28,32,64: [0.1103, 0.1325, 0.1433, 0.1539, 0.1644, 0.1747, 0.1949, 0.3401]

 W_fc.2 first 48 singular values:
 [14.8285  8.0483  7.9211  7.76    7.6596  7.5865  7.5217  7.5088  7.367   7.3642  7.1778  7.1535  7.0887  7.0697  6.9674  6.9088  6.8907  6.8028
  6.7443  6.739   6.6891  6.6337  6.6059  6.6018  6.5364  6.5158  6.4202  6.3948  6.3714  6.3508  6.3344  6.2827  6.2544  6.2211  6.163   6.1233
  6.0878  6.0704  6.0281  5.9836  5.9701  5.9319  5.8981  5.8675  5.8219  5.8084  5.8002  5.7257]
Energy fractions at ranks 16,20,22,24,26,28,32,64: [0.1655, 0.1947, 0.2088, 0.2226, 0.2361, 0.2491, 0.2745, 0.4416]

 W_proj.2 first 48 singular values:
 [10.0946  8.9847  8.0747  7.9088  7.7913  7.6557  7.588   7.3746  7.2122  7.1441  7.0892  7.0344  7.003   6.9509  6.7302  6.7086  6.6299  6.5722
  6.4523  6.3853  6.3412  6.3213  6.2519  6.2074  6.0874  6.0002  5.9051  5.8421  5.789   5.7646  5.7176  5.7159  5.6793  5.6364  5.5986  5.562
  5.5049  5.4579  5.415   5.3935  5.3704  5.3605  5.3052  5.2815  5.2674  5.2524  5.238   5.213 ]
Energy fractions at ranks 16,20,22,24,26,28,32,64: [0.1553, 0.1835, 0.1969, 0.2098, 0.222, 0.2335, 0.2555, 0.4011]

 W_fc.3 first 48 singular values:
 [14.3731  8.1763  7.9733  7.8303  7.0359  6.7574  6.7114  6.5939  6.4676  6.3875  6.3466  6.2805  6.2285  6.1792  6.1624  6.0675  5.9953  5.9874
  5.9467  5.895   5.8543  5.816   5.7909  5.7644  5.7533  5.6992  5.6811  5.6518  5.6413  5.6405  5.5883  5.5639  5.5377  5.5088  5.4604  5.4427
  5.4106  5.3865  5.3686  5.3565  5.3281  5.3013  5.2929  5.2783  5.2729  5.2493  5.2308  5.2172]
Energy fractions at ranks 16,20,22,24,26,28,32,64: [0.1466, 0.1698, 0.1809, 0.1919, 0.2026, 0.2131, 0.2337, 0.3757]

 W_proj.3 first 48 singular values:
 [28.1752 20.2934 19.2921 19.1349 18.7081 18.5697 18.1393 17.7759 17.4845 16.5396 15.9131 15.6096 14.9752 14.8725 14.4167 14.3133 13.3269 12.9707
 12.5934 12.3698 12.2251 12.0191 11.3225 11.115  10.7693  9.2267  8.5731  7.929   6.5001  5.2712  3.9657  3.1843  2.7917  2.6972  2.6532  2.5552
  2.511   2.4709  2.3788  2.29    2.25    2.2147  2.1167  2.0811  2.0364  2.0107  1.9843  1.9692]
Energy fractions at ranks 16,20,22,24,26,28,32,64: [0.7207, 0.8115, 0.8521, 0.8869, 0.9147, 0.9335, 0.9468, 0.9659]

MLP0 effective row norm quantiles: [0.2907 0.3287 0.3511 0.3939 1.7096 2.2242 4.3251]
MLP0 largest off-diagonal row cosine quantiles: [0.2331 0.3941 0.4231 0.4629 0.5541 0.8192 0.9855]

L1 QK coefficient cosines:
 [[1.     0.3023 0.3935 0.3265 0.3375 0.2656]
 [0.3023 1.     0.5243 0.5908 0.5677 0.5659]
 [0.3935 0.5243 1.     0.538  0.5781 0.4498]
 [0.3265 0.5908 0.538  1.     0.6072 0.5237]
 [0.3375 0.5677 0.5781 0.6072 1.     0.4791]
 [0.2656 0.5659 0.4498 0.5237 0.4791 1.    ]]
L1 key-linear coefficient cosines:
 [[1.     0.7977 0.8084 0.8013 0.8098 0.7573]
 [0.7977 1.     0.9231 0.9236 0.9207 0.8974]
 [0.8084 0.9231 1.     0.8985 0.9    0.8683]
 [0.8013 0.9236 0.8985 1.     0.9264 0.8987]
 [0.8098 0.9207 0.9    0.9264 1.     0.8831]
 [0.7573 0.8974 0.8683 0.8987 0.8831 1.    ]]
L1 query-linear coefficient cosines:
 [[1.     0.6085 0.7159 0.6376 0.7005 0.5869]
 [0.6085 1.     0.7067 0.872  0.8576 0.8649]
 [0.7159 0.7067 1.     0.784  0.8399 0.7013]
 [0.6376 0.872  0.784  1.     0.8987 0.864 ]
 [0.7005 0.8576 0.8399 0.8987 1.     0.8378]
 [0.5869 0.8649 0.7013 0.864  0.8378 1.    ]]
L1 key-linear versus query-linear cosines:
 [[-0.119  -0.2537 -0.1627 -0.1966 -0.1842 -0.2325]
 [-0.1606 -0.3366 -0.2368 -0.2599 -0.257  -0.3065]
 [-0.1822 -0.3153 -0.245  -0.2725 -0.2535 -0.3097]
 [-0.1535 -0.3143 -0.2374 -0.2855 -0.2597 -0.3162]
 [-0.1668 -0.3182 -0.2516 -0.2776 -0.2824 -0.3267]
 [-0.1599 -0.3014 -0.2403 -0.2662 -0.2612 -0.317 ]]
Layer 0 key-only coefficient stack SV: [0.0796 0.0542 0.047  0.0193 0.0171 0.0009]
Layer 1 key-only coefficient stack SV: [5.7026 0.9101 0.8376 0.7153 0.6953 0.4556]
Layer 2 key-only coefficient stack SV: [2.1896 0.7759 0.6682 0.5513 0.5253 0.4585]
Layer 3 key-only coefficient stack SV: [0.4312 0.3295 0.2685 0.2541 0.2394 0.2101]

```