OUTPUT of run 1 of at most 100:
```
Available weights: ['W_fc.0', 'W_fc.1', 'W_fc.2', 'W_fc.3', 'W_k.0', 'W_k.1', 'W_k.2', 'W_k.3', 'W_o.0', 'W_o.1', 'W_o.2', 'W_o.3', 'W_proj.0', 'W_proj.1', 'W_proj.2', 'W_proj.3', 'W_q.0', 'W_q.1', 'W_q.2', 'W_q.3', 'W_v.0', 'W_v.1', 'W_v.2', 'W_v.3', 'ln1_b.0', 'ln1_b.1', 'ln1_b.2', 'ln1_b.3', 'ln1_w.0', 'ln1_w.1', 'ln1_w.2', 'ln1_w.3', 'ln2_b.0', 'ln2_b.1', 'ln2_b.2', 'ln2_b.3', 'ln2_w.0', 'ln2_w.1', 'ln2_w.2', 'ln2_w.3']

LAYER NORMS: name, norm, mean, std, min, max; largest deviations from initialization
ln1_w.0 [6.72409 0.26771 0.21465 0.01655 1.01343] [(365, 0.01655), (187, 0.0226), (55, 0.02532), (24, 0.03048), (357, 0.03226), (347, 0.03366), (3, 0.03582), (248, 0.04101)]
ln1_b.0 [ 7.1723e-01  6.5000e-04  3.6600e-02 -1.2149e-01  1.4178e-01] [(153, 0.14178), (104, -0.12149), (291, -0.1168), (71, -0.1163), (329, -0.11242), (175, 0.11176), (161, -0.11093), (338, -0.10877)]
ln2_w.0 [17.27234  0.81684  0.33119  0.04681  1.64118] [(95, 0.04681), (311, 0.13376), (333, 0.17013), (377, 0.17466), (109, 0.17753), (21, 0.17766), (342, 0.20886), (303, 0.2185)]
ln2_b.0 [ 1.35297  0.01796  0.06667 -0.55371  0.16628] [(123, -0.55371), (21, -0.48686), (51, -0.45307), (307, -0.16922), (207, 0.16628), (349, 0.15666), (35, 0.15028), (264, 0.1405)]
ln1_w.1 [9.50464 0.47536 0.09639 0.19793 1.20958] [(146, 0.19793), (125, 0.25938), (51, 0.26836), (142, 0.28711), (307, 0.29104), (322, 0.29839), (109, 0.30712), (97, 0.30736)]
ln1_b.1 [ 3.03267  0.02203  0.15318 -1.0505   0.91025] [(123, -1.0505), (21, -1.02415), (307, -0.93456), (51, -0.91482), (125, 0.91025), (35, 0.90413), (360, 0.7445), (87, -0.72746)]
ln2_w.1 [2.005427e+01 1.011090e+00 1.582000e-01 1.391000e-02 1.470660e+00] [(21, 0.01391), (273, 0.02643), (109, 0.30976), (123, 0.33619), (125, 0.49986), (35, 0.50134), (258, 0.51009), (186, 1.47066)]
ln2_b.1 [ 2.87693  0.02753  0.14421 -0.37441  0.36524] [(123, -0.37441), (186, 0.36524), (321, 0.35475), (125, 0.34593), (383, 0.34122), (20, 0.31931), (27, -0.315), (35, 0.31279)]
ln1_w.2 [14.4574   0.73433  0.0712   0.34448  0.90474] [(125, 0.34448), (51, 0.34639), (307, 0.39088), (123, 0.39415), (360, 0.41219), (146, 0.4692), (35, 0.50849), (99, 0.51777)]
ln1_b.2 [ 2.62139  0.02439  0.13153 -0.94167  0.73256] [(21, -0.94167), (123, -0.84128), (51, -0.80833), (125, 0.73256), (307, -0.61286), (35, 0.57679), (360, 0.51205), (273, -0.50597)]
ln2_w.2 [25.08196  1.275    0.11256  0.07753  1.50181] [(273, 0.07753), (7, 1.50181), (21, 0.50071), (214, 1.48408), (198, 1.44415), (83, 1.44083), (200, 1.43113), (52, 1.42931)]
ln2_b.2 [ 2.91097  0.01881  0.14735 -0.41571  0.36355] [(7, -0.41571), (21, -0.39103), (103, 0.36355), (106, 0.34117), (130, 0.33613), (372, -0.32533), (321, 0.32338), (139, -0.32204)]
ln1_w.3 [15.10458  0.76938  0.04688  0.45102  0.87783] [(123, 0.45102), (307, 0.45503), (51, 0.55437), (17, 0.60271), (249, 0.61993), (86, 0.62136), (146, 0.6224), (21, 0.629)]
ln1_b.3 [ 1.86193  0.01021  0.09447 -0.51407  0.62854] [(146, 0.62854), (51, -0.51407), (123, -0.4276), (307, -0.3847), (125, 0.3566), (86, 0.34005), (21, -0.3213), (292, 0.31999)]
ln2_w.3 [24.65744  1.25516  0.08883  0.2619   1.45679] [(273, 0.2619), (197, 1.45679), (382, 1.41412), (161, 1.39887), (28, 1.38727), (348, 1.38355), (75, 1.38349), (143, 1.38275)]
ln2_b.3 [ 2.49718  0.01399  0.12666 -0.51123  0.38081] [(123, -0.51123), (275, 0.38081), (382, -0.31688), (161, 0.30605), (210, -0.3025), (162, 0.28633), (146, 0.27946), (39, 0.2748)]

MATRIX SPECTRA: name, Frobenius norm, top 6 singular values, stable rank
W_q.0 22.7038 [5.0385 4.9994 4.7025 4.377  4.3428 4.2944] 20.305
W_k.0 23.2143 [6.2042 5.0521 4.5716 4.5372 4.3914 4.2894] 14.001
W_v.0 18.1973 [4.5533 4.0243 3.7382 3.4973 3.4646 3.3672] 15.972
W_o.0 19.5995 [6.678  5.2365 2.8346 2.679  2.6198 2.5808] 8.614
W_fc.0 51.6496 [18.1104 12.2798  8.4726  7.9016  7.7793  7.442 ] 8.134
W_proj.0 45.8056 [10.9319  7.576   6.6285  6.2932  6.166   5.9239] 17.557
W_q.1 31.6763 [8.9845 7.1373 6.0786 5.874  5.1897 5.1024] 12.43
W_k.1 28.6017 [10.9672  9.4479  5.438   4.842   4.5394  4.4806] 6.801
W_v.1 28.4686 [5.3505 5.0645 4.9787 4.8191 4.6901 4.5848] 28.31
W_o.1 26.151 [4.1901 3.6511 3.3848 3.3517 3.2401 3.2159] 38.953
W_fc.1 80.2609 [20.7957 14.1429 13.8258 13.4276 13.1222 12.8832] 14.896
W_proj.1 75.1193 [7.6249 7.3593 6.9075 6.4887 6.3442 6.2228] 97.059
W_q.2 37.4829 [5.0672 4.7425 4.4563 4.3901 4.3139 4.2604] 54.718
W_k.2 36.3557 [8.2935 7.3483 5.0588 4.4667 4.3056 3.9124] 19.216
W_v.2 34.7465 [3.89   3.8245 3.7912 3.6543 3.6442 3.6283] 79.785
W_o.2 34.2053 [4.4416 4.0726 3.768  3.4554 3.4401 3.3649] 59.308
W_fc.2 79.4541 [14.8285  8.0483  7.9211  7.76    7.6596  7.5865] 28.71
W_proj.2 77.4765 [10.0946  8.9847  8.0747  7.9088  7.7913  7.6557] 58.906
W_q.3 33.9401 [4.1544 3.9457 3.875  3.6008 3.5209 3.4277] 66.743
W_k.3 33.2214 [4.8875 3.9279 3.6197 3.4371 3.3813 3.2868] 46.201
W_v.3 38.9389 [4.6968 4.3562 4.2546 4.2082 4.1706 4.1314] 68.734
W_o.3 39.0173 [9.1753 6.6297 4.4421 4.1112 4.0543 4.0399] 18.083
W_fc.3 78.1887 [14.3731  8.1763  7.9733  7.8303  7.0359  6.7574] 29.593
W_proj.3 85.0885 [28.1752 20.2934 19.2921 19.1349 18.7081 18.5697] 9.12

HEAD WEIGHT GEOMETRY
QK = W_q[head].T @ W_k[head]; OV = W_o[:,head] @ W_v[head].
Each line: ||Q||,||K||,||V||,||O||; QK top 5 SV, stable rank, trace/norm, symmetric-energy fraction; OV equivalents.
L0H0 [4.128 4.913 5.352 5.205] QK ([5.195, 3.389, 0.846, 0.657, 0.509], np.float64(1.545), np.float64(-0.655), np.float64(0.577)) OV ([6.898, 3.897, 1.936, 1.638, 1.469], np.float64(1.579), np.float64(0.187), np.float64(0.524))
L0H1 [12.463 12.643  9.257 10.205] QK ([17.223, 12.122, 11.496, 9.727, 8.594], np.float64(3.568), np.float64(-1.267), np.float64(0.582)) OV ([4.703, 4.573, 4.424, 4.248, 4.199], np.float64(13.166), np.float64(-0.048), np.float64(0.502))
L0H2 [6.458 6.828 5.99  5.536] QK ([4.486, 3.573, 2.993, 2.852, 2.588], np.float64(4.101), np.float64(-0.849), np.float64(0.588)) OV ([4.786, 3.621, 3.514, 2.916, 2.224], np.float64(3.222), np.float64(-0.086), np.float64(0.501))
L0H3 [6.627 6.822 4.882 5.466] QK ([7.675, 5.606, 3.45, 2.355, 2.231], np.float64(2.233), np.float64(-0.798), np.float64(0.543)) OV ([2.168, 1.997, 1.875, 1.555, 1.484], np.float64(5.949), np.float64(-0.013), np.float64(0.503))
L0H4 [2.173 2.623 8.285 8.942] QK ([0.319, 0.242, 0.19, 0.144, 0.137], np.float64(5.859), np.float64(-0.613), np.float64(0.52)) OV ([18.815, 15.373, 5.587, 3.989, 2.502], np.float64(1.88), np.float64(0.259), np.float64(0.52))
L0H5 [15.898 15.965  9.405 10.603] QK ([14.076, 13.759, 13.195, 12.466, 12.345], np.float64(10.809), np.float64(1.126), np.float64(0.386)) OV ([5.61, 5.503, 5.123, 4.867, 4.799], np.float64(10.941), np.float64(-0.614), np.float64(0.504))
L1H0 [10.561  9.834 12.951 11.168] QK ([8.928, 6.891, 6.051, 5.755, 5.309], np.float64(4.901), np.float64(-0.642), np.float64(0.521)) OV ([9.927, 9.271, 8.872, 7.793, 7.506], np.float64(7.677), np.float64(-1.257), np.float64(0.544))
L1H1 [14.533 12.801 11.065 10.657] QK ([19.138, 14.766, 13.796, 11.021, 8.543], np.float64(3.95), np.float64(-1.249), np.float64(0.541)) OV ([6.211, 5.658, 5.403, 5.113, 4.867], np.float64(10.328), np.float64(-1.193), np.float64(0.531))
L1H2 [12.687 11.35  12.119 11.189] QK ([14.486, 12.667, 10.219, 7.24, 6.462], np.float64(3.991), np.float64(-0.979), np.float64(0.535)) OV ([6.799, 6.707, 6.454, 6.192, 6.0], np.float64(12.16), np.float64(-1.347), np.float64(0.54))
L1H3 [12.697 11.75  11.318 10.141] QK ([15.986, 13.278, 10.22, 7.207, 6.367], np.float64(3.57), np.float64(-0.871), np.float64(0.531)) OV ([7.119, 6.39, 5.875, 5.68, 5.417], np.float64(8.574), np.float64(-1.078), np.float64(0.527))
L1H4 [12.592 11.509 10.916  9.909] QK ([14.823, 11.74, 10.94, 7.505, 6.922], np.float64(3.875), np.float64(-0.775), np.float64(0.526)) OV ([6.354, 5.996, 5.642, 5.541, 5.215], np.float64(9.489), np.float64(-1.121), np.float64(0.523))
L1H5 [14.14  12.576 11.233 10.925] QK ([20.182, 16.183, 12.284, 11.201, 8.546], np.float64(3.463), np.float64(-1.184), np.float64(0.536)) OV ([5.781, 5.643, 5.336, 4.976, 4.846], np.float64(12.395), np.float64(-1.361), np.float64(0.536))
L2H0 [15.223 14.526 13.812 13.642] QK ([7.484, 7.099, 6.816, 6.471, 5.96], np.float64(16.09), np.float64(-0.241), np.float64(0.497)) OV ([5.556, 5.132, 5.005, 4.795, 4.773], np.float64(20.518), np.float64(-1.919), np.float64(0.54))
L2H1 [14.685 14.475 14.683 14.212] QK ([7.258, 6.545, 5.975, 5.73, 5.396], np.float64(15.831), np.float64(1.308), np.float64(0.52)) OV ([8.652, 6.394, 6.333, 6.217, 6.016], np.float64(11.146), np.float64(-1.882), np.float64(0.536))
L2H2 [17.263 16.724 12.989 13.004] QK ([10.83, 10.274, 9.752, 9.397, 9.181], np.float64(14.873), np.float64(0.566), np.float64(0.517)) OV ([4.742, 4.585, 4.492, 4.275, 4.22], np.float64(22.749), np.float64(-1.766), np.float64(0.535))
L2H3 [15.012 14.613 14.455 14.253] QK ([8.092, 7.027, 6.199, 6.131, 5.692], np.float64(13.006), np.float64(-0.166), np.float64(0.493)) OV ([6.179, 5.765, 5.693, 5.322, 5.194], np.float64(19.999), np.float64(-1.872), np.float64(0.545))
L2H4 [15.576 15.198 13.472 13.505] QK ([11.608, 9.684, 7.179, 6.746, 6.178], np.float64(8.031), np.float64(0.093), np.float64(0.499)) OV ([5.445, 5.184, 5.128, 4.988, 4.903], np.float64(20.721), np.float64(-2.103), np.float64(0.548))
L2H5 [13.843 13.304 15.55  15.076] QK ([6.303, 5.923, 5.478, 4.701, 4.593], np.float64(14.895), np.float64(-0.422), np.float64(0.497)) OV ([8.467, 7.339, 6.951, 6.407, 6.182], np.float64(14.178), np.float64(-1.433), np.float64(0.527))
L3H0 [14.949 14.542 15.592 15.368] QK ([5.595, 5.383, 5.21, 5.13, 5.01], np.float64(25.045), np.float64(0.852), np.float64(0.509)) OV ([7.123, 6.403, 6.09, 5.949, 5.73], np.float64(19.702), np.float64(-1.973), np.float64(0.534))
L3H1 [13.433 13.311 15.607 15.271] QK ([4.398, 4.278, 4.169, 4.085, 4.04], np.float64(27.15), np.float64(1.331), np.float64(0.512)) OV ([9.373, 6.819, 6.701, 6.663, 6.486], np.float64(12.509), np.float64(-1.93), np.float64(0.541))
L3H2 [14.992 14.465 15.917 16.033] QK ([6.246, 5.499, 5.294, 4.868, 4.798], np.float64(19.8), np.float64(0.353), np.float64(0.496)) OV ([11.577, 7.787, 6.815, 6.5, 6.372], np.float64(8.952), np.float64(-1.879), np.float64(0.537))
L3H3 [10.537 10.518 17.864 18.199] QK ([3.686, 3.263, 2.903, 2.724, 2.622], np.float64(14.988), np.float64(0.002), np.float64(0.495)) OV ([18.625, 12.287, 11.952, 11.362, 11.111], np.float64(6.83), np.float64(-1.346), np.float64(0.533))
L3H4 [13.655 13.261 16.266 16.256] QK ([5.722, 5.04, 4.858, 4.363, 4.209], np.float64(16.786), np.float64(0.012), np.float64(0.502)) OV ([11.774, 10.138, 8.487, 7.578, 7.163], np.float64(9.686), np.float64(-1.232), np.float64(0.52))
L3H5 [15.026 14.808 13.874 14.159] QK ([5.408, 5.262, 5.14, 5.017, 4.936], np.float64(28.066), np.float64(3.817), np.float64(0.61)) OV ([13.046, 7.099, 6.721, 6.521, 6.348], np.float64(5.51), np.float64(-2.408), np.float64(0.605))

```