OUTPUT of run 1 of at most 100: ``` Keys: ['W_fc.0', 'W_fc.1', 'W_fc.2', 'W_fc.3', 'W_k.0', 'W_k.1', 'W_k.2', 'W_k.3', 'W_o.0', 'W_o.1', 'W_o.2', 'W_o.3', 'W_proj.0', 'W_proj.1', 'W_proj.2', 'W_proj.3', 'W_q.0', 'W_q.1', 'W_q.2', 'W_q.3', 'W_v.0', 'W_v.1', 'W_v.2', 'W_v.3', 'ln1_b.0', 'ln1_b.1', 'ln1_b.2', 'ln1_b.3', 'ln1_w.0', 'ln1_w.1', 'ln1_w.2', 'ln1_w.3', 'ln2_b.0', 'ln2_b.1', 'ln2_b.2', 'ln2_b.3', 'ln2_w.0', 'ln2_w.1', 'ln2_w.2', 'ln2_w.3'] LAYER 0 ln1_w norm,mean,std,min,max: [6.7241 0.2677 0.2147 0.0165 1.0134] ln1_b norm,mean,std,min,max: [ 0.7172 0.0006 0.0366 -0.1215 0.1418] ln2_w norm,mean,std,min,max: [17.2723 0.8168 0.3312 0.0468 1.6412] ln2_b norm,mean,std,min,max: [ 1.353 0.018 0.0667 -0.5537 0.1663] W_q Frobenius: 22.7038 smax,energy1,energy4,energy16,stable_rank: [ 5.0385 0.0493 0.1778 0.5305 20.3046] W_k Frobenius: 23.2143 smax,energy1,energy4,energy16,stable_rank: [ 6.2042 0.0714 0.1958 0.5535 14.0005] W_v Frobenius: 18.1973 smax,energy1,energy4,energy16,stable_rank: [ 4.5533 0.0626 0.1907 0.5408 15.9722] W_o Frobenius: 19.5995 smax,energy1,energy4,energy16,stable_rank: [6.678 0.1161 0.2271 0.4185 8.614 ] W_fc Frobenius: 51.6496 smax,energy1,energy4,energy16,stable_rank: [18.1104 0.1229 0.2298 0.4395 8.1335] W_proj Frobenius: 45.8056 smax,energy1,energy4,energy16,stable_rank: [10.9319 0.057 0.1241 0.3034 17.5569] LAYER 1 ln1_w norm,mean,std,min,max: [9.5046 0.4754 0.0964 0.1979 1.2096] ln1_b norm,mean,std,min,max: [ 3.0327 0.022 0.1532 -1.0505 0.9103] ln2_w norm,mean,std,min,max: [20.0543 1.0111 0.1582 0.0139 1.4707] ln2_b norm,mean,std,min,max: [ 2.8769 0.0275 0.1442 -0.3744 0.3652] W_q Frobenius: 31.6763 smax,energy1,energy4,energy16,stable_rank: [ 8.9845 0.0804 0.2024 0.4647 12.4303] W_k Frobenius: 28.6017 smax,energy1,energy4,energy16,stable_rank: [10.9672 0.147 0.321 0.5501 6.8013] W_v Frobenius: 28.4686 smax,energy1,energy4,energy16,stable_rank: [ 5.3505 0.0353 0.1262 0.3844 28.3105] W_o Frobenius: 26.151 smax,energy1,energy4,energy16,stable_rank: [ 4.1901 0.0257 0.0783 0.2426 38.9526] W_fc Frobenius: 80.2609 smax,energy1,energy4,energy16,stable_rank: [20.7957 0.0671 0.1558 0.4206 14.8956] W_proj Frobenius: 75.1192 smax,energy1,energy4,energy16,stable_rank: [ 7.6249 0.0103 0.0358 0.1103 97.0588] LAYER 2 ln1_w norm,mean,std,min,max: [14.4574 0.7343 0.0712 0.3445 0.9047] ln1_b norm,mean,std,min,max: [ 2.6214 0.0244 0.1315 -0.9417 0.7326] ln2_w norm,mean,std,min,max: [25.082 1.275 0.1126 0.0775 1.5018] ln2_b norm,mean,std,min,max: [ 2.911 0.0188 0.1474 -0.4157 0.3635] W_q Frobenius: 37.4829 smax,energy1,energy4,energy16,stable_rank: [ 5.0672 0.0183 0.0621 0.2013 54.7179] W_k Frobenius: 36.3557 smax,energy1,energy4,energy16,stable_rank: [ 8.2935 0.052 0.1273 0.2535 19.2163] W_v Frobenius: 34.7465 smax,energy1,energy4,energy16,stable_rank: [ 3.89 0.0125 0.0476 0.1701 79.7848] W_o Frobenius: 34.2053 smax,energy1,energy4,energy16,stable_rank: [ 4.4416 0.0169 0.0534 0.1609 59.3076] W_fc Frobenius: 79.4541 smax,energy1,energy4,energy16,stable_rank: [14.8285 0.0348 0.0646 0.1655 28.7103] W_proj Frobenius: 77.4765 smax,energy1,energy4,energy16,stable_rank: [10.0946 0.017 0.0517 0.1553 58.9061] LAYER 3 ln1_w norm,mean,std,min,max: [15.1046 0.7694 0.0469 0.451 0.8778] ln1_b norm,mean,std,min,max: [ 1.8619 0.0102 0.0945 -0.5141 0.6285] ln2_w norm,mean,std,min,max: [24.6574 1.2552 0.0888 0.2619 1.4568] ln2_b norm,mean,std,min,max: [ 2.4972 0.014 0.1267 -0.5112 0.3808] W_q Frobenius: 33.9401 smax,energy1,energy4,energy16,stable_rank: [ 4.1544 0.015 0.0528 0.169 66.7429] W_k Frobenius: 33.2214 smax,energy1,energy4,energy16,stable_rank: [ 4.8875 0.0216 0.0582 0.1695 46.2013] W_v Frobenius: 38.9389 smax,energy1,energy4,energy16,stable_rank: [ 4.6968 0.0145 0.0507 0.1776 68.7339] W_o Frobenius: 39.0173 smax,energy1,energy4,energy16,stable_rank: [ 9.1753 0.0553 0.1082 0.2289 18.0831] W_fc Frobenius: 78.1887 smax,energy1,energy4,energy16,stable_rank: [14.3731 0.0338 0.0652 0.1466 29.593 ] W_proj Frobenius: 85.0885 smax,energy1,energy4,energy16,stable_rank: [28.1752 0.1096 0.2685 0.7207 9.1203] HEADS: raw weight products only; no inputs or forward passes QK = diag(ln1_w) Q.T K diag(ln1_w); OV = O V diag(ln1_w) Fields: QK[Frob,smax,E1,E4,E16,stable_rank,symmetry,trace/Frob]; OV[same]; QK bias norms L0H0 QK [ 0.1137 0.0887 0.6084 0.9803 0.9938 1.6437 0.1808 -0.4465] OV [4.3232 3.427 0.6284 0.9782 0.9978 1.5915 0.1036 0.3853] bias [0.0185 0.0161] L0H1 QK [ 0.878 0.6014 0.4691 0.8225 0.9825 2.1316 -0.0996 -0.5564] OV [ 3.492 0.9977 0.0816 0.2774 0.7829 12.2507 0.0048 0.0475] bias [0.0738 0.1057] L0H2 QK [0.1747 0.132 0.571 0.8741 0.9836 1.7515 0.4027 0.3123] OV [2.0255 1.0554 0.2715 0.735 0.9864 3.6836 0.0091 0.0061] bias [0.0235 0.0795] L0H3 QK [ 0.2396 0.1646 0.4721 0.9452 0.995 2.1184 0.209 -0.781 ] OV [1.6238 1.0268 0.3999 0.6717 0.9638 2.5008 0.0428 0.1465] bias [0.057 0.1489] L0H4 QK [ 0.0093 0.0045 0.2333 0.4424 0.7671 4.2872 0.023 -0.4017] OV [17.164 13.8775 0.6537 0.9945 0.9998 1.5297 0.0696 0.3518] bias [0.0009 0.0014] L0H5 QK [ 0.9716 0.41 0.1781 0.4647 0.9151 5.6164 -0.2279 1.2505] OV [ 3.7103 1.0607 0.0817 0.2937 0.8197 12.2366 -0.0001 -0.6652] bias [0.0509 0.0664] L1H0 QK [ 0.6446 0.3538 0.3013 0.637 0.9406 3.3191 0.02 -0.4775] OV [13.4437 4.9469 0.1354 0.4465 0.9116 7.3855 0.0835 -1.2128] bias [0.8392 0.8416] L1H1 QK [ 1.5863 1.1468 0.5227 0.8237 0.9724 1.9132 -0.003 -0.8032] OV [ 9.5953 2.9286 0.0932 0.316 0.8088 10.7351 0.0596 -1.1678] bias [3.1501 2.1826] L1H2 QK [ 1.2384 0.8093 0.4271 0.8409 0.97 2.3412 -0.0153 -0.5996] OV [11.4977 3.3252 0.0836 0.3059 0.8276 11.9561 0.0717 -1.2953] bias [2.1999 1.8979] L1H3 QK [ 1.3117 0.9143 0.4858 0.851 0.9717 2.0583 -0.0428 -0.5311] OV [10.0448 3.4201 0.1159 0.3632 0.879 8.626 0.0507 -1.0525] bias [2.7194 2.2078] L1H4 QK [ 1.2528 0.852 0.4624 0.8389 0.9728 2.1624 -0.0363 -0.468 ] OV [ 9.5599 3.0704 0.1032 0.363 0.8779 9.6946 0.0441 -1.093 ] bias [2.5182 1.946 ] L1H5 QK [ 1.463 1.0096 0.4762 0.8243 0.9723 2.1001 0.0101 -0.8248] OV [ 9.8289 2.8188 0.0822 0.284 0.795 12.1583 0.0705 -1.3273] bias [2.4602 1.8112] L2H0 QK [ 2.0464 0.5184 0.0642 0.2121 0.5791 15.5805 -0.0029 -0.2845] OV [18.7016 4.0788 0.0476 0.1655 0.5169 21.0226 0.08 -1.9232] bias [0.8951 0.7758] L2H1 QK [ 1.9469 0.4596 0.0557 0.1849 0.5341 17.9419 0.0397 1.292 ] OV [21.4533 6.411 0.0893 0.2322 0.6051 11.1977 0.0711 -1.8751] bias [0.6381 0.3339] L2H2 QK [ 2.8055 0.709 0.0639 0.2346 0.6511 15.6583 0.0319 0.5324] OV [16.7872 3.4921 0.0433 0.1588 0.5117 23.1086 0.0678 -1.7551] bias [1.5837 1.5815] L2H3 QK [ 1.9768 0.5709 0.0834 0.2122 0.5687 11.9897 -0.0118 -0.206 ] OV [20.6209 4.6307 0.0504 0.1741 0.5221 19.8301 0.0902 -1.8701] bias [1.0369 1.2654] L2H4 QK [2.2031 0.7829 0.1263 0.2841 0.6169 7.9187 0.0004 0.0247] OV [18.4176 4.0021 0.0472 0.1749 0.5515 21.1785 0.0964 -2.1041] bias [1.2005 1.3405] L2H5 QK [ 1.6549 0.4205 0.0646 0.2118 0.544 15.4887 -0.0056 -0.4237] OV [23.7711 6.3519 0.0714 0.2127 0.5645 14.0055 0.0526 -1.4279] bias [0.6363 0.4924] L3H0 QK [ 2.0781 0.4054 0.0381 0.1422 0.4558 26.2735 0.0198 0.8438] OV [24.5048 5.5215 0.0508 0.1639 0.4941 19.6967 0.0673 -1.9719] bias [0.2989 0.3023] L3H1 QK [ 1.6954 0.3227 0.0362 0.1359 0.4478 27.6063 0.0219 1.3128] OV [25.729 7.2188 0.0787 0.2031 0.5765 12.7031 0.0819 -1.9283] bias [0.2636 0.2534] L3H2 QK [ 2.0613 0.4339 0.0443 0.1511 0.4553 22.5722 -0.0067 0.3486] OV [26.8659 9.0071 0.1124 0.2368 0.557 8.8968 0.074 -1.8824] bias [0.3424 0.2865] L3H3 QK [ 1.0538 0.2628 0.0622 0.1887 0.4999 16.0832 -0.0072 0.0023] OV [37.9438 14.6757 0.1496 0.3298 0.722 6.6848 0.0658 -1.3482] bias [0.2584 0.1604] L3H4 QK [ 1.735 0.4007 0.0533 0.1758 0.4846 18.7462 0.0041 0.0025] OV [28.4456 9.176 0.1041 0.2782 0.6026 9.61 0.0392 -1.231 ] bias [0.3069 0.2391] L3H5 QK [ 2.1176 0.4042 0.0364 0.1337 0.4409 27.4514 0.2212 3.8 ] OV [23.6935 9.9785 0.1774 0.3245 0.7239 5.638 0.2049 -2.3992] bias [0.3101 0.308 ] ```