OUTPUT of run 9 of at most 100: ``` L0 DOMINANT QK FACTORS H0: top-mode energy=0.8877, query-pos32=0.8954, key-pos32=0.2819 H1: top-mode energy=0.7148, query-pos32=0.7432, key-pos32=0.4198 H2: top-mode energy=0.1204, query-pos32=0.9868, key-pos32=0.9970 H3: top-mode energy=0.8269, query-pos32=0.8838, key-pos32=0.9328 H4: top-mode energy=0.7035, query-pos32=0.5431, key-pos32=0.4386 H5: top-mode energy=0.9578, query-pos32=0.8676, key-pos32=0.5002 absolute cosines of leading query directions [[1. 0.77 0.4272 0.7972 0.1273 0.2389] [0.77 1. 0.5194 0.7382 0.227 0.0475] [0.4272 0.5194 1. 0.4817 0.1129 0.0606] [0.7972 0.7382 0.4817 1. 0.0648 0.0329] [0.1273 0.227 0.1129 0.0648 1. 0.3261] [0.2389 0.0475 0.0606 0.0329 0.3261 1. ]] absolute cosines of leading key directions [[1. 0.264 0.0526 0.0032 0.3793 0.0979] [0.264 1. 0.0191 0.1238 0.2577 0.1429] [0.0526 0.0191 1. 0.513 0.0234 0.1277] [0.0032 0.1238 0.513 1. 0.101 0.0119] [0.3793 0.2577 0.0234 0.101 1. 0.6433] [0.0979 0.1429 0.1277 0.0119 0.6433 1. ]] POSITION-SUBSPACE AND AFFINE WEIGHT COEFFICIENTS query-pos32 fraction; key-linear norm / QK norm; key-linear fraction in common 3D writes L1H0 [0.105 1.5297 0.8698] L1H1 [0.1152 1.7366 0.9031] L1H2 [0.1234 1.6683 0.9041] L1H3 [0.1198 1.7087 0.8899] L1H4 [0.1231 1.5788 0.9054] L1H5 [0.1265 1.3096 0.8894] L2H0 [0.0826 0.6969 0.8566] L2H1 [0.0713 0.3601 0.7939] L2H2 [0.0855 0.4684 0.8053] L2H3 [0.08 0.7009 0.8732] L2H4 [0.0852 0.6729 0.8644] L2H5 [0.0807 0.6703 0.8574] L3H0 [0.0772 0.1156 0.1503] L3H1 [0.0673 0.1319 0.0366] L3H2 [0.0797 0.1923 0.5941] L3H3 [0.0831 0.2253 0.6708] L3H4 [0.0801 0.1655 0.52 ] L3H5 [0.0777 0.1971 0.0378] ACTIVE L0 MLP WEIGHT-ROW CLUSTERS cosine>0.8: components=284, non-singleton neurons=183, largest sizes=[7 5 5 5 5 5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 3 3 3 3 3 3 3] cosine>0.9: components=331, non-singleton neurons=127, largest sizes=[6 4 4 4 4 4 4 4 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 2 2 2 2 2] cosine>0.95: components=355, non-singleton neurons=95, largest sizes=[4 4 4 4 4 4 3 3 3 3 3 3 3 3 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2] cosine>0.98: components=409, non-singleton neurons=8, largest sizes=[2 2 2 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1] Previous-write top24 read-energy fraction quantiles [0.0058 0.0196 0.0675 0.1157 0.1704 0.2296 0.7185] Position top32 read-energy fraction quantiles [0.0021 0.0058 0.0167 0.0289 0.0376 0.0515 0.0909] ```