OUTPUT of run 6 of at most 100: ``` Common L1 key-selector direction: each head's cosine and bias/QK norm ratio 0 [[-0.118 0.25 ] [-0.403 0.113] [ 0.149 0.035] [-0.039 0.136] [-0.141 0.106] [ 0.387 0.21 ]] 1 [[0.98 1.531] [0.987 1.738] [0.989 1.671] [0.985 1.711] [0.984 1.58 ] [0.986 1.312]] 2 [[0.806 0.697] [0.781 0.36 ] [0.786 0.468] [0.814 0.701] [0.804 0.673] [0.814 0.67 ]] 3 [[ 0.318 0.116] [-0.129 0.132] [ 0.621 0.192] [ 0.652 0.225] [ 0.602 0.166] [ 0.08 0.197]] Common selector fraction in top 2/4/8 broad-output dimensions [np.float64(0.824), np.float64(0.9207), np.float64(0.9221)] Common selector composed with L0 OV heads: norms [np.float64(4.2877), np.float64(4.0564), np.float64(0.0362), np.float64(0.1418), np.float64(2.0539), np.float64(7.59)] Projected key-composition scores: strongest earlier heads tuple = (earlier head, trace/F, symmetry, norm) SPACE V0H2-24 L2H0 [('L0H4', np.float64(0.257), np.float64(0.09), np.float64(0.17)), ('L0H0', np.float64(0.149), np.float64(0.05), np.float64(0.443))] L2H1 [('L1H3', np.float64(0.409), np.float64(-0.011), np.float64(0.096)), ('L0H4', np.float64(0.228), np.float64(0.071), np.float64(0.116))] L2H2 [('L1H2', np.float64(0.369), np.float64(0.016), np.float64(0.056)), ('L1H3', np.float64(0.366), np.float64(0.044), np.float64(0.065))] L2H3 [('L0H0', np.float64(0.335), np.float64(0.133), np.float64(0.451)), ('L0H4', np.float64(0.317), np.float64(0.129), np.float64(0.167))] L2H4 [('L0H4', np.float64(0.227), np.float64(0.09), np.float64(0.191)), ('L0H0', np.float64(0.214), np.float64(0.064), np.float64(0.511))] L2H5 [('L1H2', np.float64(0.243), np.float64(-0.066), np.float64(0.1)), ('L0H0', np.float64(0.231), np.float64(0.082), np.float64(0.485))] L3H0 [('L1H0', np.float64(0.089), np.float64(-0.036), np.float64(0.099)), ('L1H2', np.float64(0.066), np.float64(-0.026), np.float64(0.106))] L3H1 [('L0H2', np.float64(2.907), np.float64(0.649), np.float64(0.135)), ('L1H2', np.float64(1.933), np.float64(0.216), np.float64(0.118))] L3H2 [('L0H2', np.float64(0.263), np.float64(0.069), np.float64(0.041)), ('L2H0', np.float64(0.212), np.float64(0.09), np.float64(0.08))] L3H3 [('L1H3', np.float64(0.212), np.float64(0.049), np.float64(0.064)), ('L1H4', np.float64(0.186), np.float64(0.068), np.float64(0.076))] L3H4 [('L1H5', np.float64(0.316), np.float64(0.1), np.float64(0.111)), ('L0H4', np.float64(0.077), np.float64(-0.002), np.float64(0.101))] L3H5 [('L2H3', np.float64(0.705), np.float64(0.173), np.float64(0.045)), ('L1H0', np.float64(0.416), np.float64(0.118), np.float64(0.04))] SPACE MLP0-32 L2H0 [('L0H4', np.float64(0.143), np.float64(0.025), np.float64(0.225)), ('L0H1', np.float64(0.09), np.float64(0.04), np.float64(0.464))] L2H1 [('L1H3', np.float64(0.304), np.float64(0.049), np.float64(0.168)), ('L0H4', np.float64(0.185), np.float64(0.059), np.float64(0.124))] L2H2 [('L1H2', np.float64(0.237), np.float64(0.0), np.float64(0.104)), ('L1H4', np.float64(0.134), np.float64(0.041), np.float64(0.119))] L2H3 [('L0H1', np.float64(0.14), np.float64(0.103), np.float64(0.382)), ('L0H4', np.float64(0.132), np.float64(0.066), np.float64(0.195))] L2H4 [('L0H4', np.float64(0.194), np.float64(0.031), np.float64(0.209)), ('L0H1', np.float64(0.186), np.float64(0.048), np.float64(0.419))] L2H5 [('L1H2', np.float64(0.203), np.float64(-0.012), np.float64(0.185)), ('L0H4', np.float64(0.187), np.float64(0.063), np.float64(0.195))] L3H0 [('L0H4', np.float64(0.182), np.float64(0.023), np.float64(0.083)), ('L0H0', np.float64(0.176), np.float64(0.03), np.float64(0.188))] L3H1 [('L2H1', np.float64(2.027), np.float64(0.274), np.float64(0.241)), ('L0H2', np.float64(1.772), np.float64(0.452), np.float64(0.062))] L3H2 [('L2H2', np.float64(0.216), np.float64(0.003), np.float64(0.142)), ('L2H4', np.float64(0.207), np.float64(0.021), np.float64(0.115))] L3H3 [('L0H2', np.float64(0.394), np.float64(-0.004), np.float64(0.021)), ('L1H3', np.float64(0.386), np.float64(-0.03), np.float64(0.107))] L3H4 [('L1H2', np.float64(0.18), np.float64(-0.005), np.float64(0.13)), ('L2H0', np.float64(0.169), np.float64(0.029), np.float64(0.132))] L3H5 [('L2H2', np.float64(0.638), np.float64(0.137), np.float64(0.091)), ('L2H3', np.float64(0.594), np.float64(0.158), np.float64(0.082))] SPACE MLP0-64 L2H0 [('L0H4', np.float64(0.183), np.float64(0.022), np.float64(0.393)), ('L0H1', np.float64(0.106), np.float64(0.015), np.float64(0.761))] L2H1 [('L0H4', np.float64(0.193), np.float64(0.021), np.float64(0.225)), ('L1H3', np.float64(0.173), np.float64(0.012), np.float64(0.323))] L2H2 [('L0H4', np.float64(0.223), np.float64(0.028), np.float64(0.414)), ('L0H2', np.float64(0.173), np.float64(-0.009), np.float64(0.051))] L2H3 [('L0H4', np.float64(0.147), np.float64(0.026), np.float64(0.327)), ('L0H1', np.float64(0.125), np.float64(0.036), np.float64(0.602))] L2H4 [('L0H4', np.float64(0.266), np.float64(0.015), np.float64(0.376)), ('L0H0', np.float64(0.19), np.float64(-0.014), np.float64(0.86))] L2H5 [('L0H4', np.float64(0.188), np.float64(0.02), np.float64(0.359)), ('L0H0', np.float64(0.137), np.float64(-0.007), np.float64(0.818))] L3H0 [('L0H4', np.float64(0.113), np.float64(0.005), np.float64(0.149)), ('L0H0', np.float64(0.085), np.float64(0.009), np.float64(0.337))] L3H1 [('L0H2', np.float64(2.527), np.float64(0.42), np.float64(0.138)), ('L2H1', np.float64(2.124), np.float64(0.193), np.float64(0.452))] L3H2 [('L0H2', np.float64(0.221), np.float64(0.027), np.float64(0.042)), ('L1H0', np.float64(0.135), np.float64(-0.003), np.float64(0.26))] L3H3 [('L0H2', np.float64(0.308), np.float64(0.006), np.float64(0.044)), ('L1H3', np.float64(0.103), np.float64(0.002), np.float64(0.2))] L3H4 [('L2H5', np.float64(0.144), np.float64(0.009), np.float64(0.271)), ('L1H2', np.float64(0.117), np.float64(-0.008), np.float64(0.256))] L3H5 [('L2H3', np.float64(0.568), np.float64(0.094), np.float64(0.153)), ('L1H0', np.float64(0.364), np.float64(0.068), np.float64(0.168))] SPACE MLP1-64 L2H0 [('L1H1', np.float64(0.019), np.float64(0.033), np.float64(0.191)), ('L1H0', np.float64(-0.044), np.float64(0.053), np.float64(0.263))] L2H1 [('L0H3', np.float64(0.134), np.float64(0.019), np.float64(0.044)), ('L0H4', np.float64(0.038), np.float64(0.02), np.float64(0.127))] L2H2 [('L1H4', np.float64(0.405), np.float64(0.086), np.float64(0.116)), ('L1H0', np.float64(0.266), np.float64(0.053), np.float64(0.212))] L2H3 [('L0H1', np.float64(0.129), np.float64(0.015), np.float64(0.224)), ('L1H0', np.float64(0.127), np.float64(-0.015), np.float64(0.163))] L2H4 [('L0H1', np.float64(0.221), np.float64(0.011), np.float64(0.262)), ('L0H0', np.float64(0.189), np.float64(0.012), np.float64(0.322))] L2H5 [('L1H4', np.float64(0.118), np.float64(0.049), np.float64(0.187)), ('L0H4', np.float64(0.06), np.float64(-0.0), np.float64(0.133))] L3H0 [('L0H4', np.float64(0.105), np.float64(0.002), np.float64(0.094)), ('L0H5', np.float64(0.052), np.float64(0.008), np.float64(0.246))] L3H1 [('L2H1', np.float64(2.796), np.float64(0.367), np.float64(0.452)), ('L2H3', np.float64(1.692), np.float64(0.103), np.float64(0.506))] L3H2 [('L2H0', np.float64(0.203), np.float64(-0.01), np.float64(0.218)), ('L1H0', np.float64(0.183), np.float64(0.023), np.float64(0.112))] L3H3 [('L1H0', np.float64(0.306), np.float64(-0.014), np.float64(0.103)), ('L1H1', np.float64(0.144), np.float64(0.003), np.float64(0.097))] L3H4 [('L0H4', np.float64(0.326), np.float64(0.081), np.float64(0.122)), ('L0H1', np.float64(0.306), np.float64(0.07), np.float64(0.216))] L3H5 [('L2H3', np.float64(0.697), np.float64(0.044), np.float64(0.217)), ('L2H4', np.float64(0.642), np.float64(0.047), np.float64(0.143))] Direct-match versus predecessor-match on V0H2's 24-dimensional read space L1H0 [[-1.058 -0.145 0.125] [-0.266 0.022 0.033] [-1.401 0.322 0.063]] L1H1 [[-1.489 0.037 0.215] [-0.025 0.224 0.074] [-0.682 0.264 0.096]] L1H2 [[-2.134 0.16 0.226] [ 0.636 0.078 0.08 ] [ 0.585 0.025 0.111]] L1H3 [[-1.368 -0.039 0.186] [ 0.674 0.128 0.065] [ 0.574 0.115 0.119]] L1H4 [[-1.568 0.097 0.221] [ 0.688 0.089 0.065] [-0.278 0.076 0.142]] L1H5 [[-1.199 -0.049 0.201] [ 0.201 0.022 0.086] [-0.681 0.047 0.12 ]] L2H0 [[-0.425 -0.178 0.207] [-0.451 -0.014 0.097] [ 0.019 0.11 0.089]] L2H1 [[-0.374 0.088 0.157] [-0.5 0.118 0.071] [ 1.192 0.104 0.095]] L2H2 [[-0.456 -0.18 0.149] [ 0.121 0.031 0.049] [-0.371 0.072 0.057]] L2H3 [[-0.789 -0.266 0.175] [ 0.266 -0.009 0.068] [ 0.417 0.13 0.073]] L2H4 [[-0.459 -0.172 0.209] [-0.578 -0.019 0.08 ] [-0.156 0.032 0.086]] L2H5 [[-0.497 -0.288 0.202] [-0.019 0.013 0.081] [ 0.287 -0.021 0.084]] L3H0 [[ 1.23 0.246 0.157] [-0.128 0.014 0.079] [ 1.039 0.06 0.081]] L3H1 [[ 1.236 0.271 0.139] [ 2.907 0.649 0.135] [-1.302 0.1 0.072]] L3H2 [[ 0.162 -0.208 0.116] [ 0.263 0.069 0.041] [ 0.104 0.099 0.043]] L3H3 [[ 0.015 -0.233 0.115] [-0.016 -0.083 0.043] [ 0.355 0.042 0.049]] L3H4 [[ 0.341 -0.057 0.12 ] [-0.296 0.166 0.056] [ 0.142 -0.052 0.052]] L3H5 [[0.32 0.055 0.047] [0.278 0.061 0.026] [0.123 0.06 0.02 ]] MLP neuron weight-norm quantiles 0 read [0.37 0.389 0.423 0.967 2.255] write [0.172 0.194 0.237 1.272 1.99 ] small-write fraction 0.693 1 read [1.829 1.906 2.004 2.105 2.213] write [1.78 1.847 1.922 2.007 2.085] small-write fraction 0.004 2 read [1.928 1.971 2.01 2.055 2.095] write [1.82 1.888 1.957 2.025 2.091] small-write fraction 0.0 3 read [1.907 1.938 1.976 2.014 2.053] write [1.809 1.921 2.062 2.222 2.396] small-write fraction 0.0 ```