OUTPUT of run 5 of at most 100: ``` Basis dimensions: position candidate, token candidate, low-rank L0 writes 48 20 12 QK squared-energy fractions in query/key subspaces; then bias-key fraction L0H0 [0.791 0.818 0.528 0.058 0.073 0.247 0.169 0.083 0.062] L0H1 [0.702 0.544 0.402 0.051 0.041 0.154 0.097 0.066 0.042] L0H2 [0.777 0.56 0.513 0.036 0.118 0.145 0.046 0.067 0.067] L0H3 [0.984 0.983 0.914 0.001 0.002 0.051 0.079 0.098 0.151] L0H4 [0.676 0.493 0.499 0.078 0.176 0.206 0.095 0.058 0.054] L0H5 [0.816 0.338 0.321 0.017 0.414 0.429 0.238 0.056 0.051] L1H0 [0.128 0.23 0.281 0.098 0.172 0.165 0.17 0.441 0.858] L1H1 [0.141 0.234 0.252 0.098 0.188 0.191 0.18 0.476 0.856] L1H2 [0.131 0.233 0.244 0.086 0.171 0.232 0.152 0.498 0.857] L1H3 [0.153 0.245 0.267 0.07 0.155 0.173 0.171 0.509 0.86 ] L1H4 [0.124 0.221 0.223 0.072 0.141 0.182 0.108 0.459 0.808] L1H5 [0.149 0.236 0.277 0.106 0.18 0.192 0.179 0.463 0.863] L2H0 [0.151 0.203 0.281 0.052 0.082 0.166 0.132 0.272 0.848] L2H1 [0.134 0.149 0.277 0.055 0.075 0.153 0.067 0.121 0.833] L2H2 [0.131 0.157 0.261 0.046 0.073 0.138 0.079 0.15 0.807] L2H3 [0.118 0.134 0.25 0.044 0.065 0.116 0.048 0.114 0.641] L2H4 [0.133 0.169 0.248 0.053 0.073 0.114 0.088 0.189 0.84 ] L2H5 [0.123 0.152 0.272 0.055 0.08 0.121 0.068 0.126 0.861] L3H0 [0.115 0.128 0.274 0.049 0.06 0.083 0.034 0.066 0.72 ] L3H1 [0.125 0.131 0.247 0.048 0.059 0.086 0.037 0.087 0.528] L3H2 [0.121 0.141 0.281 0.048 0.062 0.127 0.044 0.088 0.666] L3H3 [0.112 0.117 0.114 0.07 0.049 0.046 0.027 0.029 0.031] L3H4 [0.123 0.139 0.172 0.057 0.058 0.055 0.046 0.043 0.079] L3H5 [0.118 0.138 0.297 0.049 0.062 0.102 0.035 0.087 0.754] Layer 1 and 2 key-bias coefficient cosines [[1. 0.829 0.853 0.823 0.825 0.86 0.871 0.831 0.837 0.732 0.841 0.838] [0.829 1. 0.851 0.864 0.821 0.861 0.82 0.879 0.811 0.738 0.838 0.86 ] [0.853 0.851 1. 0.83 0.834 0.833 0.849 0.853 0.83 0.688 0.875 0.854] [0.823 0.864 0.83 1. 0.834 0.873 0.812 0.853 0.847 0.736 0.844 0.856] [0.825 0.821 0.834 0.834 1. 0.828 0.873 0.808 0.761 0.753 0.816 0.812] [0.86 0.861 0.833 0.873 0.828 1. 0.846 0.856 0.814 0.733 0.82 0.868] [0.871 0.82 0.849 0.812 0.873 0.846 1. 0.807 0.766 0.703 0.82 0.835] [0.831 0.879 0.853 0.853 0.808 0.856 0.807 1. 0.811 0.719 0.824 0.837] [0.837 0.811 0.83 0.847 0.761 0.814 0.766 0.811 1. 0.699 0.832 0.827] [0.732 0.738 0.688 0.736 0.753 0.733 0.703 0.719 0.699 1. 0.715 0.73 ] [0.841 0.838 0.875 0.844 0.816 0.82 0.82 0.824 0.832 0.715 1. 0.845] [0.838 0.86 0.854 0.856 0.812 0.868 0.835 0.837 0.827 0.73 0.845 1. ]] Largest coordinates of mean normalized L1 key-bias coefficient [(21, 0.9013), (58, 0.1345), (39, -0.1269), (99, -0.0247), (284, 0.0205), (144, -0.0203), (231, -0.0196), (30, -0.0192), (171, -0.0191), (185, -0.019), (210, -0.0187), (315, -0.0182), (246, -0.0175), (97, -0.0174), (351, -0.0173), (308, -0.0169)] Position-subspace operator correlations with powers of L0H3 H 0 [-0.12 0.049 -0.03 0.015 0.044 0.04 0.015] H 1 [-0.132 -0.035 -0.001 0.006 -0.012 -0.01 -0.004] H 2 [-0.158 -0.012 0.011 0.02 0.007 0.001 -0.015] H 3 [ 0.239 1. 0.347 -0.175 -0.319 -0.254 -0.101] H 4 [-0.109 -0.072 0.012 0.039 -0.001 -0.022 -0.03 ] H 5 [-0.084 -0.044 0.022 0.013 -0.027 -0.024 -0.011] MLP0 coefficient coupling in H3 read subspace (generalized eigenvalues) r 16 centered False [ 0.318+0.j 0.19 -0.148j 0.19 +0.148j -0.052-0.173j -0.052+0.173j -0.177+0.j -0.143+0.048j -0.143-0.048j] r 16 centered True [ 0.394+0.j 0.191-0.116j 0.191+0.116j -0.055-0.167j -0.055+0.167j -0.153-0.068j -0.153+0.068j -0.148+0.j ] r 20 centered False [-0.205-0.138j -0.205+0.138j 0.245-0.007j 0.245+0.007j 0.189+0.126j 0.189-0.126j 0.142-0.153j 0.142+0.153j] r 20 centered True [ 0.303+0.j -0.202-0.135j -0.202+0.135j 0.159+0.148j 0.159-0.148j -0.203+0.j 0.183+0.065j 0.183-0.065j] r 24 centered False [ 0.282-0.172j 0.282+0.172j 0.271+0.j -0.161-0.202j -0.161+0.202j 0.16 -0.172j 0.16 +0.172j -0.219+0.j ] r 24 centered True [ 0.316-0.124j 0.316+0.124j 0.294+0.j -0.157-0.203j -0.157+0.203j 0.155-0.175j 0.155+0.175j -0.221+0.j ] r 28 centered False [-0.319+0.j 0.257-0.133j 0.257+0.133j -0.281+0.j 0.28 +0.j -0.167-0.197j -0.167+0.197j 0.241+0.j ] r 28 centered True [ 0.385+0.j -0.317+0.j 0.242-0.085j 0.242+0.085j -0.25 +0.j -0.161+0.189j -0.161-0.189j 0.147+0.173j] ```