OUTPUT of run 3 of at most 100:
```
INDUCTION ALGEBRA: trace(A_l C_earlier)/(||A_l||F ||C_earlier||F)
Positive values indicate a possible query-to-preceding-token key match, not measured attention.
columns ['L0H0', 'L0H1', 'L0H2', 'L0H3', 'L0H4', 'L0H5']
L1H0 [ 0.0065 -0.0001 -0.0079  0.005   0.0207 -0.0034]
L1H1 [-0.0046  0.007  -0.0161  0.0055  0.0191  0.0154]
L1H2 [ 0.0039  0.0007 -0.0168  0.0092  0.0326 -0.0027]
L1H3 [ 0.0077 -0.0026 -0.0132  0.0063  0.0421  0.0124]
L1H4 [ 0.0093  0.0022 -0.0181  0.0114  0.0439  0.0094]
L1H5 [-0.0195  0.0045 -0.0134 -0.0004 -0.0036  0.0128]
columns ['L0H0', 'L0H1', 'L0H2', 'L0H3', 'L0H4', 'L0H5', 'L1H0', 'L1H1', 'L1H2', 'L1H3', 'L1H4', 'L1H5']
L2H0 [ 0.0049 -0.0006 -0.003   0.0078  0.0099 -0.0046 -0.0126  0.0039 -0.0003  0.0016 -0.0004  0.0057]
L2H1 [0.0065 0.0014 0.0108 0.0025 0.0038 0.0776 0.0025 0.0479 0.0482 0.0575 0.0476 0.0351]
L2H2 [ 0.0114 -0.0165 -0.0054  0.0015  0.02   -0.0387 -0.0418 -0.0162 -0.0434 -0.007  -0.023  -0.0069]
L2H3 [ 0.0177 -0.0008 -0.0065  0.0066  0.0396 -0.0026 -0.0197  0.0049 -0.0162  0.0042 -0.0048  0.0044]
L2H4 [ 0.0079 -0.0055 -0.004   0.0101  0.0246  0.0018 -0.0139 -0.0082 -0.0115 -0.0019 -0.0042 -0.006 ]
L2H5 [0.009  0.0006 0.0001 0.0046 0.0188 0.0068 0.0001 0.0083 0.0095 0.0064 0.0068 0.0068]
columns ['L0H0', 'L0H1', 'L0H2', 'L0H3', 'L0H4', 'L0H5', 'L1H0', 'L1H1', 'L1H2', 'L1H3', 'L1H4', 'L1H5', 'L2H0', 'L2H1', 'L2H2', 'L2H3', 'L2H4', 'L2H5']
L3H0 [ 0.0098 -0.0036  0.0068  0.0077  0.0098  0.019   0.0079  0.0083  0.0101  0.006   0.006   0.0009 -0.0033 -0.0135  0.0238 -0.0008  0.0044 -0.0088]
L3H1 [ 0.0078  0.0071  0.006   0.0014  0.0041  0.061   0.0127  0.0333  0.038   0.0453  0.0456  0.0271  0.0153 -0.0181  0.1104  0.0209  0.0053 -0.0002]
L3H2 [ 0.0062 -0.0032  0.0015 -0.0008  0.013  -0.0012 -0.0116  0.0013 -0.0092  0.     -0.0089 -0.0036 -0.0046 -0.007  -0.0004 -0.0033 -0.0013 -0.0031]
L3H3 [ 0.0045  0.0032  0.0048  0.0026  0.0073  0.001   0.005   0.0017  0.0079 -0.0004  0.0022  0.0042  0.0063 -0.0054  0.009   0.0103  0.0148  0.0045]
L3H4 [ 0.0085 -0.0063  0.0056  0.0029  0.0119  0.0101  0.0005  0.0069  0.0032  0.0047  0.0048  0.0083  0.0011 -0.0083  0.0081  0.0041  0.0103 -0.0038]
L3H5 [ 0.006  -0.0089  0.0004  0.0027  0.0025  0.0713  0.0041  0.0423  0.0482  0.0683  0.0572  0.0198 -0.0084 -0.0599  0.1836  0.055  -0.0216 -0.022 ]

L0 effective Q,K,V singular values (first 32):
L0H0 Q [1.242 1.067 0.419 0.304 0.261 0.221 0.213 0.198 0.187 0.182 0.177 0.175 0.163 0.157 0.152 0.144 0.14  0.138 0.134 0.13  0.126 0.122 0.117 0.116
 0.113 0.11  0.107 0.103 0.101 0.1   0.095 0.094]
L0H0 K [0.661 0.456 0.27  0.246 0.23  0.223 0.212 0.21  0.198 0.196 0.19  0.188 0.182 0.179 0.173 0.171 0.164 0.158 0.153 0.153 0.147 0.144 0.142 0.141
 0.137 0.133 0.128 0.127 0.125 0.124 0.122 0.12 ]
L0H0 V [1.331 1.075 0.446 0.374 0.304 0.294 0.256 0.247 0.237 0.224 0.218 0.205 0.203 0.199 0.188 0.181 0.18  0.174 0.169 0.162 0.162 0.155 0.151 0.147
 0.146 0.138 0.136 0.132 0.13  0.126 0.125 0.122]
L0H1 Q [1.513 1.324 1.288 1.273 1.188 1.055 1.027 0.789 0.773 0.715 0.697 0.667 0.654 0.644 0.629 0.593 0.569 0.555 0.538 0.522 0.52  0.494 0.462 0.44
 0.41  0.394 0.386 0.367 0.35  0.341 0.32  0.309]
L0H1 K [3.78  2.409 1.606 1.4   1.309 1.203 1.148 1.007 0.879 0.855 0.831 0.809 0.747 0.722 0.71  0.655 0.639 0.614 0.58  0.569 0.534 0.504 0.484 0.474
 0.447 0.427 0.381 0.353 0.344 0.324 0.32  0.31 ]
L0H1 V [0.48  0.455 0.448 0.429 0.421 0.41  0.403 0.385 0.379 0.373 0.37  0.36  0.353 0.341 0.33  0.327 0.322 0.313 0.309 0.303 0.295 0.287 0.281 0.278
 0.27  0.255 0.253 0.241 0.226 0.216 0.209 0.201]
L0H2 Q [1.143 0.768 0.644 0.498 0.488 0.438 0.434 0.399 0.358 0.33  0.31  0.298 0.279 0.261 0.252 0.249 0.243 0.231 0.217 0.21  0.208 0.202 0.198 0.195
 0.185 0.179 0.174 0.17  0.166 0.161 0.159 0.147]
L0H2 K [1.187 0.914 0.55  0.529 0.506 0.489 0.424 0.397 0.389 0.37  0.35  0.333 0.321 0.307 0.298 0.292 0.28  0.268 0.258 0.249 0.235 0.228 0.22  0.214
 0.212 0.2   0.192 0.19  0.187 0.177 0.173 0.166]
L0H2 V [0.5   0.482 0.462 0.399 0.361 0.337 0.296 0.283 0.268 0.261 0.245 0.223 0.217 0.21  0.199 0.19  0.182 0.179 0.177 0.17  0.167 0.158 0.156 0.153
 0.15  0.147 0.145 0.137 0.135 0.129 0.127 0.124]
L0H3 Q [1.547 1.125 0.98  0.628 0.571 0.521 0.409 0.372 0.364 0.343 0.299 0.29  0.282 0.279 0.275 0.255 0.243 0.226 0.212 0.198 0.198 0.194 0.184 0.171
 0.162 0.154 0.15  0.148 0.144 0.143 0.141 0.139]
L0H3 K [1.093 0.949 0.68  0.542 0.435 0.406 0.399 0.365 0.34  0.331 0.319 0.299 0.278 0.26  0.249 0.239 0.227 0.223 0.22  0.203 0.191 0.184 0.18  0.175
 0.173 0.166 0.161 0.156 0.156 0.154 0.147 0.144]
L0H3 V [0.588 0.386 0.342 0.308 0.294 0.277 0.273 0.246 0.242 0.24  0.23  0.216 0.209 0.206 0.195 0.192 0.187 0.178 0.175 0.172 0.168 0.158 0.153 0.151
 0.149 0.143 0.143 0.141 0.138 0.13  0.128 0.125]
L0H4 Q [0.205 0.143 0.139 0.129 0.126 0.121 0.119 0.114 0.112 0.111 0.103 0.101 0.1   0.098 0.097 0.094 0.091 0.088 0.088 0.085 0.084 0.082 0.079 0.077
 0.077 0.075 0.073 0.071 0.068 0.067 0.067 0.066]
L0H4 K [0.214 0.193 0.186 0.178 0.173 0.167 0.156 0.154 0.148 0.146 0.142 0.137 0.135 0.129 0.128 0.123 0.12  0.118 0.116 0.113 0.111 0.107 0.106 0.103
 0.099 0.099 0.097 0.093 0.092 0.09  0.088 0.086]
L0H4 V [2.898 2.029 1.104 0.576 0.48  0.427 0.379 0.362 0.315 0.31  0.3   0.274 0.264 0.262 0.245 0.235 0.218 0.207 0.202 0.194 0.187 0.183 0.177 0.171
 0.167 0.164 0.161 0.158 0.15  0.144 0.143 0.142]
L0H5 Q [1.396 1.368 1.336 1.315 1.25  1.232 1.208 1.19  1.145 1.132 1.095 1.079 1.066 1.063 1.015 1.011 0.984 0.951 0.912 0.855 0.797 0.777 0.704 0.622
 0.608 0.588 0.525 0.46  0.445 0.429 0.406 0.397]
L0H5 K [3.314 2.634 1.802 1.724 1.638 1.528 1.499 1.455 1.396 1.35  1.273 1.21  1.2   1.167 1.063 1.045 1.005 0.935 0.905 0.887 0.827 0.736 0.643 0.62
 0.596 0.561 0.501 0.439 0.431 0.4   0.375 0.358]
L0H5 V [0.493 0.473 0.446 0.437 0.426 0.408 0.404 0.393 0.386 0.376 0.363 0.361 0.356 0.345 0.333 0.329 0.322 0.316 0.31  0.304 0.299 0.289 0.285 0.277
 0.274 0.267 0.252 0.245 0.241 0.235 0.233 0.221]

Average squared overlap of leading 24-dimensional L0 subspaces:
['Q1', 'K1', 'Q5', 'K5', 'V1', 'V5']
[[1.    0.607 0.77  0.655 0.103 0.146]
 [0.607 1.    0.662 0.776 0.122 0.178]
 [0.77  0.662 1.    0.814 0.081 0.137]
 [0.655 0.776 0.814 1.    0.08  0.153]
 [0.103 0.122 0.081 0.08  1.    0.619]
 [0.146 0.178 0.137 0.153 0.619 1.   ]]

Layer 0 QK product correlations, centered effective matrices:
power 1 cosine with H1 0.4137 cosine with H1 transpose -0.0104
power 2 cosine with H1 0.5461 cosine with H1 transpose -0.4342
power 3 cosine with H1 -0.0482 cosine with H1 transpose -0.4405
power 4 cosine with H1 -0.6081 cosine with H1 transpose 0.0147
power 5 cosine with H1 -0.6339 cosine with H1 transpose 0.4624
power 6 cosine with H1 -0.1532 cosine with H1 transpose 0.5474

Top-key and top-query directions of each head: overlap with L0H4 output subspace (top 4).
L0H0 Q [0.343 0.245 0.035 0.212] K [0.162 0.062 0.076 0.03 ]
L0H1 Q [0.008 0.139 0.036 0.011] K [0.101 0.029 0.016 0.018]
L0H2 Q [0.183 0.341 0.034 0.185] K [0.023 0.014 0.007 0.08 ]
L0H3 Q [0.311 0.485 0.183 0.19 ] K [0.129 0.02  0.023 0.069]
L0H4 Q [0.462 0.17  0.121 0.039] K [0.112 0.081 0.038 0.008]
L0H5 Q [0.033 0.004 0.011 0.017] K [0.082 0.067 0.031 0.025]
L1H0 Q [0.049 0.073 0.061 0.071] K [0.809 0.62  0.119 0.016]
L1H1 Q [0.162 0.056 0.016 0.024] K [0.913 0.78  0.048 0.041]
L1H2 Q [0.206 0.06  0.068 0.006] K [0.885 0.802 0.074 0.016]
L1H3 Q [0.236 0.1   0.067 0.028] K [0.9   0.797 0.058 0.016]
L1H4 Q [0.154 0.08  0.049 0.103] K [0.887 0.818 0.065 0.01 ]
L1H5 Q [0.069 0.076 0.122 0.017] K [0.852 0.764 0.077 0.054]
L2H0 Q [0.01  0.007 0.002 0.033] K [0.563 0.12  0.01  0.011]
L2H1 Q [0.042 0.006 0.007 0.004] K [0.032 0.012 0.005 0.013]
L2H2 Q [0.002 0.003 0.006 0.01 ] K [0.88  0.361 0.009 0.243]
L2H3 Q [0.001 0.005 0.029 0.082] K [0.823 0.649 0.032 0.006]
L2H4 Q [0.003 0.042 0.015 0.009] K [0.762 0.041 0.532 0.099]
L2H5 Q [0.017 0.002 0.016 0.021] K [0.413 0.009 0.179 0.018]
L3H0 Q [0.014 0.003 0.002 0.001] K [0.192 0.023 0.019 0.027]
L3H1 Q [0.017 0.015 0.002 0.021] K [0.02  0.002 0.008 0.003]
L3H2 Q [0.013 0.003 0.003 0.019] K [0.488 0.095 0.004 0.001]
L3H3 Q [0.016 0.002 0.003 0.   ] K [0.065 0.01  0.032 0.029]
L3H4 Q [0.014 0.009 0.004 0.002] K [0.336 0.04  0.002 0.003]
L3H5 Q [0.003 0.002 0.019 0.002] K [0.014 0.014 0.015 0.002]

```