OUTPUT of run 3 of at most 100: ``` L0 per-head V and O singular values (first 35) 0 V [2.214 2.066 1.611 1.282 1.259 1.034 0.964 0.915 0.843 0.762 0.716 0.68 0.652 0.649 0.609 0.606 0.593 0.551 0.54 0.516 0.51 0.498 0.489 0.473 0.455 0.451 0.447 0.434 0.427 0.413 0.408 0.404 0.398 0.396 0.386] 0 Vscaled [1.331 1.075 0.446 0.374 0.304 0.294 0.256 0.247 0.237 0.224 0.218 0.205 0.203 0.199 0.188 0.181 0.181 0.174 0.169 0.163 0.162 0.155 0.151 0.147 0.146 0.138 0.136 0.132 0.13 0.126 0.125 0.122 0.121 0.118 0.117] 0 O [3.214 2.015 1.441 1.305 1.153 0.96 0.905 0.839 0.717 0.68 0.572 0.57 0.52 0.512 0.486 0.445 0.427 0.394 0.377 0.366 0.358 0.355 0.345 0.338 0.313 0.299 0.288 0.281 0.263 0.26 0.248 0.243 0.236 0.228 0.225] 1 V [2.25 2.127 2.089 2.055 1.991 1.973 1.958 1.933 1.869 1.832 1.808 1.785 1.717 1.69 1.639 1.611 1.584 1.554 1.496 1.48 1.419 1.37 1.277 1.193 1.14 1.094 1.028 0.906 0.744 0.693 0.682 0.659 0.638 0.587 0.538] 1 Vscaled [0.481 0.455 0.448 0.429 0.422 0.41 0.403 0.387 0.38 0.374 0.37 0.36 0.353 0.341 0.331 0.327 0.322 0.313 0.31 0.303 0.295 0.288 0.281 0.278 0.27 0.255 0.253 0.241 0.227 0.216 0.209 0.201 0.198 0.171 0.165] 1 O [2.342 2.336 2.292 2.187 2.168 2.157 2.117 2.073 2.052 2.021 1.998 1.979 1.944 1.931 1.898 1.827 1.781 1.751 1.727 1.654 1.595 1.541 1.442 1.324 1.247 1.21 1.125 1.111 1.047 0.963 0.908 0.827 0.755 0.72 0.677] 2 V [2.164 2.047 2.012 1.723 1.514 1.361 1.273 1.115 1.036 0.993 0.908 0.857 0.765 0.74 0.681 0.628 0.618 0.592 0.58 0.57 0.535 0.526 0.512 0.477 0.471 0.46 0.458 0.445 0.444 0.433 0.423 0.409 0.408 0.401 0.397] 2 Vscaled [0.5 0.482 0.463 0.4 0.363 0.338 0.296 0.284 0.268 0.262 0.246 0.223 0.217 0.21 0.2 0.191 0.183 0.18 0.177 0.17 0.167 0.159 0.156 0.153 0.15 0.147 0.145 0.138 0.135 0.13 0.127 0.124 0.12 0.118 0.115] 2 O [2.256 2.044 1.778 1.657 1.481 1.38 1.302 1.216 1.101 0.919 0.88 0.785 0.738 0.671 0.644 0.59 0.539 0.478 0.446 0.419 0.415 0.387 0.372 0.363 0.35 0.335 0.331 0.309 0.276 0.271 0.265 0.25 0.243 0.233 0.224] 3 V [1.443 1.28 1.237 1.154 1.129 1.076 1.038 1.013 0.958 0.917 0.897 0.841 0.816 0.734 0.696 0.67 0.64 0.604 0.581 0.568 0.529 0.519 0.513 0.475 0.471 0.464 0.457 0.446 0.432 0.417 0.414 0.404 0.395 0.391 0.386] 3 Vscaled [0.589 0.386 0.343 0.309 0.294 0.277 0.273 0.247 0.243 0.24 0.23 0.216 0.209 0.207 0.196 0.192 0.187 0.178 0.175 0.173 0.168 0.159 0.154 0.151 0.149 0.143 0.143 0.141 0.138 0.13 0.128 0.125 0.124 0.121 0.118] 3 O [1.802 1.7 1.582 1.509 1.378 1.286 1.251 1.225 1.135 1.115 1.044 0.978 0.928 0.849 0.837 0.772 0.73 0.677 0.629 0.575 0.543 0.504 0.466 0.463 0.406 0.388 0.382 0.358 0.336 0.321 0.313 0.3 0.288 0.28 0.265] 4 V [4.102 3.344 2.539 2.115 1.785 1.548 1.482 1.422 1.387 1.323 1.196 1.164 1.145 1.022 0.967 0.848 0.832 0.801 0.7 0.665 0.644 0.637 0.616 0.566 0.56 0.535 0.529 0.494 0.474 0.464 0.447 0.445 0.433 0.427 0.413] 4 Vscaled [2.899 2.03 1.104 0.577 0.48 0.428 0.379 0.362 0.315 0.311 0.301 0.274 0.264 0.262 0.245 0.235 0.218 0.207 0.202 0.194 0.188 0.183 0.177 0.171 0.167 0.164 0.161 0.158 0.15 0.144 0.144 0.142 0.136 0.133 0.128] 4 O [5.589 4.578 2.488 1.978 1.578 1.534 1.499 1.299 1.164 1.007 0.898 0.848 0.788 0.704 0.669 0.644 0.558 0.465 0.453 0.418 0.41 0.383 0.362 0.356 0.339 0.317 0.31 0.302 0.284 0.277 0.255 0.252 0.248 0.233 0.221] 5 V [2.397 2.282 2.191 2.148 2.082 2.049 2.033 1.995 1.901 1.88 1.86 1.788 1.772 1.744 1.703 1.646 1.591 1.538 1.448 1.384 1.317 1.2 1.042 0.923 0.892 0.853 0.817 0.784 0.767 0.744 0.737 0.722 0.709 0.681 0.663] 5 Vscaled [0.493 0.474 0.447 0.437 0.426 0.408 0.404 0.393 0.386 0.377 0.363 0.361 0.357 0.345 0.333 0.329 0.323 0.317 0.31 0.304 0.299 0.289 0.285 0.277 0.274 0.268 0.252 0.245 0.242 0.235 0.233 0.221 0.216 0.212 0.208] 5 O [2.529 2.416 2.372 2.359 2.342 2.325 2.263 2.246 2.235 2.178 2.123 2.056 2.049 1.968 1.957 1.84 1.779 1.64 1.567 1.465 1.378 1.28 1.227 1.213 1.162 1.142 1.095 1.056 1.036 1.018 1.012 0.949 0.929 0.919 0.897] Signed key-composition trace: sqrt(d)*tr(A_B B_A)/(||A_B||F ||B_A||F) Columns are L0H0..L0H5. Large positive values are a matching-circuit clue, not an attention measurement. L1H0 [ 0.127 -0.002 -0.154 0.098 0.406 -0.067] L1H1 [-0.091 0.137 -0.316 0.108 0.374 0.302] L1H2 [ 0.077 0.013 -0.33 0.179 0.64 -0.053] L1H3 [ 0.15 -0.05 -0.258 0.124 0.825 0.243] L1H4 [ 0.183 0.043 -0.354 0.223 0.86 0.184] L1H5 [-0.382 0.088 -0.263 -0.008 -0.071 0.251] L2H0 [ 0.097 -0.012 -0.059 0.153 0.194 -0.09 ] L2H1 [0.128 0.028 0.211 0.049 0.074 1.52 ] L2H2 [ 0.222 -0.324 -0.106 0.029 0.391 -0.758] L2H3 [ 0.348 -0.016 -0.128 0.129 0.775 -0.05 ] L2H4 [ 0.156 -0.108 -0.079 0.198 0.481 0.036] L2H5 [0.177 0.011 0.002 0.091 0.369 0.134] L3H0 [ 0.192 -0.071 0.133 0.151 0.191 0.373] L3H1 [0.153 0.139 0.117 0.027 0.081 1.195] L3H2 [ 0.121 -0.062 0.03 -0.015 0.254 -0.024] L3H3 [0.087 0.062 0.093 0.052 0.143 0.019] L3H4 [ 0.167 -0.123 0.11 0.057 0.234 0.198] L3H5 [ 0.117 -0.174 0.009 0.052 0.049 1.397] Strongest signed composition traces over all earlier heads L1H0 [('L0H4', np.float64(0.406)), ('L0H0', np.float64(0.127)), ('L0H3', np.float64(0.098)), ('L0H1', np.float64(-0.002))] L1H1 [('L0H4', np.float64(0.374)), ('L0H5', np.float64(0.302)), ('L0H1', np.float64(0.137)), ('L0H3', np.float64(0.108))] L1H2 [('L0H4', np.float64(0.64)), ('L0H3', np.float64(0.179)), ('L0H0', np.float64(0.077)), ('L0H1', np.float64(0.013))] L1H3 [('L0H4', np.float64(0.825)), ('L0H5', np.float64(0.243)), ('L0H0', np.float64(0.15)), ('L0H3', np.float64(0.124))] L1H4 [('L0H4', np.float64(0.86)), ('L0H3', np.float64(0.223)), ('L0H5', np.float64(0.184)), ('L0H0', np.float64(0.183))] L1H5 [('L0H5', np.float64(0.251)), ('L0H1', np.float64(0.088)), ('L0H3', np.float64(-0.008)), ('L0H4', np.float64(-0.071))] L2H0 [('L0H4', np.float64(0.194)), ('L0H3', np.float64(0.153)), ('L1H5', np.float64(0.112)), ('L0H0', np.float64(0.097))] L2H1 [('L0H5', np.float64(1.52)), ('L1H3', np.float64(1.127)), ('L1H2', np.float64(0.944)), ('L1H1', np.float64(0.938))] L2H2 [('L0H4', np.float64(0.391)), ('L0H0', np.float64(0.222)), ('L0H3', np.float64(0.029)), ('L0H2', np.float64(-0.106))] L2H3 [('L0H4', np.float64(0.775)), ('L0H0', np.float64(0.348)), ('L0H3', np.float64(0.129)), ('L1H1', np.float64(0.095))] L2H4 [('L0H4', np.float64(0.481)), ('L0H3', np.float64(0.198)), ('L0H0', np.float64(0.156)), ('L0H5', np.float64(0.036))] L2H5 [('L0H4', np.float64(0.369)), ('L1H2', np.float64(0.185)), ('L0H0', np.float64(0.177)), ('L1H1', np.float64(0.162))] L3H0 [('L2H2', np.float64(0.466)), ('L0H5', np.float64(0.373)), ('L1H2', np.float64(0.198)), ('L0H0', np.float64(0.192))] L3H1 [('L2H2', np.float64(2.164)), ('L0H5', np.float64(1.195)), ('L1H4', np.float64(0.894)), ('L1H3', np.float64(0.887))] L3H2 [('L0H4', np.float64(0.254)), ('L0H0', np.float64(0.121)), ('L0H2', np.float64(0.03)), ('L1H1', np.float64(0.026))] L3H3 [('L2H4', np.float64(0.29)), ('L2H3', np.float64(0.202)), ('L2H2', np.float64(0.177)), ('L1H2', np.float64(0.155))] L3H4 [('L0H4', np.float64(0.234)), ('L2H4', np.float64(0.201)), ('L0H5', np.float64(0.198)), ('L0H0', np.float64(0.167))] L3H5 [('L2H2', np.float64(3.597)), ('L0H5', np.float64(1.397)), ('L1H3', np.float64(1.338)), ('L1H4', np.float64(1.12))] L0 QK/V subspace overlap L0H0 [(1, np.float64(0.328), np.float64(0.052)), (4, np.float64(0.615), np.float64(0.833)), (5, np.float64(0.408), np.float64(0.07))] L0H1 [(1, np.float64(0.126), np.float64(0.13)), (4, np.float64(0.709), np.float64(0.679)), (5, np.float64(0.183), np.float64(0.144))] L0H2 [(1, np.float64(0.228), np.float64(0.113)), (4, np.float64(0.693), np.float64(0.778)), (5, np.float64(0.309), np.float64(0.145))] L0H3 [(1, np.float64(0.268), np.float64(0.087)), (4, np.float64(0.735), np.float64(0.741)), (5, np.float64(0.269), np.float64(0.136))] L0H4 [(1, np.float64(0.121), np.float64(0.101)), (4, np.float64(0.26), np.float64(0.213)), (5, np.float64(0.119), np.float64(0.101))] L0H5 [(1, np.float64(0.089), np.float64(0.092)), (4, np.float64(0.432), np.float64(0.414)), (5, np.float64(0.201), np.float64(0.174))] Affine key-prior coefficient norm from ln1_b, relative to bilinear Frobenius norm L0H0 0.148 0.163 L0H1 0.589 0.084 L0H2 0.187 0.134 L0H3 0.456 0.238 L0H4 0.007 0.1 L0H5 0.407 0.052 L1H0 6.713 1.303 L1H1 25.199 1.988 L1H2 17.599 1.78 L1H3 21.755 2.077 L1H4 20.144 2.016 L1H5 19.681 1.683 L2H0 7.156 0.437 L2H1 5.101 0.328 L2H2 12.669 0.565 L2H3 8.295 0.525 L2H4 9.595 0.545 L2H5 5.089 0.385 L3H0 2.389 0.144 L3H1 2.107 0.155 L3H2 2.738 0.166 L3H3 2.067 0.245 L3H4 2.453 0.177 L3H5 2.48 0.146 ```