OUTPUT of run 2 of at most 100:
```

LAYER 0
W_q sv12 [4.368 3.131 3.099 3.042 2.994 2.951 2.926 2.881 2.859 2.857 2.804 2.766] energy ranks 1,5,10,28,64 [0.061 0.182 0.316 0.676 0.875]
W_k sv12 [4.163 3.952 3.467 3.427 3.364 3.335 3.279 3.256 3.214 3.163 3.115 3.029] energy ranks 1,5,10,28,64 [0.051 0.198 0.352 0.713 0.88 ]
W_v sv12 [5.208 3.364 3.006 2.962 2.815 2.639 2.527 2.46  2.37  2.301 2.25  2.23 ] energy ranks 1,5,10,28,64 [0.122 0.288 0.424 0.715 0.856]
W_o sv12 [7.361 6.099 3.167 2.452 2.321 2.202 2.171 2.16  2.105 2.072 2.021 1.995] energy ranks 1,5,10,28,64 [0.198 0.412 0.496 0.718 0.91 ]
W_fc sv12 [18.118 12.76  11.337  9.323  9.267  9.021  8.898  8.817  8.609  8.4    8.189  8.06 ] energy ranks 1,5,10,28,64 [0.107 0.258 0.383 0.676 0.892]
W_proj sv12 [10.899  7.702  7.173  6.638  6.541  6.288  6.201  6.133  5.941  5.917  5.758  5.641] energy ranks 1,5,10,28,64 [0.048 0.127 0.201 0.394 0.633]
L0H0 QK eig12 [ 0.079+0.j     0.049+0.j    -0.024-0.006j -0.024+0.006j -0.023+0.j     0.018+0.013j  0.018-0.013j -0.02 +0.j    -0.014+0.012j -0.014-0.012j
 -0.014+0.003j -0.014-0.003j] keybias_norm=0.043 keybias_topright=-0.694
L0H1 QK eig12 [-0.062+0.j    -0.01 +0.j    -0.009+0.j     0.004+0.j    -0.004+0.j    -0.003+0.002j -0.003-0.002j -0.002+0.j    -0.002-0.j    -0.002+0.001j
 -0.002-0.001j -0.001+0.j   ] keybias_norm=0.013 keybias_topright=-0.607
L0H2 QK eig12 [-0.057+0.j    -0.007+0.003j -0.007-0.003j -0.003+0.001j -0.003-0.001j -0.003+0.j    -0.001-0.j    -0.001+0.j    -0.001-0.001j -0.001+0.001j
  0.001+0.001j  0.001-0.001j] keybias_norm=0.007 keybias_topright=-0.876
L0H3 QK eig12 [0.137+0.133j 0.137-0.133j 0.11 -0.089j 0.11 +0.089j 0.095-0.098j 0.095+0.098j 0.084-0.104j 0.084+0.104j 0.064-0.114j 0.064+0.114j 0.096+0.068j
 0.096-0.068j] keybias_norm=0.043 keybias_topright=0.152
L0H4 QK eig12 [-0.015+0.j    -0.01 +0.j    -0.008+0.j    -0.003+0.j    -0.003-0.j    -0.002+0.001j -0.002-0.001j -0.   +0.002j -0.   -0.002j -0.001+0.001j
 -0.001-0.001j -0.002+0.j   ] keybias_norm=0.017 keybias_topright=-0.942
L0H5 QK eig12 [-0.005+0.j     0.001+0.j    -0.001+0.j    -0.   +0.001j -0.   -0.001j -0.001+0.j    -0.001+0.j    -0.001-0.j     0.   +0.001j  0.   -0.001j
 -0.   +0.001j -0.   -0.001j] keybias_norm=0.005 keybias_topright=0.971
LN1 largest bias coords [(312, np.float64(-0.09542398899793625)), (146, np.float64(0.09060516208410263)), (12, np.float64(0.08561607450246811)), (104, np.float64(-0.08411034941673279)), (289, np.float64(0.08255884051322937)), (156, np.float64(-0.0779787078499794)), (303, np.float64(-0.07763753086328506)), (328, np.float64(0.07621065527200699))]

LAYER 1
W_q sv12 [10.406  8.239  7.475  7.033  6.622  6.395  6.275  6.13   6.065  5.901  5.819  5.716] energy ranks 1,5,10,28,64 [0.071 0.215 0.34  0.646 0.893]
W_k sv12 [14.477 11.787  6.37   6.054  5.523  5.407  5.268  5.167  5.091  4.924  4.776  4.529] energy ranks 1,5,10,28,64 [0.172 0.375 0.485 0.716 0.905]
W_v sv12 [4.728 4.547 4.459 4.395 4.361 4.332 4.291 4.276 4.244 4.133 4.079 3.991] energy ranks 1,5,10,28,64 [0.027 0.123 0.232 0.514 0.801]
W_o sv12 [4.045 3.267 3.188 3.179 3.163 3.101 3.071 3.046 3.046 3.036 3.009 2.985] energy ranks 1,5,10,28,64 [0.02  0.071 0.128 0.312 0.596]
W_fc sv12 [19.829 11.575 11.522 11.262 11.059 10.875 10.632 10.497 10.363 10.3   10.174 10.05 ] energy ranks 1,5,10,28,64 [0.061 0.141 0.227 0.457 0.706]
W_proj sv12 [9.038 8.457 8.103 7.333 7.318 7.088 6.772 6.666 6.566 6.429 6.372 6.278] energy ranks 1,5,10,28,64 [0.014 0.054 0.092 0.199 0.367]
L1H0 QK eig12 [-0.264+0.j     0.115+0.23j   0.115-0.23j  -0.221+0.j    -0.199+0.05j  -0.199-0.05j  -0.1  +0.121j -0.1  -0.121j -0.143+0.j    -0.121+0.04j
 -0.121-0.04j  -0.039+0.1j  ] keybias_norm=3.975 keybias_topright=0.986
L1H1 QK eig12 [ 0.093+0.245j  0.093-0.245j -0.202+0.j    -0.164+0.018j -0.164-0.018j -0.145+0.j    -0.119-0.07j  -0.119+0.07j  -0.056-0.108j -0.056+0.108j
 -0.1  +0.033j -0.1  -0.033j] keybias_norm=3.689 keybias_topright=-0.975
L1H2 QK eig12 [-0.249+0.j    -0.201+0.j     0.069-0.182j  0.069+0.182j -0.172+0.j    -0.109+0.1j   -0.109-0.1j   -0.142+0.j     0.007+0.132j  0.007-0.132j
 -0.122+0.034j -0.122-0.034j] keybias_norm=3.052 keybias_topright=0.974
L1H3 QK eig12 [ 0.083+0.224j  0.083-0.224j -0.198+0.j    -0.151+0.j    -0.127-0.062j -0.127+0.062j -0.045+0.12j  -0.045-0.12j  -0.128+0.j    -0.099+0.046j
 -0.099-0.046j -0.001+0.092j] keybias_norm=3.391 keybias_topright=0.977
L1H4 QK eig12 [-0.172+0.j    -0.045+0.145j -0.045-0.145j -0.146+0.j    -0.132+0.044j -0.132-0.044j -0.135+0.j    -0.107+0.057j -0.107-0.057j  0.079-0.091j
  0.079+0.091j -0.105+0.022j] keybias_norm=2.329 keybias_topright=-0.915
L1H5 QK eig12 [ 0.101+0.253j  0.101-0.253j -0.26 +0.j    -0.145+0.099j -0.145-0.099j -0.156+0.027j -0.156-0.027j -0.082+0.114j -0.082-0.114j -0.124+0.j
 -0.114+0.j    -0.089+0.043j] keybias_norm=3.896 keybias_topright=-0.978
LN1 largest bias coords [(39, np.float64(-1.5675644874572754)), (58, np.float64(1.36513352394104)), (99, np.float64(-1.3355246782302856)), (184, np.float64(-0.5617263317108154)), (383, np.float64(-0.462228000164032)), (21, np.float64(-0.4298906922340393)), (153, np.float64(0.29687735438346863)), (266, np.float64(0.2773352265357971))]

LAYER 2
W_q sv12 [5.966 5.019 4.865 4.773 4.695 4.568 4.496 4.433 4.38  4.28  4.224 4.208] energy ranks 1,5,10,28,64 [0.024 0.085 0.15  0.331 0.571]
W_k sv12 [9.863 9.533 4.386 4.171 4.117 3.956 3.879 3.777 3.759 3.708 3.607 3.592] energy ranks 1,5,10,28,64 [0.072 0.179 0.233 0.383 0.599]
W_v sv12 [3.914 3.78  3.727 3.662 3.582 3.553 3.51  3.464 3.449 3.432 3.396 3.385] energy ranks 1,5,10,28,64 [0.014 0.063 0.118 0.286 0.534]
W_o sv12 [4.398 4.037 3.875 3.739 3.621 3.589 3.488 3.467 3.387 3.353 3.322 3.281] energy ranks 1,5,10,28,64 [0.018 0.074 0.131 0.291 0.527]
W_fc sv12 [15.734  9.249  8.718  8.694  8.47   7.865  7.739  7.482  7.456  7.36   7.336  7.236] energy ranks 1,5,10,28,64 [0.039 0.087 0.132 0.265 0.455]
W_proj sv12 [10.197  9.035  8.882  8.411  8.224  8.088  7.988  7.827  7.679  7.491  7.372  7.26 ] energy ranks 1,5,10,28,64 [0.017 0.066 0.116 0.245 0.419]
L2H0 QK eig12 [ 0.081+0.299j  0.081-0.299j -0.104+0.j    -0.097+0.024j -0.097-0.024j -0.011-0.089j -0.011+0.089j -0.086+0.013j -0.086-0.013j -0.053+0.066j
 -0.053-0.066j -0.073+0.036j] keybias_norm=2.021 keybias_topright=-0.974
L2H1 QK eig12 [ 0.079+0.268j  0.079-0.268j -0.13 +0.j    -0.067+0.105j -0.067-0.105j -0.115+0.j     0.114+0.j     0.024+0.099j  0.024-0.099j -0.087-0.046j
 -0.087+0.046j  0.06 +0.077j] keybias_norm=1.881 keybias_topright=0.880
L2H2 QK eig12 [ 0.075+0.208j  0.075-0.208j -0.112+0.007j -0.112-0.007j -0.075+0.068j -0.075-0.068j -0.079-0.048j -0.079+0.048j -0.085+0.031j -0.085-0.031j
 -0.081+0.015j -0.081-0.015j] keybias_norm=1.609 keybias_topright=-0.974
L2H3 QK eig12 [ 0.036+0.116j  0.036-0.116j -0.055+0.076j -0.055-0.076j -0.025+0.082j -0.025-0.082j -0.073-0.031j -0.073+0.031j -0.026-0.067j -0.026+0.067j
  0.037-0.061j  0.037+0.061j] keybias_norm=1.060 keybias_topright=-0.742
L2H4 QK eig12 [ 0.077+0.324j  0.077-0.324j -0.128-0.023j -0.128+0.023j  0.029+0.125j  0.029-0.125j -0.077+0.097j -0.077-0.097j  0.122+0.j     0.067+0.083j
  0.067-0.083j -0.093+0.044j] keybias_norm=2.307 keybias_topright=0.946
L2H5 QK eig12 [ 0.076+0.275j  0.076-0.275j -0.028+0.139j -0.028-0.139j  0.125+0.022j  0.125-0.022j -0.101+0.049j -0.101-0.049j -0.109-0.004j -0.109+0.004j
 -0.08 -0.073j -0.08 +0.073j] keybias_norm=2.024 keybias_topright=-0.926
LN1 largest bias coords [(99, np.float64(-1.6588826179504395)), (58, np.float64(1.331817865371704)), (39, np.float64(-1.1991313695907593)), (383, np.float64(-0.5070002675056458)), (184, np.float64(-0.5046411752700806)), (21, np.float64(-0.4534468650817871)), (63, np.float64(-0.32454749941825867)), (266, np.float64(0.2990613281726837))]

LAYER 3
W_q sv12 [4.086 3.933 3.809 3.715 3.68  3.633 3.571 3.556 3.531 3.493 3.464 3.449] energy ranks 1,5,10,28,64 [0.015 0.067 0.124 0.295 0.537]
W_k sv12 [6.362 5.842 3.696 3.459 3.363 3.304 3.256 3.244 3.214 3.193 3.177 3.163] energy ranks 1,5,10,28,64 [0.038 0.106 0.156 0.308 0.535]
W_v sv12 [5.66  5.424 5.182 5.159 5.074 5.008 4.829 4.685 4.551 4.473 4.437 4.367] energy ranks 1,5,10,28,64 [0.021 0.091 0.162 0.351 0.596]
W_o sv12 [10.477  7.357  5.323  5.294  5.177  5.124  5.049  4.959  4.775  4.751  4.625  4.565] energy ranks 1,5,10,28,64 [0.068 0.153 0.229 0.419 0.655]
W_fc sv12 [15.114  8.611  8.486  8.073  7.769  7.529  7.449  7.334  7.094  6.971  6.891  6.808] energy ranks 1,5,10,28,64 [0.037 0.082 0.125 0.238 0.403]
W_proj sv12 [28.461 22.214 21.34  20.708 19.943 19.662 18.986 18.744 18.295 17.457 17.132 16.561] energy ranks 1,5,10,28,64 [0.112 0.357 0.597 0.938 0.966]
L3H0 QK eig12 [0.027+0.179j 0.027-0.179j 0.11 -0.076j 0.11 +0.076j 0.129-0.006j 0.129+0.006j 0.116-0.048j 0.116+0.048j 0.069-0.095j 0.069+0.095j 0.096+0.067j
 0.096-0.067j] keybias_norm=0.501 keybias_topright=0.702
L3H1 QK eig12 [ 0.037+0.098j  0.037-0.098j  0.078+0.j     0.006+0.077j  0.006-0.077j  0.067+0.j    -0.06 -0.013j -0.06 +0.013j  0.042-0.042j  0.042+0.042j
  0.059+0.j     0.052+0.023j] keybias_norm=0.280 keybias_topright=0.428
L3H2 QK eig12 [ 0.036+0.157j  0.036-0.157j  0.081+0.j     0.042+0.065j  0.042-0.065j  0.071-0.018j  0.071+0.018j -0.071+0.j    -0.066-0.007j -0.066+0.007j
 -0.03 +0.059j -0.03 -0.059j] keybias_norm=0.495 keybias_topright=-0.881
L3H3 QK eig12 [0.188+0.006j 0.188-0.006j 0.18 +0.003j 0.18 -0.003j 0.169+0.037j 0.169-0.037j 0.168+0.009j 0.168-0.009j 0.162+0.04j  0.162-0.04j  0.159-0.01j
 0.159+0.01j ] keybias_norm=0.221 keybias_topright=-0.205
L3H4 QK eig12 [ 0.031+0.026j  0.031-0.026j  0.002+0.039j  0.002-0.039j  0.03 +0.008j  0.03 -0.008j -0.029+0.011j -0.029-0.011j  0.013-0.024j  0.013+0.024j
 -0.005-0.026j -0.005+0.026j] keybias_norm=0.144 keybias_topright=0.225
L3H5 QK eig12 [0.021+0.144j 0.021-0.144j 0.103+0.j    0.093+0.027j 0.093-0.027j 0.094+0.005j 0.094-0.005j 0.067+0.063j 0.067-0.063j 0.066+0.057j 0.066-0.057j
 0.048+0.063j] keybias_norm=0.493 keybias_topright=-0.861
LN1 largest bias coords [(58, np.float64(0.8241215348243713)), (99, np.float64(-0.7638827562332153)), (5, np.float64(0.43600863218307495)), (383, np.float64(-0.39739054441452026)), (266, np.float64(0.3425202965736389)), (21, np.float64(-0.31522801518440247)), (184, np.float64(-0.2888939678668976)), (45, np.float64(0.2887653410434723))]

FIRST-LAYER HEAD QK AND OV COSINES
QK
[[ 1.     0.012  0.007  0.137  0.013  0.044]
 [ 0.012  1.     0.69  -0.026  0.57   0.026]
 [ 0.007  0.69   1.    -0.063  0.492 -0.075]
 [ 0.137 -0.026 -0.063  1.    -0.044  0.01 ]
 [ 0.013  0.57   0.492 -0.044  1.     0.033]
 [ 0.044  0.026 -0.075  0.01   0.033  1.   ]]
OV
[[ 1.     0.018  0.017 -0.035  0.026  0.018]
 [ 0.018  1.     0.925  0.006  0.868  0.643]
 [ 0.017  0.925  1.     0.009  0.81   0.692]
 [-0.035  0.006  0.009  1.     0.011 -0.001]
 [ 0.026  0.868  0.81   0.011  1.     0.592]
 [ 0.018  0.643  0.692 -0.001  0.592  1.   ]]

```