NORM PARAMETER SCALES
Values below are RMS of the learned diagonal factor (1 + weight).
layer    input      post-attn    pre-ff       post-ff      q           k
 0        14.55       2.396       6.132       6.865       2.119       2.146
 1        26.73       9.365       31.32       9.372       2.994       2.898
 2        32.79       5.446       25.16       14.16       1.871       2.747
 3        30.06       12.53       32.24       35.57       1.658       1.963
 4         50.3       13.17       17.01       28.05       1.915       2.933
 5        17.93       28.76       15.18       34.17        2.32       3.987
 6         27.4       19.14       9.845       35.87       1.579       2.204
 7        25.96       21.14       9.025       33.88           2        2.49
 8         26.1       30.62       8.426       37.79       2.021       2.437
 9        22.48       32.78       5.509        43.1       2.054        2.47
10         25.5       57.61       7.305       74.78        1.82       3.702
11        9.733       185.2       9.469       147.5       1.859       4.787
12        31.13       201.8       6.029       207.1       2.003       2.767
13        21.36       166.2       5.767       215.1       1.851       2.381
14        30.13       188.8       5.388       246.6       2.249       3.204
15        38.58       246.2       3.995       292.5       2.569        2.53
16        39.69       163.4        3.89       340.3       2.262       2.534
17        23.13       157.2       4.204       376.5       2.538       3.059
Final norm factor quantiles: [  0.0859   5.2188   9.375   56.775  301.    ]
Final norm difference: 0.0 changed coordinates: 0
Actual stored dtypes: ['BF16']

OUTPUT-WEIGHT DIFFERENCES, B-A
Approximate leading singular energy fractions; not behavioral importance.
L00 mlp       raw_norm=0.26064 scaled_norm=1.80363 energy1..4=[0.7957 0.0233 0.0188 0.0143] resid1=7.52e-08
L00 attention raw_norm=0.18165 scaled_norm=0.42613 energy1..4=[0.7332 0.0726 0.0217 0.0215] resid1=1.65e-07
L01 mlp       raw_norm=0.25805 scaled_norm=2.69293 energy1..4=[0.7583 0.023  0.0169 0.016 ] resid1=1.17e-08
L01 attention raw_norm=0.17721 scaled_norm=1.82593 energy1..4=[0.2951 0.1511 0.1066 0.0736] resid1=2.01e-07
L02 mlp       raw_norm=0.25233 scaled_norm=4.25475 energy1..4=[0.6383 0.0379 0.0254 0.0211] resid1=1.88e-08
L02 attention raw_norm=0.18127 scaled_norm=1.11459 energy1..4=[0.736  0.0413 0.0271 0.0167] resid1=1.26e-07
L03 mlp       raw_norm=0.25263 scaled_norm=9.41848 energy1..4=[0.5535 0.1483 0.0212 0.0165] resid1=1.42e-07
L03 attention raw_norm=0.17426 scaled_norm=2.77443 energy1..4=[0.7007 0.0419 0.029  0.0234] resid1=5.42e-08
L04 mlp       raw_norm=0.25449 scaled_norm=7.29887 energy1..4=[0.7885 0.1378 0.0042 0.0036] resid1=7.36e-09
L04 attention raw_norm=0.18244 scaled_norm=2.83297 energy1..4=[0.6057 0.2934 0.0189 0.0098] resid1=2.57e-08
L05 mlp       raw_norm=0.25727 scaled_norm=8.21517 energy1..4=[0.7498 0.0986 0.0194 0.0089] resid1=1.32e-08
L05 attention raw_norm=0.19373 scaled_norm=5.54001 energy1..4=[0.4473 0.1955 0.129  0.0289] resid1=2.87e-08
L06 mlp       raw_norm=0.25183 scaled_norm=9.12482 energy1..4=[0.7754 0.05   0.0138 0.0083] resid1=1.9e-08
L06 attention raw_norm=0.18784 scaled_norm=3.48165 energy1..4=[0.5466 0.1457 0.0433 0.0254] resid1=9.88e-08
L07 mlp       raw_norm=0.24943 scaled_norm=8.27223 energy1..4=[0.624  0.0436 0.0277 0.0198] resid1=7.62e-08
L07 attention raw_norm=0.18032 scaled_norm=3.60110 energy1..4=[0.4946 0.1246 0.0414 0.029 ] resid1=2.87e-08
L08 mlp       raw_norm=0.24604 scaled_norm=9.05877 energy1..4=[0.5219 0.0803 0.0434 0.0258] resid1=8.16e-08
L08 attention raw_norm=0.18011 scaled_norm=5.30286 energy1..4=[0.28   0.1561 0.0742 0.0454] resid1=2.98e-07
L09 mlp       raw_norm=0.25422 scaled_norm=10.74524 energy1..4=[0.1991 0.0961 0.0664 0.0393] resid1=1.51e-06
L09 attention raw_norm=0.18835 scaled_norm=6.09355 energy1..4=[0.2899 0.1015 0.0608 0.0531] resid1=1.83e-07
L10 mlp       raw_norm=0.25704 scaled_norm=19.42693 energy1..4=[0.1187 0.1098 0.0666 0.0493] resid1=1.15e-05
L10 attention raw_norm=0.20337 scaled_norm=11.67267 energy1..4=[0.1119 0.1055 0.059  0.0445] resid1=4.49e-05
L11 mlp       raw_norm=0.24584 scaled_norm=36.58024 energy1..4=[0.0927 0.0783 0.0587 0.0487] resid1=0.000191
L11 attention raw_norm=0.19571 scaled_norm=36.18048 energy1..4=[0.0639 0.0532 0.0432 0.0411] resid1=0.00234
L12 mlp       raw_norm=0.26587 scaled_norm=54.25157 energy1..4=[0.0699 0.045  0.0398 0.0379] resid1=0.000628
L12 attention raw_norm=0.19639 scaled_norm=38.92811 energy1..4=[0.0741 0.0627 0.0491 0.0453] resid1=0.000633
L13 mlp       raw_norm=0.27298 scaled_norm=57.90805 energy1..4=[0.0626 0.0368 0.0359 0.034 ] resid1=0.00158
L13 attention raw_norm=0.19284 scaled_norm=31.52170 energy1..4=[0.0642 0.0554 0.0464 0.0377] resid1=0.00168
L14 mlp       raw_norm=0.27840 scaled_norm=68.06571 energy1..4=[0.0551 0.0391 0.0338 0.0318] resid1=0.00128
L14 attention raw_norm=0.18524 scaled_norm=34.88332 energy1..4=[0.1439 0.0599 0.0396 0.0295] resid1=3.23e-05
L15 mlp       raw_norm=0.28416 scaled_norm=82.97353 energy1..4=[0.058  0.0497 0.0481 0.0353] resid1=0.00113
L15 attention raw_norm=0.19383 scaled_norm=46.72399 energy1..4=[0.075  0.057  0.0468 0.0433] resid1=0.001
L16 mlp       raw_norm=0.27826 scaled_norm=94.81598 energy1..4=[0.0926 0.0739 0.0517 0.0453] resid1=0.000298
L16 attention raw_norm=0.18382 scaled_norm=29.95256 energy1..4=[0.106  0.0648 0.0597 0.0472] resid1=4.86e-05
L17 mlp       raw_norm=0.26354 scaled_norm=99.89915 energy1..4=[0.1106 0.067  0.0536 0.0492] resid1=0.000117
L17 attention raw_norm=0.17479 scaled_norm=27.35824 energy1..4=[0.12   0.0447 0.0406 0.0385] resid1=4.64e-05

English-looking vocabulary subset size: 139429

VOCABULARY ALIGNMENTS OF DIFFERENCE AXES
These are weight-direction cosines, NOT logits or predictions.

L00 mlp, top energy fractions [0.7957 0.0233]
 axis 1 English +: [("'setPrototypeOf'", 0.0148), ("' yang'", 0.0144), ("'ptide'", 0.0142), ("' ce'", 0.0138), ("'ilberger'", 0.0135), ("'ccak'", 0.0129), ("' then'", 0.0129)]
 axis 1 English -: [("'dni'", -0.0279), ("'nlm'", -0.0251), ("' Namely'", -0.0231), ("'npmjs'", -0.0216), ("'tio'", -0.0216), ("'sala'", -0.0213), ("'marca'", -0.0213)]
 all-vocabulary +: [("',’”'", 0.0161), ("'同'", 0.0153), ("'పంచ'", 0.015), ("'setPrototypeOf'", 0.0148)]  -: [("'<tfoot>'", -0.0296), ("'dni'", -0.0279), ("'nlm'", -0.0251), ("'))*'", -0.0248)]
 axis 2 English +: [("' rspace'", 0.1195), ("'onics'", 0.1124), ("' con'", 0.1086), ("' takes'", 0.1056), ("' despite'", 0.1023), ("' take'", 0.1009), ("' rest'", 0.0993)]
 axis 2 English -: [("'ramethyl'", -0.1411), ("'ywidual'", -0.1073), ("'imsy'", -0.1043), ("'indd'", -0.1039), ("' LoginForm'", -0.1028), ("'iexpress'", -0.1025), ("'ufth'", -0.1009)]

L04 mlp, top energy fractions [0.7885 0.1378]
 axis 1 English +: [("'ar'", 0.0051), ("' ce'", 0.0047), ("'is'", 0.0036), ("'and'", 0.0036), ("'at'", 0.0035), ("'ut'", 0.0035), ("' kan'", 0.0035)]
 axis 1 English -: [("' magnetores'", -0.0061), ("'atinum'", -0.0061), ("'zkod'", -0.0061), ("' onlyOwner'", -0.0058), ("' Namely'", -0.0057), ("' browserSync'", -0.0057), ("'elessly'", -0.0056)]
 all-vocabulary +: [("'ar'", 0.0051), ("' ce'", 0.0047), ("' ='", 0.0038), ("'」'", 0.0038)]  -: [("'<tfoot>'", -0.0074), ("' []:'", -0.007), ("'ṇa'", -0.0067), ("'वेश'", -0.0065)]
 axis 2 English +: [("'setPrototypeOf'", 0.0155), ("' ce'", 0.0137), ("'ar'", 0.0131), ("' then'", 0.0125), ("' sur'", 0.0122), ("' at'", 0.0121), ("'romino'", 0.0117)]
 axis 2 English -: [("' Namely'", -0.0208), ("'dni'", -0.0191), ("'nlm'", -0.0189), ("'zkod'", -0.0183), ("'atinum'", -0.0182), ("'newblock'", -0.0179), ("' magnetores'", -0.0178)]

L06 mlp, top energy fractions [0.7754 0.05  ]
 axis 1 English +: [("'ar'", 0.0032), ("'on'", 0.0032), ("'is'", 0.0029), ("'em'", 0.0028), ("'en'", 0.0027), ("' is'", 0.0026), ("'us'", 0.0026)]
 axis 1 English -: [("' browserSync'", -0.0048), ("'thorace'", -0.0048), ("' autoWatch'", -0.0047), ("' renderEncoder'", -0.0046), ("' artistdata'", -0.0045), ("' menuMobile'", -0.0045), ("'LuaPush'", -0.0045)]
 all-vocabulary +: [("'\\n'", 0.0032), ("'ar'", 0.0032), ("'on'", 0.0032), ("'S'", 0.0031)]  -: [("' *}('", -0.0051), ("' দেই'", -0.0051), ("' *}\\\\'", -0.005), ("'<unused1563>'", -0.0049)]
 axis 2 English +: [("'setPrototypeOf'", 0.0175), ("'index'", 0.0173), ("'requireNonNull'", 0.0166), ("' di'", 0.016), ("'is'", 0.0159), ("'Share'", 0.0158), ("'ut'", 0.0157)]
 axis 2 English -: [("'dni'", -0.021), ("'atinum'", -0.0184), ("'oconvex'", -0.0183), ("'quiera'", -0.0182), ("'infini'", -0.0174), ("'anych'", -0.0173), ("'udsman'", -0.0172)]

L01 mlp, top energy fractions [0.7583 0.023 ]
 axis 1 English +: [("'setPrototypeOf'", 0.0195), ("'romino'", 0.0189), ("'ocatore'", 0.0186), ("'StdHandle'", 0.0181), ("' UserManager'", 0.0178), ("' Zij'", 0.0177), ("'itriangular'", 0.0177)]
 axis 1 English -: [("'dni'", -0.0177), ("'newblock'", -0.0145), ("'tio'", -0.0141), ("'adoria'", -0.0137), ("'nlm'", -0.0133), ("'diphenyl'", -0.0133), ("'ibacter'", -0.0129)]
 all-vocabulary +: [("'」'", 0.0198), ("'setPrototypeOf'", 0.0195), ("'новні'", 0.0192), ("'romino'", 0.0189)]  -: [("'<tfoot>'", -0.0181), ("'جراء'", -0.0179), ("'dni'", -0.0177), ("' каждым'", -0.0169)]
 axis 2 English +: [("'It'", 0.0733), ("'Le'", 0.0727), ("'there'", 0.0683), ("'If'", 0.0632), ("'There'", 0.063), ("'has'", 0.0625), ("'Used'", 0.0611)]
 axis 2 English -: [("'bandingan'", -0.0714), ("'rophes'", -0.0686), ("'ctree'", -0.0669), ("'perclip'", -0.0667), ("' utmost'", -0.0662), ("'ollo'", -0.0608), ("'fice'", -0.0601)]

L05 mlp, top energy fractions [0.7498 0.0986]
 axis 1 English +: [("'ar'", 0.0035), ("'ol'", 0.0031), ("'will'", 0.003), ("'is'", 0.003), ("' ce'", 0.0029), ("' como'", 0.0029), ("'ic'", 0.0029)]
 axis 1 English -: [("'atinum'", -0.0043), ("'tive'", -0.0042), ("' Namely'", -0.0041), ("' magnetores'", -0.0041), ("' linestyle'", -0.004), ("' MaterialApp'", -0.004), ("' useNewUrlParser'", -0.0039)]
 all-vocabulary +: [('"\'"', 0.0037), ("'ar'", 0.0035), ("'\\xad'", 0.0033), ("' d'", 0.0033)]  -: [("' দিবেন'", -0.0045), ("' []:'", -0.0045), ("' আশায়'", -0.0045), ("'<tfoot>'", -0.0044)]
 axis 2 English +: [("'setPrototypeOf'", 0.0174), ("'ilberger'", 0.0165), ("'ocatore'", 0.0164), ("'itriangular'", 0.0153), ("'romino'", 0.0145), ("'oncesto'", 0.0143), ("'requireNonNull'", 0.0136)]
 axis 2 English -: [("'anych'", -0.0184), ("'beitung'", -0.0176), ("' Namely'", -0.0173), ("'zkod'", -0.0172), ("'atinum'", -0.0169), ("'dni'", -0.0166), ("'propoxy'", -0.0164)]

L02 attention, top energy fractions [0.736  0.0413]
 axis 1 English +: [("'ut'", 0.0183), ("'acute'", 0.0166), ("'setPrototypeOf'", 0.0163), ("'ocatore'", 0.0162), ("'ir'", 0.0158), ("'StdHandle'", 0.0158), ("'ID'", 0.0157)]
 axis 1 English -: [("'dni'", -0.0181), ("'newblock'", -0.0168), ("'tio'", -0.0168), ("'diphenyl'", -0.0167), ("'dropna'", -0.0166), ("'fice'", -0.0165), ("'npmjs'", -0.0164)]
 all-vocabulary +: [("'」'", 0.0201), ("'ut'", 0.0183), ("'новні'", 0.0179), ("'னியா'", 0.0174)]  -: [('\'="#">\'', -0.0208), ("'<tfoot>'", -0.0207), ("'蟆'", -0.0206), ("'هَا'", -0.0196)]
 axis 2 English +: [("'eer'", 0.029), ("'eem'", 0.0277), ("'ingar'", 0.0255), ("'es'", 0.0242), ("'een'", 0.0239), ("'iado'", 0.0239), ("'icionar'", 0.0228)]
 axis 2 English -: [("'reducer'", -0.0297), ("'MEAS'", -0.0286), ("'Red'", -0.0285), ("'Em'", -0.0283), ("'itta'", -0.0275), ("'ratio'", -0.0275), ("'Paras'", -0.0272)]

L00 attention, top energy fractions [0.7332 0.0726]
 axis 1 English +: [("'TestSuite'", 0.0188), ("'StdHandle'", 0.0184), ("' ce'", 0.0174), ("'setPrototypeOf'", 0.0163), ("'ar'", 0.0163), ("'roweak'", 0.0159), ("'ut'", 0.0157)]
 axis 1 English -: [("'crollView'", -0.0189), ("'CloseOperation'", -0.0189), ("' thereto'", -0.0185), ("'atinum'", -0.0185), ("'zkod'", -0.0179), ("'anych'", -0.0178), ("'lgende'", -0.0177)]
 all-vocabulary +: [("'TestSuite'", 0.0188), ("'StdHandle'", 0.0184), ("'外'", 0.0179), ("' ce'", 0.0174)]  -: [("'<tfoot>'", -0.0284), ("'))*'", -0.0205), ("'鹕'", -0.0197), ("'蟆'", -0.0195)]
 axis 2 English +: [("'NIS'", 0.0531), ("'zantine'", 0.049), ("'LER'", 0.0488), ("'EK'", 0.0482), ("'VERTISEMENT'", 0.0461), ("'ZL'", 0.0458), ("'gine'", 0.0455)]
 axis 2 English -: [("' Qaraldi'", -0.0631), ("'ierung'", -0.0611), ("'glial'", -0.0586), ("'gestaltung'", -0.0582), ("'NullException'", -0.0576), ("' powie'", -0.0575), ("'abilistic'", -0.0555)]

L03 attention, top energy fractions [0.7007 0.0419]
 axis 1 English +: [("'defineProperty'", 0.0202), ("'requireNonNull'", 0.0188), ("'ut'", 0.0187), ("'ir'", 0.0182), ("' ce'", 0.0178), ("'StdHandle'", 0.0173), ("'setPrototypeOf'", 0.0171)]
 axis 1 English -: [("'dni'", -0.0207), ("'labelledby'", -0.0197), ("'maxcdn'", -0.0197), ("'diphenyl'", -0.0194), ("'quiera'", -0.0183), ("'retch'", -0.0181), ("' yandan'", -0.018)]
 all-vocabulary +: [("'новні'", 0.0211), ("'defineProperty'", 0.0202), ("'」'", 0.0194), ("'當'", 0.0193)]  -: [("'هَا'", -0.0214), ("'dni'", -0.0207), ("'<tfoot>'", -0.02), ("'labelledby'", -0.0197)]
 axis 2 English +: [("'isnull'", 0.0348), ("'agel'", 0.0332), ("'atol'", 0.0313), ("'arci'", 0.0309), ("'shint'", 0.0305), ("'ricies'", 0.0303), ("'sic'", 0.0302)]
 axis 2 English -: [("' la'", -0.0314), ("'ViewHolder'", -0.0311), ("' took'", -0.0311), ("'ptidase'", -0.031), ("'subTest'", -0.0306), ("' held'", -0.0305), ("'izioni'", -0.0304)]

L02 mlp, top energy fractions [0.6383 0.0379]
 axis 1 English +: [("'ut'", 0.0183), ("'defineProperty'", 0.0171), ("'ocatore'", 0.0171), ("'StdHandle'", 0.017), ("'ir'", 0.0168), ("'index'", 0.0163), ("' ce'", 0.0162)]
 axis 1 English -: [("'maxcdn'", -0.0188), ("'labelledby'", -0.0182), ("'dni'", -0.0176), ("'quiera'", -0.0176), ("'diphenyl'", -0.0171), ("'zkod'", -0.0168), ("'monium'", -0.0166)]
 all-vocabulary +: [("'」'", 0.0188), ("'ut'", 0.0183), ("'новні'", 0.0177), ("'defineProperty'", 0.0171)]  -: [("'<tfoot>'", -0.0233), ("'هَا'", -0.0225), ("'جراء'", -0.0197), ("'ẵ'", -0.0189)]
 axis 2 English +: [("'tas'", 0.0491), ("'torrent'", 0.0472), ("'tand'", 0.0449), ("'tic'", 0.0444), ("'ektion'", 0.0429), ("'middels'", 0.042), ("'skar'", 0.0413)]
 axis 2 English -: [("'irez'", -0.0419), ("'seau'", -0.0387), ("'gebied'", -0.0384), ("'ostino'", -0.0372), ("'inception'", -0.0358), ("'intera'", -0.0357), ("'imenti'", -0.0355)]

L07 mlp, top energy fractions [0.624  0.0436]
 axis 1 English +: [("'causing'", 0.0044), ("'negl'", 0.004), ("'ction'", 0.004), ("' blame'", 0.0039), ("'oping'", 0.0038), ("'ennem'", 0.0038), ("' Located'", 0.0038)]
 axis 1 English -: [("' lancio'", -0.0028), ("'lettes'", -0.0027), ("'ainer'", -0.0026), ("'lette'", -0.0026), ("' Epub'", -0.0025), ("'setType'", -0.0025), ("' lanzamiento'", -0.0023)]
 all-vocabulary +: [("'causing'", 0.0044), ("' ↑'", 0.0042), ("'negl'", 0.004), ("'пова'", 0.004)]  -: [("' শুদ্ধ'", -0.0034), ("' জলে'", -0.0034), ("'ভিত্তিক'", -0.0034), ("' সমঝো'", -0.0033)]
 axis 2 English +: [("'anche'", 0.0163), ("'iers'", 0.0157), ("'romino'", 0.0152), ("'resse'", 0.0149), ("'authors'", 0.0148), ("' readers'", 0.0147), ("'irce'", 0.0144)]
 axis 2 English -: [("'ction'", -0.0329), ("' situate'", -0.0293), ("' adjectives'", -0.0293), ("'evich'", -0.0282), ("' aras'", -0.0281), ("' ceinture'", -0.0278), ("'atero'", -0.0276)]

L17 mlp, top energy fractions [0.1106 0.067 ]
 axis 1 English +: [("'dcsset'", 0.0803), ("' threx'", 0.0792), ("'LookAndFeelInfo'", 0.0776), ("'INTRESOURCE'", 0.0775), ("'DebuggingMode'", 0.0764), ("'QAry'", 0.0759), ("'actionBarTab'", 0.0759)]
 axis 1 English -: [("' in'", -0.0387), ("' and'", -0.0354), ("' In'", -0.033), ("' one'", -0.0324), ("'one'", -0.0324), ("' to'", -0.0322), ("' un'", -0.0318)]
 all-vocabulary +: [("'<unused690>'", 0.0881), ("'<unused691>'", 0.0859), ("'㜌'", 0.0855), ("'<unused1806>'", 0.0847)]  -: [("'1'", -0.053), ("' '", -0.0476), ("'\\n'", -0.0426), ("'*'", -0.0422)]
 axis 2 English +: [("' two'", 0.0382), ("'two'", 0.0376), ("' Two'", 0.0292), ("'Two'", 0.0263), ("' dvije'", 0.0221), ("' second'", 0.0213), ("'forth'", 0.02)]
 axis 2 English -: [("' Sixty'", -0.0421), ("' compren'", -0.0419), ("'PTMR'", -0.0414), ("'StubCompat'", -0.0413), ("'StatusInternal'", -0.0407), ("'StateManager'", -0.0405), ("'MDEwOlJlcG'", -0.0403)]

L17 attention, top energy fractions [0.12   0.0447]
 axis 1 English +: [("' sanitizer'", 0.0303), ("' mujhe'", 0.0302), ("'reibung'", 0.03), ("' creatinine'", 0.0293), ("' eest'", 0.0289), ("'onacci'", 0.0289), ("' Serengeti'", 0.0289)]
 axis 1 English -: [("'yet'", -0.0237), ("'aff'", -0.0219), ("' renferme'", -0.0216), ("'ana'", -0.0208), ("'est'", -0.0203), ("'udd'", -0.0203), ("' verden'", -0.0201)]
 all-vocabulary +: [("'\\\\:'", 0.0318), ("' sanitizer'", 0.0303), ("' mujhe'", 0.0302), ("'reibung'", 0.03)]  -: [("'会将'", -0.0237), ("'yet'", -0.0237), ("'쉽'", -0.0235), ("'ны'", -0.0228)]
 axis 2 English +: [("' and'", 0.0218), ("' sub'", 0.0186), ("' to'", 0.0176), ("' red'", 0.0172), ("' in'", 0.0158), ("' or'", 0.0157), ("' as'", 0.015)]
 axis 2 English -: [("'Therates'", -0.0634), ("' SwitchCompat'", -0.0631), ("'ovali'", -0.0631), ("' temos'", -0.0629), ("' FXR'", -0.0629), ("' canaliculato'", -0.0628), ("' threx'", -0.0627)]

CROSS-COMPONENT ALIGNMENT OF LEADING DIFFERENCE AXES
L06/mlp vs L05/mlp: |cos|=0.9958
L02/attention vs L01/mlp: |cos|=0.9958
L07/attention vs L06/mlp: |cos|=0.9958
L01/mlp vs L00/mlp: |cos|=0.9952
L01/mlp vs L00/attention: |cos|=0.9945
L06/attention vs L05/mlp: |cos|=0.9944
L04/mlp vs L03/mlp: |cos|=0.9941
L04/attention vs L03/mlp: |cos|=0.9933
L08/mlp vs L07/mlp: |cos|=0.9932
L05/mlp vs L04/mlp: |cos|=0.9926
L00/attention vs L00/mlp: |cos|=0.9926
L02/attention vs L02/mlp: |cos|=0.9924
