VOCABULARY METRIC
Sample size: 32768
Covariance eigenvalue quantiles: [  0.0001   0.0212   0.0748   2.7958 127.6836]
Top 3 eigenvalue energy share: 0.15600397851200448

VOCABULARY-VISIBLE OUTPUT-DIFFERENCE SPECTRA
Scale is RMS across sampled vocabulary rows, summed over input columns.
Neither scale nor spectral concentration is causal importance.
L00 | mlp: scale=1.0688, fractions=[0.3761 0.1264 0.0842] | attention: scale=0.1871, fractions=[0.2376 0.1321 0.099 ]
L01 | mlp: scale=1.1941, fractions=[0.3548 0.1462 0.0638] | attention: scale=1.1068, fractions=[0.3264 0.2032 0.0804]
L02 | mlp: scale=1.5366, fractions=[0.192  0.0663 0.0535] | attention: scale=0.4094, fractions=[0.3277 0.117  0.0768]
L03 | mlp: scale=3.0349, fractions=[0.087  0.0632 0.0509] | attention: scale=0.8995, fractions=[0.3103 0.0819 0.0569]
L04 | mlp: scale=1.4135, fractions=[0.1826 0.079  0.0625] | attention: scale=0.5541, fractions=[0.3431 0.1496 0.0577]
L05 | mlp: scale=1.9305, fractions=[0.2382 0.1011 0.0556] | attention: scale=1.9312, fractions=[0.3457 0.1548 0.1063]
L06 | mlp: scale=2.0493, fractions=[0.2195 0.0745 0.0469] | attention: scale=0.9499, fractions=[0.2814 0.1043 0.0715]
L07 | mlp: scale=2.1500, fractions=[0.2005 0.0754 0.0533] | attention: scale=1.2439, fractions=[0.239  0.0799 0.0648]
L08 | mlp: scale=2.2598, fractions=[0.2024 0.0911 0.0496] | attention: scale=1.4661, fractions=[0.2106 0.1022 0.0786]
L09 | mlp: scale=3.2160, fractions=[0.1688 0.0924 0.0624] | attention: scale=1.7396, fractions=[0.2083 0.0822 0.0783]
L10 | mlp: scale=5.8571, fractions=[0.1698 0.1039 0.0715] | attention: scale=3.1221, fractions=[0.1667 0.1006 0.0686]
L11 | mlp: scale=9.9496, fractions=[0.157  0.1018 0.0841] | attention: scale=7.5209, fractions=[0.1063 0.0631 0.0469]
L12 | mlp: scale=14.8247, fractions=[0.1661 0.0989 0.0524] | attention: scale=8.4407, fractions=[0.1278 0.0659 0.0525]
L13 | mlp: scale=16.1750, fractions=[0.1761 0.0632 0.0462] | attention: scale=8.5063, fractions=[0.2048 0.0943 0.0616]
L14 | mlp: scale=20.8396, fractions=[0.2055 0.0922 0.0469] | attention: scale=11.7270, fractions=[0.2992 0.0902 0.0543]
L15 | mlp: scale=24.7238, fractions=[0.1628 0.1329 0.049 ] | attention: scale=11.4828, fractions=[0.1527 0.0828 0.0499]
L16 | mlp: scale=30.2426, fractions=[0.2635 0.1466 0.0457] | attention: scale=10.4124, fractions=[0.2654 0.1249 0.0529]
L17 | mlp: scale=48.8722, fractions=[0.5024 0.1769 0.0331] | attention: scale=9.2327, fractions=[0.215  0.116  0.0888]

SEMANTICS OF VOCABULARY-VISIBLE DIFFERENCE AXES
Numbers are static weight projections divided by sampled RMS.
ASCII-like tokens are not necessarily English words.

L00 mlp; first two energy fractions=[0.3761 0.1264]
 axis1 ASCII +: [("' TR'", 3.397), ("' IL'", 3.389), ("' EN'", 3.337), ("' Ken'", 3.298), ("' Ay'", 3.29)]
 axis1 ASCII -: [("'DataManager'", -4.102), ("'buildCommand'", -3.519), ("'IPTS'", -3.426), ("'jumpa'", -3.383), ("'temuan'", -3.371)]
 all tokens +: [("' TR'", 3.397), ("' IL'", 3.389), ("' EN'", 3.337)]  -: [("'DataManager'", -4.102), ("'手数'", -3.818), ("'buildCommand'", -3.519)]
 row-length-controlled +: [("' Kang'", 3.568), ("' EN'", 3.554), ("' RT'", 3.529), ("' BL'", 3.519)]  -: [("'ytical'", -3.198), ("'tenance'", -3.069), ("'polyfills'", -3.016), ("'AndroidJUnit'", -3.011)]
 axis2 ASCII +: [("' rspace'", 5.11), ("' results'", 4.783), ("'ilberger'", 4.457), ("'iyeti'", 4.432), ("' even'", 4.377)]
 axis2 ASCII -: [("'oblot'", -4.392), ("' Watercolour'", -4.199), ("' Gorgeous'", -4.161), ("'Buffered'", -4.148), ("'ywidual'", -4.139)]

L14 mlp; first two energy fractions=[0.2055 0.0922]
 axis1 ASCII +: [("'izowane'", 3.086), ("'mainan'", 3.027), ("'leyebilirsiniz'", 3.024), ("' probabilidades'", 2.961), ("'kaisi'", 2.945)]
 axis1 ASCII -: [("' Mol'", -3.561), ("'Ren'", -3.533), ("'Ak'", -3.462), ("'Sol'", -3.454), ("'Lin'", -3.425)]
 all tokens +: [("'izowane'", 3.086), ("'mainan'", 3.027), ("'leyebilirsiniz'", 3.024)]  -: [("' Mol'", -3.561), ("'Ren'", -3.533), ("'Ak'", -3.462)]
 row-length-controlled +: [("' legato'", 2.868), ("'leyebilirsiniz'", 2.819), ("'zeptember'", 2.801), ("'cimientos'", 2.786)]  -: [("'Mol'", -4.13), ("'Ta'", -3.902), ("'Mo'", -3.811), ("'Dal'", -3.765)]
 axis2 ASCII +: [("' in'", 6.061), ("' and'", 5.537), ("' on'", 5.501), ("' to'", 5.49), ("' for'", 5.338)]
 axis2 ASCII -: [("' Chirurgien'", -3.265), ("' RUDDER'", -2.934), ("'ClkInitStruct'", -2.906), ("' alcoved'", -2.875), ("' Oldborough'", -2.855)]

L14 attention; first two energy fractions=[0.2992 0.0902]
 axis1 ASCII +: [("'abila'", 3.482), ("'jahteraan'", 3.329), ("' cookieParser'", 3.231), ("' concursos'", 3.198), ("'izowane'", 3.133)]
 axis1 ASCII -: [("' Bl'", -3.538), ("' SH'", -3.446), ("' SL'", -3.434), ("' LT'", -3.421), ("' Roger'", -3.419)]
 all tokens +: [("'abila'", 3.482), ("'jahteraan'", 3.329), ("' />}></'", 3.317)]  -: [("' Bl'", -3.538), ("' SH'", -3.446), ("' SL'", -3.434)]
 row-length-controlled +: [("' cookieParser'", 3.343), ("' drawRight'", 3.199), ("' menuMobile'", 3.19), ("'abila'", 3.061)]  -: [("' LT'", -3.824), ("'Se'", -3.63), ("' TX'", -3.511), ("' Rem'", -3.5)]
 axis2 ASCII +: [("' in'", 5.864), ("' as'", 5.331), ("' the'", 5.193), ("' on'", 5.065), ("' new'", 5.025)]
 axis2 ASCII -: [("' JACKSON'", -3.243), ("' BLUEN'", -3.218), ("'linkedExternal'", -3.18), ("' WARREN'", -3.146), ("' envoyer'", -3.061)]

L15 mlp; first two energy fractions=[0.1628 0.1329]
 axis1 ASCII +: [("' in'", 5.096), ("' to'", 4.672), ("' for'", 4.54), ("' and'", 4.407), ("' same'", 4.379)]
 axis1 ASCII -: [("' Penis'", -3.358), ("' Lola'", -3.155), ("' Mosquito'", -3.115), ("'Luna'", -3.102), ("'Fox'", -3.1)]
 all tokens +: [("' '", 5.429), ("','", 5.249), ("' ('", 5.2)]  -: [("' Penis'", -3.358), ("' Lola'", -3.155), ("' Mosquito'", -3.115)]
 row-length-controlled +: [("' at'", 4.131), ("' in'", 4.078), ("' to'", 3.889), ("' or'", 3.73)]  -: [("'Mol'", -3.817), ("' Lola'", -3.713), ("' Rei'", -3.572), ("' Phyllis'", -3.564)]
 axis2 ASCII +: [("'zynarod'", 2.907), ("' sikap'", 2.817), ("'zovaniyu'", 2.801), ("' BoxFit'", 2.792), ("' cualidades'", 2.766)]
 axis2 ASCII -: [("' sub'", -4.243), ("' St'", -4.105), ("' st'", -4.031), ("' Ben'", -3.986), ("' re'", -3.967)]

L16 mlp; first two energy fractions=[0.2635 0.1466]
 axis1 ASCII +: [("' in'", 6.332), ("' and'", 6.032), ("' to'", 5.789), ("' for'", 5.578), ("' the'", 5.478)]
 axis1 ASCII -: [("'ufieurs'", -3.39), ("'ClkInitStruct'", -3.089), ("' ThemeOverlay'", -3.012), ("'MSFont'", -2.994), ("'MDEwOlJlcG'", -2.97)]
 all tokens +: [("' '", 7.175), ("'.'", 6.887), ("','", 6.802)]  -: [("'<unused930>'", -3.921), ("'<unused453>'", -3.889), ("'<unused481>'", -3.805)]
 row-length-controlled +: [("' in'", 4.609), ("' at'", 4.534), ("' to'", 4.383), ("' and'", 4.337)]  -: [("'ConformanceMode'", -3.601), ("'ClkInitStruct'", -3.401), ("'RAchievement'", -3.265), ("'ufieurs'", -3.157)]
 axis2 ASCII +: [("'ibilidades'", 2.93), ("' velocidades'", 2.874), ("'ungkinan'", 2.867), ("'ruari'", 2.867), ("'temuan'", 2.836)]
 axis2 ASCII -: [("' Ten'", -3.579), ("' Mag'", -3.549), ("' Fox'", -3.524), ("' Ta'", -3.519), ("' Mi'", -3.492)]

L16 attention; first two energy fractions=[0.2654 0.1249]
 axis1 ASCII +: [("'acles'", 3.009), ("' velocidades'", 2.964), ("' nyingi'", 2.915), ("'katapos'", 2.909), ("'izaciones'", 2.898)]
 axis1 ASCII -: [("' NF'", -3.51), ("' Leon'", -3.493), ("' GS'", -3.454), ("' Mik'", -3.398), ("' COMP'", -3.368)]
 all tokens +: [("'ないので'", 3.434), ("'り返'", 3.382), ("'方向に'", 3.295)]  -: [("' NF'", -3.51), ("' Leon'", -3.493), ("' GS'", -3.454)]
 row-length-controlled +: [("' nyingi'", 2.857), ("'iless'", 2.856), ("'acijama'", 2.77), ("' inimes'", 2.724)]  -: [("'Leon'", -3.95), ("'Mol'", -3.913), ("' Craig'", -3.894), ("' Lek'", -3.871)]
 axis2 ASCII +: [("'zynarod'", 2.853), ("'MDEwOlJlcG'", 2.841), ("' MDET'", 2.816), ("' favoriser'", 2.809), ("'ufieurs'", 2.808)]
 axis2 ASCII -: [("' in'", -5.473), ("' and'", -5.355), ("' the'", -5.178), ("' to'", -5.147), ("' sub'", -5.064)]

L17 mlp; first two energy fractions=[0.5024 0.1769]
 axis1 ASCII +: [("'wallepics'", 3.374), ("' coveredmethods'", 3.297), ("'setKeepCallback'", 3.217), ("' Atomnoj'", 3.183), ("' RtlOverlay'", 3.161)]
 axis1 ASCII -: [("' in'", -6.391), ("' the'", -6.257), ("' and'", -5.982), ("' to'", -5.848), ("' In'", -5.787)]
 all tokens +: [("'<unused452>'", 3.855), ("'<unused467>'", 3.773), ("'<unused453>'", 3.736)]  -: [("' '", -7.392), ("'.'", -6.997), ("','", -6.616)]
 row-length-controlled +: [("' coveredmethods'", 3.76), ("' threx'", 3.476), ("'wallepics'", 3.459), ("'setKeepCallback'", 3.348)]  -: [("' In'", -4.725), ("' in'", -4.547), ("' la'", -4.394), ("' at'", -4.36)]
 axis2 ASCII +: [("' Bog'", 3.679), ("' Bon'", 3.538), ("' Nord'", 3.531), ("' Bert'", 3.514), ("' Ken'", 3.513)]
 axis2 ASCII -: [("' funzionamento'", -3.042), ("'temuan'", -3.026), ("' tolerancia'", -2.888), ("' prestaciones'", -2.83), ("' costituito'", -2.826)]

L17 attention; first two energy fractions=[0.215 0.116]
 axis1 ASCII +: [("' and'", 5.169), ("' in'", 5.034), ("' non'", 4.803), ("' en'", 4.71), ("' the'", 4.679)]
 axis1 ASCII -: [("'ufieurs'", -3.25), ("'idcoin'", -3.23), ("'hetiti'", -3.145), ("'cevam'", -3.093), ("'URNIZOR'", -3.069)]
 all tokens +: [("' '", 5.81), ("'.'", 5.583), ("' ('", 5.415)]  -: [("'<unused461>'", -3.548), ("'<unused1674>'", -3.475), ("'<unused477>'", -3.467)]
 row-length-controlled +: [("' con'", 4.094), ("' la'", 3.993), ("' le'", 3.919), ("' pr'", 3.887)]  -: [("' llrp'", -3.303), ("' threx'", -3.284), ("' SwitchCompat'", -3.206), ("' ivvu'", -3.202)]
 axis2 ASCII +: [("' safari'", 3.906), ("' biogas'", 3.878), ("' Pepsi'", 3.811), ("' Argentina'", 3.807), ("' Panama'", 3.756)]
 axis2 ASCII -: [("' in'", -3.441), ("' tehd'", -3.43), ("'ness'", -3.425), ("' isten'", -3.29), ("' avvic'", -3.262)]

SHARED VOCABULARY-VISIBLE AXES
L03/attention vs L02/mlp: |cos|=0.9598
L06/mlp vs L05/mlp: |cos|=0.9545
L07/mlp vs L06/mlp: |cos|=0.9519
L13/attention vs L13/mlp: |cos|=0.9501
L07/attention vs L06/mlp: |cos|=0.9495
L01/attention vs L00/mlp: |cos|=0.9487
L08/mlp vs L07/mlp: |cos|=0.9473
L06/attention vs L06/mlp: |cos|=0.9468
