OUTPUT of run 10 of at most 100: ``` Static MLP read-direction alignments for current / H5-value / H1-value channels. These are products of weight matrices, with no inputs or nonlinearities. rank 20 Channel norm quantiles [[0.353 0.706 1.196] [0.179 0.448 0.697] [0.116 0.361 0.605]] Pair cosine quantiles [[-0.918 -0.776 -0.57 ] [-0.663 -0.443 -0.222] [-0.082 0.15 0.252] [ 0.494 0.589 0.608] [ 0.747 0.744 0.79 ]] all-three pair cosines > 0.5 count 17 all-three pair cosines > 0.65 count 2 all-three pair cosines > 0.8 count 0 best same-direction neurons (index, 3 cosines, 3 norms): 14 [0.758, 0.755, 0.801] [0.386, 0.398, 0.262] 643 [0.703, 0.679, 0.746] [1.141, 0.647, 0.374] 795 [0.612, 0.631, 0.673] [0.473, 0.452, 0.392] 1065 [0.622, 0.609, 0.638] [0.402, 0.341, 0.338] 310 [0.574, 0.59, 0.705] [0.39, 0.569, 0.259] 799 [0.693, 0.573, 0.692] [0.442, 0.5, 0.417] 129 [0.583, 0.567, 0.598] [0.518, 0.477, 0.371] 581 [0.761, 0.566, 0.87] [0.304, 0.511, 0.32] 1265 [0.562, 0.7, 0.62] [0.491, 0.392, 0.387] 85 [0.555, 0.677, 0.573] [0.271, 0.344, 0.2] 26 [0.547, 0.563, 0.676] [0.361, 0.392, 0.383] 325 [0.54, 0.653, 0.677] [0.616, 0.291, 0.333] 1446 [0.54, 0.641, 0.735] [0.558, 0.288, 0.351] 251 [0.669, 0.533, 0.601] [0.38, 0.374, 0.225] 1354 [0.55, 0.518, 0.616] [0.449, 0.275, 0.295] rank 28 Channel norm quantiles [[0.426 0.856 1.341] [0.189 0.471 0.731] [0.134 0.386 0.642]] Pair cosine quantiles [[-0.898 -0.722 -0.509] [-0.629 -0.432 -0.183] [-0.084 0.151 0.22 ] [ 0.471 0.558 0.586] [ 0.711 0.697 0.743]] all-three pair cosines > 0.5 count 13 all-three pair cosines > 0.65 count 1 all-three pair cosines > 0.8 count 0 best same-direction neurons (index, 3 cosines, 3 norms): 14 [0.737, 0.682, 0.745] [0.415, 0.407, 0.268] 795 [0.652, 0.623, 0.64] [0.524, 0.498, 0.412] 251 [0.662, 0.636, 0.605] [0.474, 0.392, 0.257] 1065 [0.668, 0.589, 0.639] [0.484, 0.363, 0.349] 1265 [0.62, 0.647, 0.588] [0.59, 0.44, 0.403] 799 [0.644, 0.582, 0.678] [0.492, 0.525, 0.43] 310 [0.572, 0.601, 0.669] [0.478, 0.593, 0.276] 129 [0.557, 0.603, 0.607] [0.618, 0.495, 0.398] 753 [0.665, 0.555, 0.62] [0.384, 0.417, 0.363] 1206 [0.542, 0.732, 0.565] [0.572, 0.446, 0.359] 581 [0.725, 0.524, 0.78] [0.426, 0.553, 0.348] 85 [0.521, 0.55, 0.533] [0.315, 0.356, 0.22] 26 [0.511, 0.55, 0.661] [0.387, 0.404, 0.395] 325 [0.497, 0.652, 0.622] [0.641, 0.323, 0.34] 1354 [0.489, 0.566, 0.521] [0.525, 0.305, 0.32] Similarities among strongest repeated-direction candidates: [[ 1. -0.01 0.09 0.04 0.31 0.31 -0.01 0.45 0.33 -0.2 -0.07 0.28 0.57] [-0.01 1. 0.35 0.03 0.05 0.27 0.05 -0.06 0.23 0.31 0.18 -0.33 0.03] [ 0.09 0.35 1. -0.02 0.17 0.2 0.61 0.08 0.16 0. -0.09 -0.15 0.26] [ 0.04 0.03 -0.02 1. 0.22 0.07 -0.4 0.14 -0.25 -0.06 -0.59 0.13 -0.24] [ 0.31 0.05 0.17 0.22 1. 0.54 0.21 0.58 0.18 -0.1 0.22 0.04 0.34] [ 0.31 0.27 0.2 0.07 0.54 1. 0.22 0.36 0.32 -0.06 0.2 -0.18 0.21] [-0.01 0.05 0.61 -0.4 0.21 0.22 1. 0.21 0.04 -0.05 0.13 -0.29 0.33] [ 0.45 -0.06 0.08 0.14 0.58 0.36 0.21 1. 0.22 -0.19 0.06 -0.08 0.67] [ 0.33 0.23 0.16 -0.25 0.18 0.32 0.04 0.22 1. 0.48 0.3 -0.05 0.46] [-0.2 0.31 0. -0.06 -0.1 -0.06 -0.05 -0.19 0.48 1. 0.28 0.18 -0.11] [-0.07 0.18 -0.09 -0.59 0.22 0.2 0.13 0.06 0.3 0.28 1. -0.26 0.1 ] [ 0.28 -0.33 -0.15 0.13 0.04 -0.18 -0.29 -0.08 -0.05 0.18 -0.26 1. 0.01] [ 0.57 0.03 0.26 -0.24 0.34 0.21 0.33 0.67 0.46 -0.11 0.1 0.01 1. ]] current/H5 actual/random mean -0.0771 -0.0008 actual/random fraction > .65 0.032 0.0038 current/H1 actual/random mean 0.1083 -0.0077 actual/random fraction > .65 0.0339 0.0 H5/H1 actual/random mean 0.2086 0.0146 actual/random fraction > .65 0.0452 0.0 rank 40 Channel norm quantiles [[0.484 0.898 1.405] [0.194 0.479 0.737] [0.137 0.39 0.646]] Pair cosine quantiles [[-0.886 -0.708 -0.508] [-0.619 -0.439 -0.174] [-0.07 0.146 0.214] [ 0.426 0.526 0.57 ] [ 0.669 0.673 0.729]] all-three pair cosines > 0.5 count 9 all-three pair cosines > 0.65 count 0 all-three pair cosines > 0.8 count 0 best same-direction neurons (index, 3 cosines, 3 norms): 14 [0.676, 0.632, 0.734] [0.464, 0.41, 0.271] 251 [0.614, 0.607, 0.593] [0.521, 0.397, 0.262] 795 [0.631, 0.587, 0.615] [0.559, 0.509, 0.415] 1265 [0.599, 0.595, 0.56] [0.626, 0.449, 0.407] 799 [0.56, 0.541, 0.642] [0.525, 0.541, 0.434] 1065 [0.63, 0.537, 0.621] [0.542, 0.372, 0.351] 129 [0.524, 0.587, 0.592] [0.646, 0.501, 0.401] 310 [0.524, 0.537, 0.664] [0.562, 0.597, 0.281] 1206 [0.522, 0.716, 0.548] [0.593, 0.449, 0.364] 753 [0.624, 0.494, 0.603] [0.425, 0.421, 0.367] 325 [0.478, 0.625, 0.601] [0.696, 0.332, 0.344] 570 [0.59, 0.496, 0.475] [0.516, 0.476, 0.24] 311 [0.509, 0.651, 0.471] [0.538, 0.451, 0.456] 85 [0.464, 0.525, 0.522] [0.338, 0.36, 0.221] 581 [0.646, 0.463, 0.764] [0.484, 0.561, 0.351] ```