OUTPUT of run 6 of at most 100: ``` current dominant-axis counts (>1.5 SD): [28, 20, 16, 11, 20, 20, 17, 18, 21, 22, 18, 18, 39, 39, 13, 13, 25, 8, 24, 19, 11, 6, 28, 16, 12, 17, 13, 17] H1 route dominant-axis counts (>1.5 SD): [27, 19, 17, 27, 15, 20, 13, 20, 14, 18, 20, 18, 14, 17, 12, 14, 20, 12, 14, 19, 14, 9, 19, 9, 17, 11, 14, 17] H5 route dominant-axis counts (>1.5 SD): [23, 16, 16, 20, 24, 19, 17, 26, 23, 26, 22, 16, 13, 22, 15, 10, 23, 10, 21, 17, 14, 9, 23, 12, 8, 11, 12, 22] Weight-row co-specialization: current axis versus H5-route axis Rows count neurons with both dominant coefficients >1.5 SD; positive coefficient fraction is also shown. Selected neurons 483 positive current/route fractions 0.828 0.609 0 rowtotal 26 [(0, 11), (16, 3), (13, 3), (9, 2), (23, 2), (21, 1)] 1 rowtotal 16 [(22, 4), (1, 3), (16, 2), (0, 1), (25, 1), (4, 1)] 2 rowtotal 15 [(2, 9), (11, 1), (4, 1), (21, 1), (16, 1), (15, 1)] 3 rowtotal 8 [(3, 5), (25, 1), (7, 1), (8, 1)] 4 rowtotal 18 [(4, 6), (27, 2), (26, 2), (5, 2), (0, 2), (22, 1)] 5 rowtotal 18 [(5, 4), (22, 2), (16, 2), (27, 1), (10, 1), (3, 1)] 6 rowtotal 17 [(6, 8), (4, 2), (22, 1), (5, 1), (19, 1), (18, 1)] 7 rowtotal 16 [(7, 7), (22, 2), (27, 1), (2, 1), (3, 1), (20, 1)] 8 rowtotal 20 [(8, 4), (11, 3), (5, 3), (27, 1), (1, 1), (3, 1)] 9 rowtotal 20 [(9, 8), (16, 3), (27, 1), (26, 1), (14, 1), (18, 1)] 10 rowtotal 14 [(10, 5), (8, 2), (9, 2), (1, 1), (19, 1), (7, 1)] 11 rowtotal 18 [(11, 8), (22, 2), (8, 2), (27, 1), (3, 1), (21, 1)] 12 rowtotal 39 [(13, 4), (19, 4), (0, 4), (26, 3), (4, 3), (5, 3)] 13 rowtotal 33 [(13, 11), (7, 3), (0, 3), (12, 2), (1, 2), (4, 2)] 14 rowtotal 13 [(14, 5), (10, 3), (12, 1), (1, 1), (3, 1), (19, 1)] 15 rowtotal 12 [(15, 5), (8, 2), (9, 2), (25, 1), (19, 1), (12, 1)] 16 rowtotal 24 [(16, 4), (27, 3), (5, 3), (12, 2), (14, 2), (17, 1)] 17 rowtotal 8 [(20, 3), (19, 2), (12, 1), (1, 1), (10, 1)] 18 rowtotal 20 [(18, 6), (10, 3), (27, 2), (2, 1), (4, 1), (20, 1)] 19 rowtotal 17 [(4, 3), (19, 3), (1, 2), (18, 2), (27, 1), (24, 1)] 20 rowtotal 10 [(20, 6), (13, 1), (12, 1), (2, 1), (16, 1)] 21 rowtotal 6 [(21, 3), (18, 1), (17, 1), (9, 1)] 22 rowtotal 24 [(22, 7), (18, 3), (1, 2), (3, 2), (6, 2), (7, 2)] 23 rowtotal 15 [(23, 6), (8, 2), (11, 1), (21, 1), (18, 1), (6, 1)] 24 rowtotal 12 [(24, 2), (7, 2), (27, 1), (18, 1), (2, 1), (5, 1)] 25 rowtotal 15 [(25, 5), (17, 2), (27, 1), (18, 1), (7, 1), (15, 1)] 26 rowtotal 12 [(26, 4), (0, 1), (20, 1), (16, 1), (18, 1), (3, 1)] 27 rowtotal 17 [(27, 6), (4, 3), (3, 2), (10, 2), (7, 1), (16, 1)] L0H4 effective OV singular values: [13.878 9.65 2.507 0.969 0.595 0.554 0.483 0.433 0.345 0.295 0.28 0.228] Rank 2 L0H4 output subspace: later QK query/key energy fractions layer 1 [(0, np.float64(0.041), np.float64(0.308)), (1, np.float64(0.067), np.float64(0.569)), (2, np.float64(0.065), np.float64(0.52)), (3, np.float64(0.093), np.float64(0.581)), (4, np.float64(0.076), np.float64(0.545)), (5, np.float64(0.05), np.float64(0.479))] layer 2 [(0, np.float64(0.006), np.float64(0.048)), (1, np.float64(0.002), np.float64(0.003)), (2, np.float64(0.013), np.float64(0.064)), (3, np.float64(0.016), np.float64(0.088)), (4, np.float64(0.016), np.float64(0.09)), (5, np.float64(0.004), np.float64(0.039))] layer 3 [(0, np.float64(0.001), np.float64(0.015)), (1, np.float64(0.001), np.float64(0.007)), (2, np.float64(0.002), np.float64(0.023)), (3, np.float64(0.003), np.float64(0.007)), (4, np.float64(0.002), np.float64(0.024)), (5, np.float64(0.001), np.float64(0.003))] Rank 3 L0H4 output subspace: later QK query/key energy fractions layer 1 [(0, np.float64(0.061), np.float64(0.342)), (1, np.float64(0.107), np.float64(0.627)), (2, np.float64(0.108), np.float64(0.631)), (3, np.float64(0.137), np.float64(0.677)), (4, np.float64(0.119), np.float64(0.645)), (5, np.float64(0.084), np.float64(0.524))] layer 2 [(0, np.float64(0.01), np.float64(0.051)), (1, np.float64(0.005), np.float64(0.005)), (2, np.float64(0.014), np.float64(0.07)), (3, np.float64(0.022), np.float64(0.095)), (4, np.float64(0.023), np.float64(0.095)), (5, np.float64(0.006), np.float64(0.041))] layer 3 [(0, np.float64(0.002), np.float64(0.017)), (1, np.float64(0.003), np.float64(0.009)), (2, np.float64(0.003), np.float64(0.025)), (3, np.float64(0.005), np.float64(0.01)), (4, np.float64(0.004), np.float64(0.025)), (5, np.float64(0.003), np.float64(0.005))] Rank 6 L0H4 output subspace: later QK query/key energy fractions layer 1 [(0, np.float64(0.1), np.float64(0.353)), (1, np.float64(0.161), np.float64(0.636)), (2, np.float64(0.175), np.float64(0.642)), (3, np.float64(0.209), np.float64(0.684)), (4, np.float64(0.186), np.float64(0.654)), (5, np.float64(0.135), np.float64(0.534))] layer 2 [(0, np.float64(0.02), np.float64(0.06)), (1, np.float64(0.019), np.float64(0.013)), (2, np.float64(0.023), np.float64(0.078)), (3, np.float64(0.036), np.float64(0.102)), (4, np.float64(0.032), np.float64(0.105)), (5, np.float64(0.024), np.float64(0.051))] layer 3 [(0, np.float64(0.012), np.float64(0.026)), (1, np.float64(0.013), np.float64(0.018)), (2, np.float64(0.013), np.float64(0.032)), (3, np.float64(0.018), np.float64(0.018)), (4, np.float64(0.017), np.float64(0.034)), (5, np.float64(0.012), np.float64(0.015))] ```