CONFIG A
{
  "_sliding_window_pattern": 6,
  "architectures": [
    "Gemma3ForCausalLM"
  ],
  "attention_bias": false,
  "attention_dropout": 0.0,
  "attn_logit_softcapping": null,
  "bos_token_id": 2,
  "dtype": "float32",
  "eos_token_id": 1,
  "final_logit_softcapping": null,
  "head_dim": 256,
  "hidden_activation": "gelu_pytorch_tanh",
  "hidden_size": 640,
  "initializer_range": 0.02,
  "intermediate_size": 2048,
  "layer_types": [
    "sliding_attention",
    "sliding_attention",
    "sliding_attention",
    "sliding_attention",
    "sliding_attention",
    "full_attention",
    "sliding_attention",
    "sliding_attention",
    "sliding_attention",
    "sliding_attention",
    "sliding_attention",
    "full_attention",
    "sliding_attention",
    "sliding_attention",
    "sliding_attention",
    "sliding_attention",
    "sliding_attention",
    "full_attention"
  ],
  "max_position_embeddings": 32768,
  "model_type": "gemma3_text",
  "num_attention_heads": 4,
  "num_hidden_layers": 18,
  "num_key_value_heads": 1,
  "pad_token_id": 0,
  "query_pre_attn_scalar": 256,
  "rms_norm_eps": 1e-06,
  "rope_parameters": {
    "full_attention": {
      "rope_theta": 1000000.0,
      "rope_type": "default"
    },
    "sliding_attention": {
      "rope_theta": 10000.0,
      "rope_type": "default"
    }
  },
  "sliding_window": 512,
  "tie_word_embeddings": true,
  "transformers_version": "5.20.2",
  "use_bidirectional_attention": false,
  "use_cache": true,
  "vocab_size": 262144
}

CONFIG DIFFERENCES

KEY DIFFERENCES
Only A: []
Only B: []
Shape mismatches: []

LAYERS: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]

FIRST-LAYER TENSOR SHAPES
model.layers.0.input_layernorm.weight (640,)
model.layers.0.mlp.down_proj.weight (640, 2048)
model.layers.0.mlp.gate_proj.weight (2048, 640)
model.layers.0.mlp.up_proj.weight (2048, 640)
model.layers.0.post_attention_layernorm.weight (640,)
model.layers.0.post_feedforward_layernorm.weight (640,)
model.layers.0.pre_feedforward_layernorm.weight (640,)
model.layers.0.self_attn.k_norm.weight (256,)
model.layers.0.self_attn.k_proj.weight (256, 640)
model.layers.0.self_attn.o_proj.weight (640, 1024)
model.layers.0.self_attn.q_norm.weight (256,)
model.layers.0.self_attn.q_proj.weight (1024, 640)
model.layers.0.self_attn.v_proj.weight (256, 640)

TENSOR DIFFERENCES
Relative difference = ||B-A|| / RMS(||A||, ||B||).
These are structural differences, NOT importance scores.
Elements: 100326016
Exactly changed: 79541882 fraction: 0.7928340541300872

Largest 20 tensor-relative differences:
model.layers.4.mlp.down_proj.weight                                    rel=0.0188927 abs=0.263012 changed=91.9057%
model.layers.14.mlp.down_proj.weight                                   rel=0.018741 abs=0.288605 changed=90.6656%
model.layers.8.mlp.down_proj.weight                                    rel=0.0171211 abs=0.254042 changed=89.8425%
model.layers.1.mlp.down_proj.weight                                    rel=0.0170993 abs=0.266182 changed=89.8982%
model.layers.13.mlp.down_proj.weight                                   rel=0.0168742 abs=0.27971 changed=89.6366%
model.layers.10.mlp.down_proj.weight                                   rel=0.0165194 abs=0.257345 changed=89.8518%
model.layers.3.mlp.down_proj.weight                                    rel=0.0164501 abs=0.261297 changed=89.9266%
model.layers.15.mlp.down_proj.weight                                   rel=0.015908 abs=0.291926 changed=88.9783%
model.layers.6.mlp.down_proj.weight                                    rel=0.0149442 abs=0.255666 changed=88.5247%
model.layers.17.mlp.down_proj.weight                                   rel=0.0144757 abs=0.264389 changed=88.1110%
model.layers.16.mlp.down_proj.weight                                   rel=0.0143463 abs=0.28325 changed=87.8399%
model.layers.2.mlp.down_proj.weight                                    rel=0.0140619 abs=0.260818 changed=87.9953%
model.layers.7.mlp.down_proj.weight                                    rel=0.0133444 abs=0.255363 changed=87.1011%
model.layers.11.mlp.down_proj.weight                                   rel=0.0121551 abs=0.249632 changed=85.8959%
model.layers.9.mlp.down_proj.weight                                    rel=0.0121532 abs=0.256792 changed=85.8909%
model.layers.12.mlp.down_proj.weight                                   rel=0.0119406 abs=0.27038 changed=85.8234%
model.layers.3.self_attn.o_proj.weight                                 rel=0.0110028 abs=0.176902 changed=85.0229%
model.layers.16.self_attn.o_proj.weight                                rel=0.0109819 abs=0.191689 changed=84.5650%
model.layers.0.mlp.down_proj.weight                                    rel=0.0109329 abs=0.269196 changed=84.5435%
model.layers.4.self_attn.o_proj.weight                                 rel=0.0109289 abs=0.187839 changed=86.2372%

LAYER SUMMARY
L00: rel=0.00134596 abs=0.564606 changed=76.7567%
L01: rel=0.000535428 abs=0.56879 changed=79.0118%
L02: rel=0.000526498 abs=0.570384 changed=77.9557%
L03: rel=0.000390541 abs=0.562423 changed=79.6750%
L04: rel=0.000376724 abs=0.577649 changed=80.7116%
L05: rel=0.000459421 abs=0.577494 changed=78.7517%
L06: rel=0.000464769 abs=0.576992 changed=79.4334%
L07: rel=0.000476707 abs=0.572171 changed=78.7593%
L08: rel=0.000410987 abs=0.56923 changed=79.7598%
L09: rel=0.00039518 abs=0.577223 changed=78.6360%
L10: rel=0.000236804 abs=0.580494 changed=79.3020%
L11: rel=9.63843e-05 abs=0.575973 changed=78.8841%
L12: rel=8.12529e-05 abs=0.595671 changed=79.4979%
L13: rel=8.81163e-05 abs=0.60546 changed=80.3784%
L14: rel=7.71962e-05 abs=0.607012 changed=80.1797%
L15: rel=6.42637e-05 abs=0.622983 changed=79.8060%
L16: rel=6.31366e-05 abs=0.604505 changed=79.9507%
L17: rel=5.63721e-05 abs=0.581168 changed=79.6606%

BARE ATTENTION PRODUCTS
Assumes standard contiguous GQA head layout.
QK excludes Q/K normalization and positional transformations.
OV excludes attention routing and block normalization.
Identity alignment does NOT identify copying the user's guess.

Largest 12 by ov_rel (not causal importance):
L03 H03 KV00 OV_rel=0.0187565 OV_abs=0.101656 QK_rel=0.00866287 QK_abs=0.165531 OV_identity_A=-0.0554177 OV_identity_B=-0.0553868
L04 H01 KV00 OV_rel=0.0174709 OV_abs=0.113219 QK_rel=0.0143496 QK_abs=0.180153 OV_identity_A=0.0766251 OV_identity_B=0.0766296
L16 H03 KV00 OV_rel=0.0169859 OV_abs=0.127067 QK_rel=0.0103221 QK_abs=0.172524 OV_identity_A=0.0142888 OV_identity_B=0.0142662
L16 H01 KV00 OV_rel=0.0168211 OV_abs=0.130153 QK_rel=0.0132528 QK_abs=0.170178 OV_identity_A=0.13185 OV_identity_B=0.131819
L04 H00 KV00 OV_rel=0.0159686 OV_abs=0.113256 QK_rel=0.0123794 QK_abs=0.173652 OV_identity_A=0.0425157 OV_identity_B=0.0424891
L16 H00 KV00 OV_rel=0.0151427 OV_abs=0.131747 QK_rel=0.0121581 QK_abs=0.171943 OV_identity_A=0.0366155 OV_identity_B=0.0365897
L04 H02 KV00 OV_rel=0.0142119 OV_abs=0.1176 QK_rel=0.0120409 QK_abs=0.169354 OV_identity_A=-0.00528449 OV_identity_B=-0.00525959
L08 H02 KV00 OV_rel=0.0139522 OV_abs=0.111562 QK_rel=0.0147265 QK_abs=0.173054 OV_identity_A=0.169079 OV_identity_B=0.169064
L15 H02 KV00 OV_rel=0.0139037 OV_abs=0.153682 QK_rel=0.0117017 QK_abs=0.193404 OV_identity_A=0.286934 OV_identity_B=0.286942
L07 H03 KV00 OV_rel=0.0137639 OV_abs=0.110013 QK_rel=0.00825571 QK_abs=0.169222 OV_identity_A=0.209415 OV_identity_B=0.209431
L05 H03 KV00 OV_rel=0.0137571 OV_abs=0.143897 QK_rel=0.010785 QK_abs=0.164475 OV_identity_A=-0.101361 OV_identity_B=-0.101359
L03 H01 KV00 OV_rel=0.0135825 OV_abs=0.103181 QK_rel=0.0130651 QK_abs=0.154347 OV_identity_A=0.0661186 OV_identity_B=0.0661419

Largest 12 by qk_rel (not causal importance):
L08 H02 KV00 OV_rel=0.0139522 OV_abs=0.111562 QK_rel=0.0147265 QK_abs=0.173054 OV_identity_A=0.169079 OV_identity_B=0.169064
L04 H01 KV00 OV_rel=0.0174709 OV_abs=0.113219 QK_rel=0.0143496 QK_abs=0.180153 OV_identity_A=0.0766251 OV_identity_B=0.0766296
L11 H01 KV00 OV_rel=0.0128887 OV_abs=0.140722 QK_rel=0.0143267 QK_abs=0.176724 OV_identity_A=0.40658 OV_identity_B=0.406591
L11 H00 KV00 OV_rel=0.0115653 OV_abs=0.13827 QK_rel=0.0135827 QK_abs=0.177395 OV_identity_A=0.0474959 OV_identity_B=0.0475246
L16 H01 KV00 OV_rel=0.0168211 OV_abs=0.130153 QK_rel=0.0132528 QK_abs=0.170178 OV_identity_A=0.13185 OV_identity_B=0.131819
L03 H01 KV00 OV_rel=0.0135825 OV_abs=0.103181 QK_rel=0.0130651 QK_abs=0.154347 OV_identity_A=0.0661186 OV_identity_B=0.0661419
L04 H03 KV00 OV_rel=0.0104633 OV_abs=0.130094 QK_rel=0.0127386 QK_abs=0.166638 OV_identity_A=-0.0234079 OV_identity_B=-0.0234252
L16 H02 KV00 OV_rel=0.0126484 OV_abs=0.130167 QK_rel=0.0127168 QK_abs=0.17125 OV_identity_A=0.227631 OV_identity_B=0.227612
L15 H01 KV00 OV_rel=0.0119272 OV_abs=0.190321 QK_rel=0.012488 QK_abs=0.184915 OV_identity_A=-0.457098 OV_identity_B=-0.45707
L13 H01 KV00 OV_rel=0.0131245 OV_abs=0.134766 QK_rel=0.0124058 QK_abs=0.171004 OV_identity_A=0.0973341 OV_identity_B=0.0973497
L04 H00 KV00 OV_rel=0.0159686 OV_abs=0.113256 QK_rel=0.0123794 QK_abs=0.173652 OV_identity_A=0.0425157 OV_identity_B=0.0424891
L16 H00 KV00 OV_rel=0.0151427 OV_abs=0.131747 QK_rel=0.0121581 QK_abs=0.171943 OV_identity_A=0.0366155 OV_identity_B=0.0365897

Interpretation boundary:
The output identifies parameter and composed-map differences.
It does not establish which checkpoint is sycophantic,
which heads attend to the user's guess, or causal importance.
