MODEL ATLAS / 01
Architectures I’m tracking.
A small library of models I’m curious about, opened up mechanism by mechanism—from the whole pipeline to the head, filter, state, and expert routes inside.
THE LIBRARY
Pick a model to open.
This is a watchlist, not a ranking. Each guide uses one visual grammar while preserving the architecture’s actual counts and relationships.
01 / GENOMICS
Evo 2 7B
A 32-layer genomic model that alternates three Hyena filter families with periodic global attention.
- Family
- StripedHyena 2
- Scale
- 7B dense
- Context
- Up to 1M bases
02 / OPEN REASONING
gpt-oss 20B
A sparse decoder where 64 query heads share eight key/value pairs and four experts work per token.
- Depth
- 24 layers
- Scale
- 20.91B / 3.61B active
- Context
- 131,072 tokens
03 / EFFICIENT SCALE
DeepSeek V3
A 671B model that compresses its attention cache and activates a fine-grained expert subset.
- Depth
- 61 layers
- Scale
- 671B / 37B active
- Cache idea
- 512 content + 64 position
04 / HYBRID SEQUENCES
Qwen3-Next 80B-A3B
A 3:1 hybrid of recurrent linear attention and periodic full attention with ultra-sparse experts.
- Depth
- 48 layers
- Scale
- 80B / ~3B active
- Context
- 262,144 native