FIELD GUIDE 01 / GENOMICS
Evo 2 7B
A dense genomic foundation model that stretches to million-base context by interleaving three Hyena filter families with periodic global attention.
- Family
- StripedHyena 2
- Scale
- 7B dense parameters
- Shape
- 32 layers · width 4,096
- Context
- Up to 1,048,576 bases
THE 60-SECOND READ
Follow one DNA base.
- 01
A nucleotide becomes a learned vector in a 4,096-wide residual stream.
- 02
Short explicit (Hyena-SE), medium regularized (Hyena-MR), and long implicit (Hyena-LI) operators mix sequence information without full attention at every layer.
- 03
Five multi-head attention (MHA) layers provide periodic global bridges through the 32-layer stack.
- 04
The final vector becomes logits—unnormalized scores—for the next DNA token.
OUTSIDE IN
Whole model
The released 7B checkpoint is a 32-layer, width-4,096 autoregressive model. It was trained on 2.4 trillion tokens; the more-than-8.8-trillion-nucleotide figure describes the broader OpenGenome2 dataset, not the amount consumed by this checkpoint.
MECHANISM CUTAWAYS
Open the blocks
01DNA token pathTurn bases into a next-base prediction.
Plain language. Evo 2 reads DNA the way an autoregressive language model reads text: one discrete token at a time, always predicting what follows.
Technical read. A learned embedding places tokens in the width-4,096 residual stream. After 32 sequence- and channel-mixing layers, an output projection produces logits over the tokenizer vocabulary.
Why it matters. The same objective can expose biological regularities because the probability of a base depends on sequence context across motifs, genes, and longer genomic regions.
02Exact 32-layer ribbonSee every mixer in the 1M checkpoint.
Plain language. The model cycles through short, medium, and long filters, then periodically inserts global attention.
Technical read. Zero-based indices 0–31 contain 9 Hyena-SE, 9 Hyena-MR, 9 Hyena-LI, and 5 MHA layers. Attention is sparse in depth—only five layers use it—but each MHA layer itself uses a global 32-head attention map with interpolated Rotary Position Embedding (RoPE).
Why it matters. Most layers keep sequence mixing efficient, while global MHA layers create direct long-range communication checkpoints.
03Hyena operator anatomyProject, mix, filter, gate, return.
Plain language. A Hyena operator makes three views of the input, uses one to control another through a convolutional filter, then gates the result.
Technical read. The conceptual signal flow projects query-like (q), key-like (k), and value-like (v) streams, applies a width-3 short depthwise causal convolution, multiplies k and v, applies the operator-specific inner convolution, gates with q, and output-projects back to the residual stream.
Why it matters. The skeleton stays fixed while the inner filter cartridge changes the range and computational form of sequence mixing.
CONCEPTUAL SIGNAL FLOW
SE / MR / LI
04Replaceable filter cartridgeThree exact filter families, one operator skeleton.
Plain language. The short filter catches local motifs, the medium filter covers a wider explicit window, and the long filter represents very long dependencies compactly.
Technical read. Hyena-SE uses an explicit finite impulse response (FIR) filter of length 7 with 256 groups. Hyena-MR uses length 128 with 256 groups plus an exponential-decay envelope. Hyena-LI uses an implicit exponential/pole-residue filter with 4,096 groups and state size 16.
Why it matters. Hyena-LI can switch from convolutional processing to a compact recurrent form during autoregressive decoding, while SE and MR supply efficient explicit local structure.
- Length
- 7
- Groups
- 256
- Form
- short explicit
- Length
- 128
- Groups
- 256
- Envelope
- exponential decay
- Groups
- 4,096
- State
- 16
- Decode
- recurrent form
05Long-range viewDifferent spans, plus five global bridges.
Plain language. Several kinds of memory look across the same DNA: local filters see motifs, wider filters see neighborhoods, long implicit filters carry distant structure, and attention occasionally connects everything directly.
Technical read. The range bands below are conceptual, not literal receptive-field boundaries. The five MHA bridge locations correspond to exact layers 3, 10, 17, 24, and 31.
Why it matters. Range specialization avoids paying global-attention cost in every layer while preserving periodic direct access across the full sequence.
CONCEPTUAL RANGE; EXACT MHA LAYERS
READ THE ORIGINALS
Primary sources
- Evo 2 manuscript Model, data, training, and evaluations.
- StripedHyena 2 architecture paper Operator and filter-family design.
- Official Evo 2 repository Released code and checkpoints.
- Exact Evo 2 7B 1M configuration Layer placement and filter parameters shown here.
The diagrams on this page are original teaching renderings. They simplify layout to make relationships readable; the linked paper, configuration, and implementation remain authoritative for exact computation and checkpoint behavior.