SAM 2 tiny
Fit diagram
Read at 100%
Clear selection
Download SVG
Download PNG
Click a block to read its description, or select it with Tab and Enter.
SAM 2 tiny
Image segmentation, 1024 × 1024 input, one box, three mask outputs. Eight decoder tokens include an object-score token.
LibreYOLO
SAM 2 tiny
Image segmentation, 1024 × 1024 input, one box, three mask outputs. Eight decoder tokens include an object-score token.
Hiera encoder
RGB input after resize/normalize
3 × 1,024 × 1,024
Conv2d 7×7 / 4
256 × 256 × 96, p=3
Add image position
Interpolated background + tiled window term
Stage 1 Hiera blocks
256 × 256 × 96, n=1
Stage 2 Hiera blocks
128 × 128 × 192, n=2
Stage 3 Hiera blocks
64 × 64 × 384, n=7
Stage 4 Hiera blocks
32 × 32 × 768, n=2
Feature pyramid
Lateral 1×1 projections to 256 channels
Main image embedding
256 × 64 × 64, plus no-memory embedding
Stages 2, 3, 4 pool queries with stride 2.
Global blocks (0-based): 5, 7, 9
Other blocks use stage-specific local windows.
Box prompt encoder
One box
2 corner coordinates × 2
Shift pixel centers
Add 0.5 to x and y
Normalize coordinates
Divide by 1024, map to [-1,1]
MatMul random Fourier matrix
2 coords to 128 frequencies
Multiply by 2 pi
2 × 128
Sin
2 × 128
Cos
2 × 128
Concat sin and cos
2 × 256
Corner type
2 learned × 256
+
Sparse prompt embeddings
2 × 256
Learned no-mask embedding
Broadcast to 256 × 64 × 64
Selected input: one box, no mask prompt.
Points use the same Fourier encoding, plus
positive/negative or not-a-point embeddings.
Dense image position: this encoding evaluated
on the centers of the 64 × 64 feature grid.
Mask decoder inputs
Image embedding
256 × 64 × 64
Dense prompt
256 × 64 × 64
+
Flatten image features
4,096 × 256
Learned output tokens
Object + IoU + 4 mask = 6 × 256
Sparse prompts
2 × 256
Concat output and prompt tokens
8 × 256
Two-way block 1
8 queries, 4,096 image tokens
Two-way block 2
8 queries, 4,096 image tokens
Final token-to-image attention
Q+Q0, K+P, V=updated image
+
Final LayerNorm on queries
8 × 256
Q0 is the original token sequence.
P is fixed Fourier image position.
4,096 image tokens feed block 1 keys/values.
Masks and quality outputs
Updated query tokens
1 IoU + 4 mask tokens + 1 object token
Updated image tokens
4,096 × 256
Reshape to image
256 × 64 × 64
ConvTranspose2d 2×2 / 2
256 to 64; 64 × 128 × 128
Channel LayerNorm
64 × 128 × 128
GELU
64 × 128 × 128
ConvTranspose2d 2×2 / 2
64 to 32; 32 × 256 × 256
GELU
32 × 256 × 256
Select four mask tokens
4 × 256
Four independent Linear layers
Each 256 to 256
ReLU
4 × 256
Four independent Linear layers
Each 256 to 256
ReLU
4 × 256
Four independent Linear layers
Each 256 to 32
MatMul mask coefficients × upscaled image
4 × 32 times 32 × 65,536; reshape to 4 × 256 × 256
Select masks 1 to 3
3 × 256 × 256
Mask logits are resized to the input canvas,
then unpadded/resized to the original image.
multimask_output=True is drawn.
False selects mask/score 0 instead.
Object token: Linear 256, ReLU, Linear 256,
ReLU, Linear 1 produces object-score logit.
+
High-res feature s1
64 × 128 × 128
+
High-res feature s0
32 × 256 × 256
Select IoU token 1
256
Linear
256 to 256
ReLU
256
Linear
256 to 256
ReLU
256
Linear
256 to 4
Sigmoid
4 mask-quality scores
Select scores 1 to 3
3 quality scores
Select object token 0
256
Linear
256 to 256
ReLU
256
Linear
256 to 256
ReLU
256
Linear
256 to 1 object-score logit
Two-way block 1
Query state
8 × 256
Image state
4,096 × 256
Token self-attention
8 heads, 32 channels per head
First self-attention has no PE or residual.
LayerNorm
8 × 256
Original tokens Q0
8 × 256
+
Image position P
4,096 × 256
+
Token-to-image attention
Q=8, K/V=4,096; 8 heads × 16
+
LayerNorm
8 × 256
Linear
256 to 2,048
ReLU
8 × 2,048
Linear
2,048 to 256
+
LayerNorm
8 × 256
Original tokens Q0
8 × 256
+
Image-to-token attention
Q=4,096, K/V=8; 8 heads × 16
+
LayerNorm on image state
4,096 × 256
Updated queries: 8 × 256
Two-way block 2
Query state
8 × 256
Image state
4,096 × 256
Token self-attention
8 heads, 32 channels per head
Original tokens Q0
8 × 256
+
Self-attention Q/K include Q0.
+
LayerNorm
8 × 256
Original tokens Q0
8 × 256
+
Image position P
4,096 × 256
+
Token-to-image attention
Q=8, K/V=4,096; 8 heads × 16
+
LayerNorm
8 × 256
Linear
256 to 2,048
ReLU
8 × 2,048
Linear
2,048 to 256
+
LayerNorm
8 × 256
Original tokens Q0
8 × 256
+
Image-to-token attention
Q=4,096, K/V=8; 8 heads × 16
+
LayerNorm on image state
4,096 × 256
Updated queries: 8 × 256
Token self-attention
Query input
8 × 256
Key/value input
8 × 256
Linear Q
256 to 256
Reshape heads
8 × 8 × 32
Linear K
256 to 256
Reshape heads
8 × 8 × 32
Linear V
256 to 256
Reshape heads
8 × 8 × 32
MatMul Q K-transpose
8 × 8 × 8
Scale
Divide by sqrt(32)
Softmax over keys
8 × 8 × 8
MatMul attention × V
8 × 8 × 32
Merge heads
8 × 256
Linear output
256 to 256
Token-to-image attention
Query input
8 × 256
Key/value input
4096 × 256
Linear Q
256 to 128
Reshape heads
8 × 8 × 16
Linear K
256 to 128
Reshape heads
8 × 4096 × 16
Linear V
256 to 128
Reshape heads
8 × 4096 × 16
MatMul Q K-transpose
8 × 8 × 4096
Scale
Divide by sqrt(16)
Softmax over keys
8 × 8 × 4096
MatMul attention × V
8 × 8 × 16
Merge heads
8 × 128
Linear output
128 to 256
Image-to-token attention
Query input
4096 × 256
Key/value input
8 × 256
Linear Q
256 to 128
Reshape heads
8 × 4096 × 16
Linear K
256 to 128
Reshape heads
8 × 8 × 16
Linear V
256 to 128
Reshape heads
8 × 8 × 16
MatMul Q K-transpose
8 × 4096 × 8
Scale
Divide by sqrt(16)
Softmax over keys
8 × 4096 × 8
MatMul attention × V
8 × 4096 × 16
Merge heads
4096 × 128
Linear output
128 to 256
Image-mode decoder
Q0: original eight output/prompt tokens; P: Fourier image position.
Final cross-attention uses the token-to-image equation, with residual and LayerNorm.
Features s0 and s1 are computed once, then added at the two upscaling steps.
The no-memory embedding is added to the 64×64 image feature.
This is image inference. Video memory attention and memory encoder do not run.
multimask_output=True selects mask/IoU slots 1, 2 and 3.
Single-mask eval can fall back to the best multimask using stability thresholds.
Hiera stage 1 block
Output spatial size 256 × 256, channels 96
Stage 1 has no query pooling.
LayerNorm
Entry width 96; remaining blocks 96
Partition local windows
Entry window 8; others 8
Linear QKV
Entry 96 to 288
Reshape Q, K and V
1 heads; 96 channels per head
MaxPool2d on Q only
Identity in this stage
MatMul Q K-transpose
Q pooled at entry; K/V keep input tokens
Scale attention scores
Divide by sqrt(96)
Softmax over keys
Local windows or full spatial token grid
MatMul probabilities × V
96 total output channels
Linear output
96 to 96
Reverse windows and crop
256 × 256 × 96
+
LayerNorm
256 × 256 × 96
Linear
96 to 384
GELU
256 × 256 × 384
Linear
384 to 96
+
Non-entry blocks use the raw-input identity shortcut.
Global blocks bypass window partition/unpartition.
Hiera stage 2 block
Output spatial size 128 × 128, channels 192
Entry block pools 2×2 Q and shortcut.
LayerNorm
Entry width 96; remaining blocks 192
Partition local windows
Entry window 8; others 4
Linear QKV
Entry 96 to 576
Reshape Q, K and V
2 heads; 96 channels per head
MaxPool2d on Q only
2×2 / 2 in entry block
MatMul Q K-transpose
Q pooled at entry; K/V keep input tokens
Scale attention scores
Divide by sqrt(96)
Softmax over keys
Local windows or full spatial token grid
MatMul probabilities × V
192 total output channels
Linear output
192 to 192
Reverse windows and crop
128 × 128 × 192
Linear shortcut
96 to 192
MaxPool2d 2×2 / 2
128 × 128 × 192
+
Choose shortcut
Projected entry / identity rest
LayerNorm
128 × 128 × 192
Linear
192 to 768
GELU
128 × 128 × 768
Linear
768 to 192
+
Non-entry blocks use the raw-input identity shortcut.
Global blocks bypass window partition/unpartition.
Hiera stage 3 block
Output spatial size 64 × 64, channels 384
Entry block pools 2×2 Q and shortcut.
LayerNorm
Entry width 192; remaining blocks 384
Partition local windows
Entry window 4; others 14
Linear QKV
Entry 192 to 1152
Reshape Q, K and V
4 heads; 96 channels per head
MaxPool2d on Q only
2×2 / 2 in entry block
MatMul Q K-transpose
Q pooled at entry; K/V keep input tokens
Scale attention scores
Divide by sqrt(96)
Softmax over keys
Local windows or full spatial token grid
MatMul probabilities × V
384 total output channels
Linear output
384 to 384
Reverse windows and crop
64 × 64 × 384
Linear shortcut
192 to 384
MaxPool2d 2×2 / 2
64 × 64 × 384
+
Choose shortcut
Projected entry / identity rest
LayerNorm
64 × 64 × 384
Linear
384 to 1536
GELU
64 × 64 × 1536
Linear
1536 to 384
+
Non-entry blocks use the raw-input identity shortcut.
Global blocks bypass window partition/unpartition.
Hiera stage 4 block
Output spatial size 32 × 32, channels 768
Entry block pools 2×2 Q and shortcut.
LayerNorm
Entry width 384; remaining blocks 768
Partition local windows
Entry window 14; others 7
Linear QKV
Entry 384 to 2304
Reshape Q, K and V
8 heads; 96 channels per head
MaxPool2d on Q only
2×2 / 2 in entry block
MatMul Q K-transpose
Q pooled at entry; K/V keep input tokens
Scale attention scores
Divide by sqrt(96)
Softmax over keys
Local windows or full spatial token grid
MatMul probabilities × V
768 total output channels
Linear output
768 to 768
Reverse windows and crop
32 × 32 × 768
Linear shortcut
384 to 768
MaxPool2d 2×2 / 2
32 × 32 × 768
+
Choose shortcut
Projected entry / identity rest
LayerNorm
32 × 32 × 768
Linear
768 to 3072
GELU
32 × 32 × 3072
Linear
3072 to 768
+
Non-entry blocks use the raw-input identity shortcut.
Global blocks bypass window partition/unpartition.
Image feature pyramid
Stage 1 map
96 × 256 × 256
Lateral Conv2d 1×1
96 to 256, bias=True
High-res Conv2d 1×1
256 to 32
s0: 32 × 256 × 256
Stage 2 map
192 × 128 × 128
Lateral Conv2d 1×1
192 to 256, bias=True
High-res Conv2d 1×1
256 to 64
s1: 64 × 128 × 128
Stage 3 map
384 × 64 × 64
Lateral Conv2d 1×1
384 to 256, bias=True
+
Add no-memory embedding
256 × 64 × 64
Main image embedding: 256 × 64 × 64
Stage 4 map
768 × 32 × 32
Lateral Conv2d 1×1
768 to 256, bias=True
Nearest upsample ×2
256 × 64 × 64
Only the 32×32 to 64×64 top-down addition runs for fpn_top_down_levels=[2,3].
The 128×128 and 256×256 lateral outputs remain independent. The 32×32 output is not passed to the mask decoder.
Variant values
tiny
C1..C4: 96, 192, 384, 768; n1..n4: 1, 2, 7, 2
h1..h4: 1, 2, 4, 8; w1..w4: 8, 4, 14, 7
Global block indices (0-based): 5, 7, 9
small
C1..C4: 96, 192, 384, 768; n1..n4: 1, 2, 11, 2
h1..h4: 1, 2, 4, 8; w1..w4: 8, 4, 14, 7
Global block indices (0-based): 7, 10, 13
base-plus
C1..C4: 112, 224, 448, 896; n1..n4: 2, 3, 16, 3
h1..h4: 2, 4, 8, 16; w1..w4: 8, 4, 14, 7
Global block indices (0-based): 12, 16, 20
large
C1..C4: 144, 288, 576, 1152; n1..n4: 2, 6, 36, 4
h1..h4: 2, 4, 8, 16; w1..w4: 8, 4, 16, 8
Global block indices (0-based): 23, 33, 43
The decoder and three output feature shapes are fixed across all four sizes.
Source: libreyolo/models/sam/sam2.py. Revision a4d0ecc9e17f.
libreyolo.com