Grounding DINO t
Fit diagram
Read at 100%
Clear selection
Download SVG
Download PNG
Click a block to read its description, or select it with Tab and Enter.
Grounding DINO t
Open-vocabulary detection, 800 × 800 square image, 16 text tokens, 900 object queries. Tensor sizes exclude batch.
LibreYOLO
Grounding DINO t
Open-vocabulary detection, 800 × 800 square image, 16 text tokens, 900 object queries. Tensor sizes exclude batch.
Image backbone and feature projection
Image
3 × 800 × 800
Swin patch embedding
4×4 /4 Conv, channel LayerNorm
Swin stage 1
200 × 200 × 96, n=2
Swin stage 2
100 × 100 × 192, n=2
Swin stage 3
50 × 50 × 384, n=6
Swin stage 4
25 × 25 × 768, n=2
Three 1×1 projections + GroupNorm
192/384/768 to 256; groups=32
Fourth level from raw stage 4
Conv3×3 /2, 768 to256; GroupNorm32
Flatten and concat image levels
13,294 × 256
Sine position + learned level embedding
13,294 × 256
Feature sizes: 100×100, 50×50, 25×25, 13×13.
BERT text backbone
Input IDs + token types + phrase position IDs
16 tokens; vocabulary 30,522
Sum word, position and token-type embeddings
16 × 768
LayerNorm
16 × 768
BERT encoder blocks, repeats=12
768 width, 12 heads, MLP 3,072
Linear text projection
768 to 256
Native phrase-mask values are added as 0/1 bias
inside this BERT path, preserving library behavior.
Padding and token masks are carried separately.
Multimodal encoder and decoder
Image and projected text states
13,294 × 256 and 16 × 256
Encoder layer, repeats=6
Bidirectional fusion; parallel text enhancement and deformable vision layer
Linear + LayerNorm proposal features
13,294 × 256
Contrastive token scoring
13,294 × 16, padded to256 token slots
Top 900 by max valid-token score
900 proposal indices
Proposal box MLP + grid box logits
13,294 × 4
Gather selected boxes, sigmoid
900 × 4 initial reference boxes
Learned query content
900 × 256
Decoder layer, repeats=6
Query self-attention, text cross-attention, deformable image attention, MLP
Final-layer contrastive logits and refined boxes
900 × 256 token logits; 900 × 4 normalized cxcywh
Box deltas refine inverse-sigmoid references at every decoder layer.
Postprocess applies sigmoid, threshold and phrase-to-class alignment.
No NMS in the learned detector.
Bidirectional vision-text fusion
LayerNorm vision
13,294 × 256
LayerNorm text
16 × 256
Linear vision values VV
256 to 1,024
VV: 4 × 13,294 × 256
Linear text values TV
256 to 1,024
TV: 4 × 16 × 256
Linear vision Q
256 to 1,024; 4 heads × 256; scale 1/16
Linear text K
256 to 1,024; 4 heads × 256
MatMul vision Q × text K-transpose
4 × 13,294 × 16
Subtract max and clamp
Clamp [-50,000,50,000]
Mask text padding, softmax over text
4 × 13,294 × 16
Transpose, stabilize, mask vision, softmax
4 × 16 × 13,294
MatMul vision probabilities × TV
TV is the text-value projection shown above
MatMul text probabilities × VV
VV is the vision-value projection shown above
Linear output
1,024 to 256
Multiply learned channel scale
256 weights, initialized 1e-4
+
Residual starts at the normalized vision input.
Linear output
1,024 to 256
Multiply learned channel scale
256 weights, initialized 1e-4
+
Residual starts at the normalized text input.
Text enhancer
Input 16 × 256
Self-attention
4 heads, 256 channels
+
LayerNorm
16 × 256
Linear
256 to 1024
ReLU
16 × 1024
Linear
1024 to 256
+
LayerNorm
16 × 256
Deformable vision layer
Input 13294 × 256
Multi-scale deformable attention
8 heads, 256 channels
+
LayerNorm
13294 × 256
Linear
256 to 2048
ReLU
13294 × 2048
Linear
2048 to 256
+
LayerNorm
13294 × 256
Decoder layer
Query self-attention
Q/K/V=900
+
LayerNorm
900 × 256
Text cross-attention
Q=900, K/V=16
+
LayerNorm
900 × 256
Deformable image attention
Q=900, four image levels
+
LayerNorm
900 × 256
Linear
256 to 2,048
ReLU
900 × 2,048
Linear
2,048 to 256
+
LayerNorm
900 × 256
Query self-attention
Query input
900 × 256
Key/value input
900 × 256
Linear Q
256 to 256
Reshape heads
8 × 900 × 32
Linear K
256 to 256
Reshape heads
8 × 900 × 32
Linear V
256 to 256
Reshape heads
8 × 900 × 32
MatMul Q K-transpose
8 × 900 × 900
Scale
Divide by sqrt(32)
Softmax over keys
8 × 900 × 900
MatMul attention × V
8 × 900 × 32
Merge heads
900 × 256
Linear output
256 to 256
Query-to-text attention
Query input
900 × 256
Key/value input
16 × 256
Linear Q
256 to 256
Reshape heads
8 × 900 × 32
Linear K
256 to 256
Reshape heads
8 × 16 × 32
Linear V
256 to 256
Reshape heads
8 × 16 × 32
MatMul Q K-transpose
8 × 900 × 16
Scale
Divide by sqrt(32)
Softmax over keys
8 × 900 × 16
MatMul attention × V
8 × 900 × 32
Merge heads
900 × 256
Linear output
256 to 256
Reference / box MLPs
Normalized query state
900 × 256
Linear
256 to256
ReLU
256
Linear
256 to256
ReLU
256
Linear
256 to4
Add inverse-sigmoid reference
900 ×4
Sigmoid refined box
900 ×4
Proposal MLP uses the same 256/256/4 dimensions.
Position MLP: sine-encoded box512,
Linear512 to256, ReLU, Linear256 to256.
Multi-scale deformable attention
Query + positional embedding
Encoder 13,294 / decoder 900 queries × 256
Value features
13,294 × 256 from four image levels
Reference coordinates
Encoder: normalized XY; decoder: normalized XYWH
Linear sampling offsets
256 to 256; 8 heads × 4 levels × 4 points × 2
Linear attention weights
256 to 128
Softmax levels and points
8 heads × 16 probabilities per query
Linear value projection
256 to 256; reshape 13,294 × 8 × 32
Compute sampling positions
XY reference + normalized offsets; decoder offsets scaled by box WH
Split values by four levels
100×100, 50×50, 25×25 and 13×13, each 256 channels
Bilinear grid_sample
100 × 100 map; 4 samples per query/head
Values: level 1; coordinates: level 1
Multiply sample values × attention weights
4 × 32 values per query/head
Weights: level 1, four of the 16 probabilities
Bilinear grid_sample
50 × 50 map; 4 samples per query/head
Values: level 2; coordinates: level 2
Multiply sample values × attention weights
4 × 32 values per query/head
Weights: level 2, four of the 16 probabilities
Bilinear grid_sample
25 × 25 map; 4 samples per query/head
Values: level 3; coordinates: level 3
Multiply sample values × attention weights
4 × 32 values per query/head
Weights: level 3, four of the 16 probabilities
Bilinear grid_sample
13 × 13 map; 4 samples per query/head
Values: level 4; coordinates: level 4
Multiply sample values × attention weights
4 × 32 values per query/head
Weights: level 4, four of the 16 probabilities
Sum all four levels and four sample points
8 heads × 32 channels; merge to 256
Linear output projection
256 to 256
BERT encoder block
Input 16 × 768
Self-attention
12 heads, 768 channels
+
LayerNorm
16 × 768
Linear
768 to 3072
GELU
16 × 3072
Linear
3072 to 768
+
LayerNorm
16 × 768
BERT self-attention
Query input
16 × 768
Key/value input
16 × 768
Linear Q
768 to 768
Reshape heads
12 × 16 × 64
Linear K
768 to 768
Reshape heads
12 × 16 × 64
Linear V
768 to 768
Reshape heads
12 × 16 × 64
MatMul Q K-transpose
12 × 16 × 16
Scale
Divide by sqrt(64)
Phrase-mask bias
12 × 16 × 16
+
Softmax over keys
12 × 16 × 16
MatMul attention × V
12 × 16 × 64
Merge heads
16 × 768
Linear output
768 to 768
Swin block and patch merging
LayerNorm
Stage channels 96/192/384/768
Pad to window multiple
Window 7; padding precedes shift
Cyclic shift on even blocks
Shift -3 in H and W
Window self-attention
49 tokens; heads 3/6/12/24
Reverse window partition
Restore padded spatial grid
Reverse cyclic shift
Shift +3 in H and W
Crop padding
Restore stage H × W
+
LayerNorm
Stage channels 96/192/384/768
Linear
Outputs 384/768/1536/3072
GELU
MLP channels 384/768/1536/3072
Linear
Outputs 96/192/384/768
+
Input for next Swin stage
2×2 neighboring patches
Reshape and concatenate neighborhood
Four times current channel width
LayerNorm
Concatenated neighborhood channels
Linear reduction (no bias)
Neighborhood channels to next stage width
Resolved merge widths:
96 input; 384 concatenated; 192 output
192 input; 768 concatenated; 384 output
384 input; 1536 concatenated; 768 output
Output grids: 100×100, 50×50, 25×25.
Swin window self-attention
Query input
49 × 96/192/384/768
Key/value input
49 × 96/192/384/768
Linear Q
96/192/384/768 to 96/192/384/768
Reshape heads
3/6/12/24 × 49 × 32
Linear K
96/192/384/768 to 96/192/384/768
Reshape heads
3/6/12/24 × 49 × 32
Linear V
96/192/384/768 to 96/192/384/768
Reshape heads
3/6/12/24 × 49 × 32
MatMul Q K-transpose
3/6/12/24 × 49 × 49
Scale
Divide by sqrt(32)
Position bias
3/6/12/24 × 49 × 49
+
Softmax over keys
3/6/12/24 × 49 × 49
MatMul attention × V
3/6/12/24 × 49 × 32
Merge heads
49 × 96/192/384/768
Linear output
96/192/384/768 to 96/192/384/768
Text-enhancer self-attention
Query input
16 × 256
Key/value input
16 × 256
Linear Q
256 to 256
Reshape heads
4 × 16 × 64
Linear K
256 to 256
Reshape heads
4 × 16 × 64
Linear V
256 to 256
Reshape heads
4 × 16 × 64
MatMul Q K-transpose
4 × 16 × 16
Scale
Divide by sqrt(64)
Position bias
4 × 16 × 16
+
Softmax over keys
4 × 16 × 16
MatMul attention × V
4 × 16 × 64
Merge heads
16 × 256
Linear output
256 to 256
Variant values
t
Swin channels: 96, 192, 384, 768
Stage repeats: 2, 2, 6, 2; window 7
Heads: 3, 6, 12, 24
b
Swin channels: 128, 256, 512, 1024
Stage repeats: 2, 2, 18, 2; window 12
Heads: 4, 8, 16, 32
Both sizes: 256 model width, 6 encoder layers, 6 decoder layers, 900 queries.
Deformable attention: 8 heads, 4 levels, 4 points per level.
Vision/text fusion: 1,024 internal width, 4 heads; text enhancer: 4 heads, MLP1,024.
Backbone input here is square 800px. Other aspect ratios change spatial token counts.
Source: libreyolo/models/grounding_dino/nn.py. Revision a4d0ecc9e17f.
libreyolo.com