OMDet-Turbo t
Fit diagram
Read at 100%
Clear selection
Download SVG
Download PNG
Click a block to read its description, or select it with Tab and Enter.
OMDet-Turbo t
Open-vocabulary detection, 640 × 640 image, 3 class prompts, 16 valid task tokens. 900 selected object queries.
LibreYOLO
OMDet-Turbo t
Open-vocabulary detection, 640 × 640 image, 3 class prompts, 16 valid task tokens. 900 selected object queries.
Swin-T image backbone
Image
3 × 640 × 640
Conv2d 4×4 / 4
96 × 160 × 160
LayerNorm in NHWC
160 × 160 × 96
Swin stage 1, n=2
160 × 160 × 96, 3 heads
Swin stage 2, n=2
80 × 80 × 192, 6 heads
Swin stage 3, n=6
40 × 40 × 384, 12 heads
Swin stage 4, n=2
20 × 20 × 768, 24 heads
LayerNorm on selected stage maps
Stages 2,3,4; then permute NCHW
Conv2d 1×1 and BatchNorm
192/384/768 to 256 channels
All windows are 7×7. Even blocks shift by 3.
Timm order: shift first, then pad to window multiple.
Shared CLIP text backbone
Class and task token IDs
Three class rows and one task row, length77
Token embedding
77 × 512, vocabulary49,408
Add learned positions
77 × 512
Causal text encoder, repeats=12
512 width, 8 heads, MLP2,048
Final LayerNorm
77 × 512
Class EOT selection
3 × 512
Truncate task to valid tokens
16 × 512
MatMul text projection
512 to512, no bias
Task residual MLP
512 to1,024 to512
Linear task projection
512 to256
Decoder initialization and output
Hybrid encoder output
256-channel maps at80×80,40×40,20×20
Conv2d 1×1 + BatchNorm per level
256 to256, no activation
Flatten and concatenate
8,400 × 256
Linear + LayerNorm proposal features
8,400 × 256
Cosine score against projected class rows
Class Linear512 to256; scores8,400 ×3
Top900 by maximum class score
Gather features and box proposals
Decoder block, repeats=6
900 query states +16 task tokens
Final prediction heads
900 ×3 class logits;900 ×4 boxes
Initial content is selected image proposal features, not learned query embeddings.
Task tokens are updated by joint self-attention and carried to the next layer.
Postprocess rescales boxes and applies class-aware NMS (default IoU0.5).
Top-down FPN
Projected P3
256 × 80 × 80
Projected P4
256 × 40 × 40
Projected P5
256 × 20 × 20
One transformer encoder block
400 tokens ×256
Lateral ConvNormGELU 1×1
256 ×20 ×20
Nearest upsample ×2
256 ×40 ×40
Concat upsample and P4
512 ×40 ×40
CSPRep layer
256 ×40 ×40
Lateral ConvNormGELU 1×1
256 ×40 ×40
Nearest upsample ×2
256 ×80 ×80
Concat upsample and P3
512 ×80 ×80
CSPRep layer
256 ×80 ×80
FPN routes preserve the lateral256-channel features for PAN concatenation.
Bottom-up PAN
Finest FPN output
256 ×80 ×80 (output P3)
ConvNormGELU 3×3 /2
256 ×40 ×40
FPN lateral feature
256 ×40 ×40
Concat downsample and lateral
512 ×40 ×40
CSPRep layer
256 ×40 ×40 (output P4)
ConvNormGELU 3×3 /2
256 ×20 ×20
FPN lateral feature
256 ×20 ×20
Concat downsample and lateral
512 ×20 ×20
CSPRep layer
256 ×20 ×20 (output P5)
P3/P4/P5 output sizes:6,400+1,600+400=8,400 locations.
Highest-level encoder block
Input 400 × 256
Self-attention
8 heads, 256 channels
+
LayerNorm
400 × 256
Linear
256 to 2048
ReLU
400 × 2048
Linear
2048 to 256
+
LayerNorm
400 × 256
Highest-level self-attention
Query input
400 × 256
Key/value input
400 × 256
Linear Q
256 to 256
Reshape heads
8 × 400 × 32
Linear K
256 to 256
Reshape heads
8 × 400 × 32
Linear V
256 to 256
Reshape heads
8 × 400 × 32
MatMul Q K-transpose
8 × 400 × 400
Scale
Divide by sqrt(32)
Softmax over keys
8 × 400 × 400
MatMul attention × V
8 × 400 × 32
Merge heads
400 × 256
Linear output
256 to 256
CSPRep layer
Input feature
512 channels; spatial80/40/20
Conv2d 1×1 / 1
512 to 256, bias=False
BatchNorm2d
256 channels
SiLU
256 channels
Conv2d 1×1 / 1
512 to 256, bias=False
BatchNorm2d
256 channels
SiLU
256 channels
RepVGG block
256 channels
RepVGG block
256 channels
RepVGG block
256 channels
+
Output256 channels. Final projection is identity.
RepVGG block
Input256 channels; same tensor feeds both convolutions.
Conv2d 3×3 / 1
256 to 256, bias=False
BatchNorm2d
256 channels
Conv2d 1×1 / 1
256 to 256, bias=False
BatchNorm2d
256 channels
+
SiLU
256 channels
Unfused graph: independent3×3 and1×1 paths, each with BN.
Decoder block
Object query states
900 ×256
Task token states
16 ×256
Concat object and task tokens
916 ×256; Q/K add object query position
Joint self-attention
916 tokens,8 heads ×32
+
LayerNorm
916 ×256
Split first900 object tokens
900 ×256
Split last16 task tokens
16 ×256, carried to next layer
Deformable image attention
900 queries,3 image levels,4 points/level
+
LayerNorm
900 ×256
Linear
256 to2,048
ReLU
900 ×2,048
Linear
2,048 to256
+
LayerNorm
900 ×256
Joint object/task self-attention
Query input
916 × 256
Key/value input
916 × 256
Linear Q
256 to 256
Reshape heads
8 × 916 × 32
Linear K
256 to 256
Reshape heads
8 × 916 × 32
Linear V
256 to 256
Reshape heads
8 × 916 × 32
MatMul Q K-transpose
8 × 916 × 916
Scale
Divide by sqrt(32)
Softmax over keys
8 × 916 × 916
MatMul attention × V
8 × 916 × 32
Merge heads
916 × 256
Linear output
256 to 256
Task residual MLP
Task tokens
16 ×512
Linear
512 to1,024
ReLU
16 ×1,024
Linear
1,024 to512
+
LayerNorm
16 ×512
Linear
512 to256
Query position MLP:
Reference box coordinates
900 ×4
Linear
4 to512
ReLU
900 ×512
Linear
512 to256
Three-level deformable attention
Query + box position
900 ×256
Linear offsets
256 to192
Linear weights + softmax
256 to96;8 heads ×12 points
Linear value projection
8,400 ×256, reshape8 heads ×32
Sample locations from reference XYWH and offsets
900 ×8 ×3 ×4 ×2; referenceXY + offset/4 ×WH ×0.5
Bilinear grid_sample
256 ×80 ×80 value map;4 samples/query/head
Multiply sampled values × weights
4 ×32 values/query/head
Bilinear grid_sample
256 ×40 ×40 value map;4 samples/query/head
Multiply sampled values × weights
4 ×32 values/query/head
Bilinear grid_sample
256 ×20 ×20 value map;4 samples/query/head
Multiply sampled values × weights
4 ×32 values/query/head
Sum all3 levels and4 points; merge8 heads
900 ×256
Linear output projection
256 to256
CLIP text encoder block
Input 77 × 512
LayerNorm
77 × 512
Multi-head self-attention
8 heads
+
LayerNorm
77 × 512
Linear
512 to 2048
QuickGELU
77 × 2048
Linear
2048 to 512
+
Output 77 × 512
CLIP causal text self-attention
Query input
77 × 512
Key/value input
77 × 512
Linear Q
512 to 512
Reshape heads
8 × 77 × 64
Linear K
512 to 512
Reshape heads
8 × 77 × 64
Linear V
512 to 512
Reshape heads
8 × 77 × 64
MatMul Q K-transpose
8 × 77 × 77
Scale
Divide by sqrt(64)
Causal mask
77 × 77
+
Softmax over keys
8 × 77 × 77
MatMul attention × V
8 × 77 × 64
Merge heads
77 × 512
Linear output
512 to 512
Class scoring and box heads
Per-layer class projection
3 class rows, Linear512 to256
L2 normalize class and object features
3 ×256 and900 ×256
MatMul cosine similarity
900 ×3
Scale by1/0.07
900 ×3 logits
Box MLP, independently parameterized per decoder layer:
Linear
256 to256
ReLU
256
Linear
256 to256
ReLU
256
Linear
256 to4
Add inverse-sigmoid reference; sigmoid
900 ×4 normalized boxes
Swin block and ConvNorm definitions
LayerNorm
Channels96/192/384/768
Shift, pad and partition7×7 windows
Stages160,80,40,20 spatial;even shifts3
Window self-attention
Heads3/6/12/24;32 channels/head
Reverse windows, crop, reverse shift
Restore stage spatial shape
Residual addition
Add block input
LayerNorm
Channels96/192/384/768
Linear, GELU, Linear
MLP widths384/768/1,536/3,072
Residual addition
Add post-attention state
Patch merging: flatten2×2 neighbors, LayerNorm,
Linear384 to192 /768 to384 /1,536 to768.
ConvNormGELU: Conv2d, BatchNorm2d, GELU.
Lateral uses1×1/1; PAN downsample uses3×3/2.
QuickGELU: x × sigmoid(1.702 ×x).
CPU composition used the in-tree timm-parity Swin-T and installed Apache-2.0 OmDet modules, without downloaded weights.
Source: libreyolo/models/openvocab/omdet_turbo.py. Revision a4d0ecc9e17f.
libreyolo.com