Florence-2 grounding base
Fit diagram
Read at 100%
Clear selection
Download SVG
Download PNG
Click a block to read its description, or select it with Tab and Enter.
Florence-2 grounding base
Image inference at 768 × 768. DaViT vision encoder, BART encoder-decoder and LibreYOLO output adapter.
LibreYOLO
Florence-2 grounding base
Image inference at 768 × 768. DaViT vision encoder, BART encoder-decoder and LibreYOLO output adapter.
DaViT stage sequence
Processor image tensor
1 × 3 × 768 × 768
Conv2d then LayerNorm
3 to 128; k=7, s=4, p=3
Spatial block then channel block
n=1; 128 × 192 × 192
LayerNorm then Conv2d
128 to 256; k=3, s=2, p=1
Spatial block then channel block
n=1; 256 × 96 × 96
LayerNorm then Conv2d
256 to 512; k=3, s=2, p=1
Spatial block then channel block
n=9; 512 × 48 × 48
LayerNorm then Conv2d
512 to 1024; k=3, s=2, p=1
Spatial block then channel block
n=1; 1024 × 24 × 24
Stage heads / channel groups: 4, 8, 16, 32
Both blocks are expanded below; channel width C follows the stage.
Vision projector and source sequence
Final feature map
1 × 1024 × 24 × 24
Add learned 2D positions
Row and column embedding concatenation
Flatten spatial dimensions
1 × 576 × 1024
Add cosine/sine temporal positions
One image frame; temporal index 0
Mean over 576 positions
1 × 1 × 1024
Mean over one frame
1 × 576 × 1024
Concat token axis
577 × 1024
Linear, no bias
1024 to 768
LayerNorm
577 × 768
Replace image placeholders
Prompt tokens use vocabulary embeddings
Add learned source positions
LayerNorm; sequence length L, width 768
BART encoder and decoder
Image and prompt sequence
L × 768; vocabulary51328
Encoder layers
n=6; 12 heads; FFN=3072
Encoder memory
L × 768; supplies cross-attention K/V
Generated prefix embedding
Vocabulary51328; width 768
Add learned target positions
LayerNorm before decoder stack
Decoder layers
n=6; 12 heads; FFN=3072
Vocabulary Linear
768 to 51328; shared embedding weights
Beam search
3 beams; autoregressive token selection
Generated token IDs
Decode with processor
Dropout is disabled in inference. Cached decoding reuses K/V.
LibreYOLO adapter
Task token and query
Phrase grounding
Processor post-process
Decode tags and pixel-coordinate boxes
Extract bboxes and labels
Original image size supplies geometry
Remove whole-image boxes
Match normalized query text
Choose tightest matching box
Fallback to tightest remaining candidate
Box center point
x=(x1+x2)/2; y=(y1+y2)/2
Results.points
One pixel point; synthetic score1
Task: <CAPTION_TO_PHRASE_GROUNDING>
No pretrained inference was run for this diagram.
DaViT spatial / channel block
Use spatial attention first, then channel attention; same skeleton.
Input
Depthwise Conv2d
C channels; kernel3, stride1, padding1
+
Input
LayerNorm + selected attention
See separate spatial and channel insets
+
Input
Depthwise Conv2d
C channels; kernel3, stride1, padding1
+
Input
LayerNorm + MLP
Linear C to 4C; GELU; Linear 4C to C
+
LayerNorm acts on channels; reshape between grids and tokens.
C by stage: 128, 256, 512, 1024. FFN widths: 512, 1024, 2048, 4096.
Spatial attention primitives
LayerNorm
C channels
Pad and partition windows
12 × 12 positions per window
Q/K/V Linear
C to 3C; split 3, heads, 32
Q × transpose(K)
144 × 144 per head; scale1/sqrt(32)
Softmax over keys
144 spatial positions
Attention weights × V
144 × 32 per head
Concat heads
Width C
Output Linear
C to C
Merge windows and crop padding
Restore original H × W × C
Channel attention primitives
LayerNorm
C channels
Q/K/V Linear
C to 3C; split into G groups
Transpose each grouped tensor
32 channels × S spatial positions
Q × transpose(K)
32 × 32 per group; scale1/sqrt(S)
Softmax over channel keys
32 channels per group
Attention weights × V
32 channels × S positions
Restore token and channel order
S × C; S = H × W
Output Linear
C to C
BART decoder layer
Input
Causal self-attention
Q/K/V from target prefix; 12 heads
+
LayerNorm
768 channels
Input
Cross-attention
Q from target; K/V from encoder memory
+
LayerNorm
768 channels
Input
MLP
Linear 768 to 3072; GELU; Linear 3072 to 768
+
LayerNorm
768 channels
Each sublayer uses residual addition followed by LayerNorm.
BART encoder layer
Input
Bidirectional self-attention
Q/K/V from source; 12 heads
+
LayerNorm
768 channels
Input
MLP
Linear 768 to 3072; GELU; Linear 3072 to 768
+
LayerNorm
768 channels
BART attention primitives
Q from sublayer input
Linear 768 to 768; reshape 12 heads
K from input or memory
Linear 768 to 768; reshape 12 heads
V from input or memory
Linear 768 to 768; reshape 12 heads
Q × transpose(K)
Head width 64; inverse sqrt scaling
Add mask; softmax over keys
Causal for decoder self-attention only
Attention weights × V
Encoder memory used only in cross-attention
Concat heads; output Linear
768 to 768
No rotary positions. Learned positions enter embeddings before the encoder and decoder.
MLP primitives
LayerNorm (DaViT only)
BART normalizes after the residual sum
Linear
DaViT C to 4C; BART 768 to 3072
GELU
Elementwise activation
Linear
DaViT 4C to C; BART 3072 to 768
Checked: pinned config and native meta module geometry. Runtime token counts depend on the processor; no weight-based execution.
Source: libreyolo/models/ground/florence.py; Transformers 5.16.1. Revision a4d0ecc9e17f.
libreyolo.com