ViTMatte Small

Click a block to read its description, or select it with Tab and Enter.

ViTMatte SmallGuided alpha matting,input normalized RGB3 channels + trimap1 channel at512 × 512,native eval.LibreYOLOViTMatte SmallGuided alpha matting,input normalized RGB3 channels + trimap1 channel at512 × 512,native eval.ViTDet-S backboneRGB + trimap input4 × 512 × 512; trimap0,0.5,1Conv2d16×16,stride164 to384;384 × 32 × 32Add bicubic absolute positionsLearned14×14 grid resized32×32; discardCLS slotWindow transformer blocks,n=2Indices0,1;384channels;6headsGlobal transformer + CNN bottleneckIndex2;384 × 32 × 32Window transformer blocks,n=2Indices3,4;384channels;6headsGlobal transformer + CNN bottleneckIndex5;384 × 32 × 32Window transformer blocks,n=2Indices6,7;384channels;6headsGlobal transformer + CNN bottleneckIndex8;384 × 32 × 32Window transformer blocks,n=2Indices9,10;384channels;6headsGlobal transformer + CNN bottleneckIndex11;384 × 32 × 32Window blocks pad32 to42,partition nine14×14 windows,then crop back. There is no cyclic window shift orCLS token.Convolution detail streamInput RGB + trimap4 × 512 × 512ConvBNReLU3×3,stride24 to48;256×256ConvBNReLU3×3,stride248 to96;128×128ConvBNReLU3×3,stride296 to192;64×64Retain raw input plus all three detail levels as decoder skips.All three convolutions usepadding1 andbias=False.BatchNorm epsilon1e-5; ReLU follows normalization.Detail fusion and alphaBilinear resize×2384 × 64 × 64;align_cornersFalseConcat detail skip thenupsampledfeature192+384=576 channelsConvBNReLU3×3,stride1576 to256;padding1Bilinear resize×2256 × 128 × 128;align_cornersFalseConcat detail skip thenupsampledfeature96+256=352 channelsConvBNReLU3×3,stride1352 to128;padding1Bilinear resize×2128 × 256 × 256;align_cornersFalseConcat detail skip thenupsampledfeature48+128=176 channelsConvBNReLU3×3,stride1176 to64;padding1Bilinear resize×264 × 512 × 512;align_cornersFalseConcat detail skip thenupsampledfeature4+64=68 channelsConvBNReLU3×3,stride168 to32;padding1Matting headConv3×3:32to16;BN;ReLU;Conv1×1:16to1Sigmoid1 × 512 × 512 alpha probabilitiesApply exact known trimap labelsBackground0 forcesalpha0;foreground1 forcesalpha1Transformer layerInput384channels at32×32LayerNorm384channels,epsilon1e-6Attention on local windows orfull gridLocal196tokens;global1024tokens;6heads+LayerNorm384channels,epsilon1e-6Linear384 to1536GELULinear1536 to384+Global layers subsequently run the residual CNN bottleneck.Attention and decomposed positionsLinearQKV384 to1152;split into6 heads of64Q × transpose(K) /8196² local or1024² global logits/headAdd Q-dependent height relative termeinsum(Q,relative_height)Add Q-dependent width relative termeinsum(Q,relative_width)Softmax over keysWeights × V64 channels/headConcat heads + Linear384 to384Each relative table is27×64 for14×14 window attention;63×64 for32×32 global attention. Tables resize if needed.CNN residual after each global layerGlobal transformer output384 × 32 × 32Conv1×1384 to192;biasFalseChannelLayerNorm + GELU192channels,epsilon1e-6Conv3×3192 to192;padding1,biasFalseChannelLayerNorm + GELU192channels,epsilon1e-6Conv1×1192 to384;biasFalseChannelLayerNorm384channels,epsilon1e-6+No additional finalbackbone normalization after layer11.Convolutional primitivesConv2d3×3Numericchannels/stride at occurrenceBatchNorm2dReLUMatting head differs:Conv3×3(32to16,biasTrue),BatchNorm16,ReLU,Conv1×1(16to1,biasTrue).Known trimap pixels are constrained after the single decoder sigmoid; unknown0.5 pixels keep predictedalpha.Source: libreyolo/models/vitmatte/nn.py and model.py. Revision a4d0ecc9e17f.libreyolo.com