ViTMatte

ViTMatte predicts a foreground alpha matte from an image and a trimap.

Tasks: background removal
Sizes: s at 512 px
Install: pip install libreyolo
Support tier: Inference only, since v1.6.0. Predict, validate and export only. Training features do not apply.
Upstream: ViTMatte by HUST Vision Lab, MIT code; Apache-2.0 weights. Paper, source
Licenses: Code MIT, weights MIT code; Apache-2.0 weights. Commercial use

Install

bash

pip install "libreyolo"

Predict

Python

from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreViTMattes-matte.pt", device="cpu")import numpy as npfrom PIL import Image image = Image.open(SAMPLE_IMAGE).convert("RGB")# An all-unknown guide demonstrates the input contract. Supply a real trimap for matting.trimap = np.full((image.height, image.width), 128, dtype=np.uint8)result = model(image, trimap=trimap)result.save("cutout.png")

A trimap marks background as 0, unknown pixels as 128 and foreground as 255. Pass it through trimap= for a single image. Validation accepts trimap_dir or derives guides with trimap_radius. Training and export are not supported.

Validate

Python

from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreViTMattes-matte.pt", device="cpu")# A folder with images/ and mattes/ (grayscale alpha mattes, same file stems), or a matte YAML.metrics = model.val(data="path/to/your/matte_dataset", workers=0)print(metrics)

Use the dataset format for this task. Validation explains the dataset requirements and returned metrics.

Checkpoints

File	Weights license
Background removal
LibreViTMattes-matte.pt	other

Every file above exists in the LibreYOLO org today and downloads on first use.

Licensing

Check the license on the Hugging Face repository of the specific weights you download. Every checkpoint in the LibreYOLO org carries one, and they are not always the same across a family. That repository is the authoritative source; the summary below describes what applied when this page was last verified.

This is a description of the licenses involved, not legal advice. If the answer matters commercially, read the licenses yourself and take your own counsel.

Original work: ViTMatte, HUST Vision Lab
Upstream license: MIT code; Apache-2.0 weights
Upstream source: github.com/hustvl/ViTMatte
LibreYOLO code: MIT
Weights: MIT code; Apache-2.0 weights, republished at huggingface.co/LibreYOLO
Interpretation: The original code is MIT. The publisher card for hustvl/vitmatte-small-composition-1k declares Apache-2.0 for the weights.

Citation

@article{yao2024vitmatte,
  title={ViTMatte: Boosting image matting with pre-trained plain vision transformers},
  author={Yao, Jingfeng and Wang, Xinggang and Yang, Shusheng and Wang, Baoyuan},
  journal={Information Fusion},
  volume={103},
  pages={102091},
  year={2024},
  publisher={Elsevier}
}

Copied from the authors' citation block at raw.githubusercontent.com/hustvl/ViTMatte/main/README.md.