SmolVLM2

Install

SmolVLM2 belongs to LibreYOLO's VLM-as-detector tier, a separate product surface from the checkpoint-based families with its own factory. It needs the vlm extra, which also pulls in num2words, a dependency of SmolVLM2's own processor.

pip install "libreyolo[vlm]"

Predict

Weights download from Hugging Face on first use and are cached locally.

from libreyolo import LibreVLM, SAMPLE_IMAGE model = LibreVLM("smolvlm2-500m")model.set_classes(["cat", "dog"])result = model.predict(SAMPLE_IMAGE, save=True) for box in result.boxes:    print(box.cls, box.conf, box.xyxy)

from libreyolo import LibreVLM, SAMPLE_IMAGE model = LibreVLM("smolvlm2-500m") # The escape hatch beneath the detection convenience: any question,# not just a bounding-box query.answer = model.chat(SAMPLE_IMAGE, "What is the cat doing?")print(answer)

This family loads through the LibreVLM() factory, not LibreYOLO(): VLM families declare no checkpoint loader, so the file-suffix routing described on other model pages does not apply here. set_classes() sets the vocabulary SmolVLM2 is asked to find; it is sticky, so it stays in effect across every later predict()/track() call until you set it again. SmolVLM2 needs no parser override in LibreYOLO: it follows the same chat-template-plus-JSON output as the tier's shared default, so its detection prompt and box format are not family-specific. Every detection carries the same placeholder confidence, so conf filtering is all-or-nothing rather than a ranking; iou does have an effect, dropping a later same-class box once it overlaps an already-kept one past the threshold, since a repeating generator can otherwise emit near-duplicate boxes for one object. SmolVLM2 also answers free-form questions through chat(), the same escape hatch documented on the LibreVLM factory. LibreYOLO's CLI does not cover this tier: there is no libreyolo predict model=... form for it. See prediction for sources, streaming and result handling.

Variants

One size in the registry: SmolVLM2-500M-Video-Instruct, loaded as LibreVLM("smolvlm2-500m"). SmolVLM2 is a weaker detector than the purpose-built grounding models in this tier; LibreYOLO's own wrapper describes it as a demonstration that a new family needs no special-case parsing to work here, not as its strongest open-vocabulary option.

LibreYOLO does not train, validate or export SmolVLM2: train(), val() and export() all raise NotImplementedError for every family in this tier (see the support tier above). Fine-tune SmolVLM2 upstream and load the resulting weights if you need a custom vocabulary baked in; check predict() output by eye instead of a COCO-style validation pass, since every detection carries the same placeholder confidence.

Licensing

Citation

@article{marafioti2025smolvlm,
  title={SmolVLM: Redefining small and efficient multimodal models}, 
  author={Andrés Marafioti and Orr Zohar and Miquel Farré and Merve Noyan and Elie Bakouch and Pedro Cuenca and Cyril Zakka and Loubna Ben Allal and Anton Lozhkov and Nouamane Tazi and Vaibhav Srivastav and Joshua Lochner and Hugo Larcher and Mathieu Morlon and Lewis Tunstall and Leandro von Werra and Thomas Wolf},
  journal={arXiv preprint arXiv:2504.05299},
  year={2025}
}