DINOv2

DINOv2 is a self-supervised vision transformer trained by Meta AI to produce general-purpose image features without labels. LibreYOLO wraps its DINOv2-with-Registers backbone for three tasks: semantic segmentation, classification and whole-image embedding.

Tasks: semantic, classify, embed
Sizes: n, s, m, l at 518 px
Install: pip install libreyolo
Support tier: Supported, since v. Supporting trainables: kept green in CI, features land opportunistically.
Upstream: DINOv2 by Meta AI (FAIR), Apache-2.0. Paper, source
Licenses: Code MIT, weights Apache-2.0. Commercial use

Install

LibreDINOv2 registers only when transformers is installed, the same optional dependency RF-DETR needs for its DINOv2 backbone, so it needs the same extra.

bash

pip install "libreyolo[rfdetr]"

Predict

LibreYOLO does not publish a LibreDINOv2 checkpoint. Construct the wrapper directly instead of loading a file: model_path=None (the default) downloads Meta's Apache-2.0 facebook/dinov2-with-registers-small backbone from Hugging Face on first use. task= selects what runs on top of it.

Semantic

from libreyolo import SAMPLE_IMAGEfrom libreyolo.models.dinov2.model import LibreDINOv2 # No LibreYOLO-hosted checkpoint exists for this family: this# downloads the Apache-2.0 DINOv2-with-Registers-small backbone from# Meta's Hugging Face org. The dense head starts at random# initialization until you train it (see Train below).model = LibreDINOv2(size="s", task="semantic", nb_classes=19)result = model(SAMPLE_IMAGE) mask = result.semantic_maskprint(mask.data.shape, mask.classes)

Classify

from libreyolo import SAMPLE_IMAGEfrom libreyolo.models.dinov2.model import LibreDINOv2 # nb_classes= is your dataset's class count; the linear head starts# at random initialization until you train it.model = LibreDINOv2(size="s", task="classify", nb_classes=10)result = model(SAMPLE_IMAGE) print(result.probs.top1, result.probs.top1conf)

Embed

from libreyolo import SAMPLE_IMAGEfrom libreyolo.models.dinov2.model import LibreDINOv2 # Bypasses every task head: the backbone alone is enough, so this# needs no fine-tuning to be useful.model = LibreDINOv2(size="s", task="embed")result = model(SAMPLE_IMAGE) print(result.embeddings.data.shape)   # (1, D), L2-normalized

Embed a batch

from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="embed") # Convenience wrapper: runs predict() and stacks every row into one# (N, D) tensor.features = model.embed(["a.jpg", "b.jpg", "c.jpg"])print(features.shape)

task="semantic" and task="classify" add a dense or linear head on top of the backbone; that head is randomly initialized and only useful after you train it (see Train). task="embed" skips every head and returns the backbone's final normalized CLS token as one whole-image row in result.embeddings, so it needs no training at all. result.boxes is always None: none of the three tasks produce per-instance detections. See prediction for sources, streaming and result handling.

Variants

size selects the RF-DETR-style projector width layered on top of the backbone, not the backbone itself: every size shares the same DINOv2-S (small) encoder. Semantic segmentation runs at DINOv2's native square patch grid; classification and embedding run at the smaller classification resolution used to train the linear probe.

Train

task="semantic" and task="classify" both train; task="embed" has no class-dependent head to fit and raises NotImplementedError if you call train() on it.

Semantic

from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="semantic", nb_classes=19)model.train(data="my-dataset.yaml", epochs=100, batch_size=4, lr=1e-4)

Classify

from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="classify", nb_classes=10)model.train(data="my-dataset.yaml", epochs=100, batch_size=4, lr=1e-4)

Multi-GPU

from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="semantic", nb_classes=19)model.train(    data="my-dataset.yaml", epochs=100, batch_size=4, lr=1e-4,    device="0,1",)

The primary keyword arguments here are batch_size and lr, not batch and lr0 used by most other families; batch and lr0 are still accepted and mapped onto them, but passing both raises a conflict error. output_dir= (default "runs/train") replaces project=/name= as the primary way to place a run, though passing project=/name= directly still works. See training for datasets, augmentation, multi-GPU and loggers.

Validate

val() returns a dictionary of metrics/ keys: mIoU and pixel accuracy for task="semantic", top-1 and top-5 accuracy for task="classify". task="embed" has no ground truth to score against and raises NotImplementedError if you call val() on it.

Semantic

from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="semantic", nb_classes=19)metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mIoU"])print(metrics["metrics/pixel_accuracy"])

Classify

from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="classify", nb_classes=10)metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/accuracy_top1"])print(metrics["metrics/accuracy_top5"])

Export

Task	ONNX	TorchScript	ExecuTorch	TensorRT	OpenVINO	Paddle	MNN	RKNN	ncnn	TFLite	CoreML	Core AI
semantic	semantic to ONNX: supported.	semantic to TorchScript: supported.	semantic to ExecuTorch: supported.	semantic to TensorRT: supported.	semantic to OpenVINO: supported.	semantic to Paddle: not supported	semantic to MNN: not supported	semantic to RKNN: not supported	semantic to ncnn: not supported	semantic to TFLite: not supported	semantic to CoreML: not supported	semantic to Core AI: not supported
classify	classify to ONNX: supported.	classify to TorchScript: supported.	classify to ExecuTorch: supported.	classify to TensorRT: supported.	classify to OpenVINO: supported.	classify to Paddle: not supported	classify to MNN: not supported	classify to RKNN: not supported	classify to ncnn: not supported	classify to TFLite: not supported	classify to CoreML: not supported	classify to Core AI: supported.
embed	embed to ONNX: supported.	embed to TorchScript: supported.	embed to ExecuTorch: supported.	embed to TensorRT: supported.	embed to OpenVINO: supported.	embed to Paddle: not supported	embed to MNN: not supported	embed to RKNN: not supported	embed to ncnn: not supported	embed to TFLite: supported.	embed to CoreML: not supported	embed to Core AI: not supported

Each task supports a different subset of formats, shown above. An exported artifact loads back through LibreYOLO() on its file suffix, so a .onnx or .engine file behaves like a checkpoint and returns the same Results. Export lists the arguments every format accepts.

Semantic

from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="semantic", nb_classes=19)model.export(format="onnx")

Classify

from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="classify", nb_classes=10)model.export(format="onnx")

Embed

from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="embed")model.export(format="tflite")

Use the exported file

from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads# like any checkpoint and returns the same Results object. Export# names the file from the task, here LibreDINOv2s-sem.onnx.model = LibreYOLO("LibreDINOv2s-sem.onnx")result = model(SAMPLE_IMAGE)

Licensing

Check the license on the Hugging Face repository of the specific weights you download. Every checkpoint in the LibreYOLO org carries one, and they are not always the same across a family. That repository is the authoritative source; the summary below describes what applied when this page was last verified.

This is a description of the licenses involved, not legal advice. If the answer matters commercially, read the licenses yourself and take your own counsel.

Original work: DINOv2, Meta AI (FAIR)
Upstream license: Apache-2.0
Upstream source: github.com/facebookresearch/dinov2
LibreYOLO code: MIT
Weights: Apache-2.0, distributed by their authors. LibreYOLO does not host or mirror them.
Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. LibreYOLO does not host or republish a DINOv2 checkpoint of its own: LibreDINOv2 downloads the pretrained backbone directly from Meta's facebook/dinov2-with-registers-small repository on Hugging Face the first time it runs, unmodified. The semantic and classification heads start at random initialization until you train them, since LibreYOLO does not publish a trained head for this family.

The "Weights" row above names the license that applies, Apache-2.0, but nothing is actually republished under the LibreYOLO Hugging Face org for this family: LibreYOLO hosts no LibreDINOv2 checkpoint of its own. What LibreDINOv2(model_path=None) downloads is Meta's own facebook/dinov2-with-registers-small repository, untouched.

Citation

@misc{oquab2023dinov2,
  title={DINOv2: Learning Robust Visual Features without Supervision},
  author={Oquab, Maxime and Darcet, Timothée and Moutakanni, Theo and Vo, Huy V. and Szafraniec, Marc and Khalidov, Vasil and Fernandez, Pierre and Haziza, Daniel and Massa, Francisco and El-Nouby, Alaaeldin and Howes, Russell and Huang, Po-Yao and Xu, Hu and Sharma, Vasu and Li, Shang-Wen and Galuba, Wojciech and Rabbat, Mike and Assran, Mido and Ballas, Nicolas and Synnaeve, Gabriel and Misra, Ishan and Jegou, Herve and Mairal, Julien and Labatut, Patrick and Joulin, Armand and Bojanowski, Piotr},
  journal={arXiv:2304.07193},
  year={2023}
}

Copied from the authors' citation block at github.com/facebookresearch/dinov2#citing-dinov2.