> This is the documentation for LibreYOLO 1.5.0. Current version: https://www.libreyolo.com/docs/models/dinov2

# DINOv2
DINOv2 is a self-supervised vision transformer trained by Meta AI to produce general-purpose image features without labels. LibreYOLO wraps its DINOv2-with-Registers backbone for three tasks: semantic segmentation, classification and whole-image embedding.
Tasks: semantic, classify, embed. Install: pip install libreyolo.
Verified against LibreYOLO v1.5.0.

## Install

LibreDINOv2 registers only when `transformers` is installed, the same
optional dependency RF-DETR needs for its DINOv2 backbone, so it needs the
same extra.

```bash
pip install "libreyolo[rfdetr]"
```

## Predict

LibreYOLO does not publish a LibreDINOv2 checkpoint. Construct the wrapper
directly instead of loading a file: `model_path=None` (the default) downloads
Meta's Apache-2.0 `facebook/dinov2-with-registers-small` backbone from Hugging
Face on first use. `task=` selects what runs on top of it.

**Semantic**

```python
from libreyolo import SAMPLE_IMAGE
from libreyolo.models.dinov2.model import LibreDINOv2

# No LibreYOLO-hosted checkpoint exists for this family: this
# downloads the Apache-2.0 DINOv2-with-Registers-small backbone from
# Meta's Hugging Face org. The dense head starts at random
# initialization until you train it (see Train below).
model = LibreDINOv2(size="s", task="semantic", nb_classes=19)
result = model(SAMPLE_IMAGE)

mask = result.semantic_mask
print(mask.data.shape, mask.classes)
```

**Classify**

```python
from libreyolo import SAMPLE_IMAGE
from libreyolo.models.dinov2.model import LibreDINOv2

# nb_classes= is your dataset's class count; the linear head starts
# at random initialization until you train it.
model = LibreDINOv2(size="s", task="classify", nb_classes=10)
result = model(SAMPLE_IMAGE)

print(result.probs.top1, result.probs.top1conf)
```

**Embed**

```python
from libreyolo import SAMPLE_IMAGE
from libreyolo.models.dinov2.model import LibreDINOv2

# Bypasses every task head: the backbone alone is enough, so this
# needs no fine-tuning to be useful.
model = LibreDINOv2(size="s", task="embed")
result = model(SAMPLE_IMAGE)

print(result.embeddings.data.shape)   # (1, D), L2-normalized
```

**Embed a batch**

```python
from libreyolo.models.dinov2.model import LibreDINOv2

model = LibreDINOv2(size="s", task="embed")

# Convenience wrapper: runs predict() and stacks every row into one
# (N, D) tensor.
features = model.embed(["a.jpg", "b.jpg", "c.jpg"])
print(features.shape)
```

`task="semantic"` and `task="classify"` add a dense or linear head on top of
the backbone; that head is randomly initialized and only useful after you
train it (see [Train](#train)). `task="embed"` skips every head and returns
the backbone's final normalized CLS token as one whole-image row in
`result.embeddings`, so it needs no training at all. `result.boxes` is
always `None`: none of the three tasks produce per-instance detections. See
[prediction](/docs/v1.5.0/predict) for sources, streaming and result handling.

## Variants

`size` selects the RF-DETR-style projector width layered on top of the
backbone, not the backbone itself: every size shares the same DINOv2-S
(small) encoder. Semantic segmentation runs at DINOv2's native square patch
grid; classification and embedding run at the smaller classification
resolution used to train the linear probe.

## Train

`task="semantic"` and `task="classify"` both train; `task="embed"` has no
class-dependent head to fit and raises `NotImplementedError` if you call
`train()` on it.

**Semantic**

```python
from libreyolo.models.dinov2.model import LibreDINOv2

model = LibreDINOv2(size="s", task="semantic", nb_classes=19)
model.train(data="my-dataset.yaml", epochs=100, batch_size=4, lr=1e-4)
```

**Classify**

```python
from libreyolo.models.dinov2.model import LibreDINOv2

model = LibreDINOv2(size="s", task="classify", nb_classes=10)
model.train(data="my-dataset.yaml", epochs=100, batch_size=4, lr=1e-4)
```

**Multi-GPU**

```python
from libreyolo.models.dinov2.model import LibreDINOv2

model = LibreDINOv2(size="s", task="semantic", nb_classes=19)
model.train(
    data="my-dataset.yaml", epochs=100, batch_size=4, lr=1e-4,
    device="0,1",
)
```

The primary keyword arguments here are `batch_size` and `lr`, not `batch` and
`lr0` used by most other families; `batch` and `lr0` are still accepted and
mapped onto them, but passing both raises a conflict error. `output_dir=`
(default `"runs/train"`) replaces `project=`/`name=` as the primary way to
place a run, though passing `project=`/`name=` directly still works. See
[training](/docs/v1.5.0/train) for datasets, augmentation, multi-GPU and loggers.

## Validate

`val()` returns a dictionary of `metrics/` keys: mIoU and pixel accuracy for
`task="semantic"`, top-1 and top-5 accuracy for `task="classify"`.
`task="embed"` has no ground truth to score against and raises
`NotImplementedError` if you call `val()` on it.

**Semantic**

```python
from libreyolo.models.dinov2.model import LibreDINOv2

model = LibreDINOv2(size="s", task="semantic", nb_classes=19)
metrics = model.val(data="my-dataset.yaml")

print(metrics["metrics/mIoU"])
print(metrics["metrics/pixel_accuracy"])
```

**Classify**

```python
from libreyolo.models.dinov2.model import LibreDINOv2

model = LibreDINOv2(size="s", task="classify", nb_classes=10)
metrics = model.val(data="my-dataset.yaml")

print(metrics["metrics/accuracy_top1"])
print(metrics["metrics/accuracy_top5"])
```

## Export

| Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| semantic | yes | yes | yes | yes | yes |  |  |  |  |  |  |  |
| classify | yes | yes | yes | yes | yes |  |  |  |  |  |  | yes |
| embed | yes | yes | yes | yes | yes |  |  |  |  | yes |  |  |

A "yes" means the export is supported. An empty cell means the exporter refuses that combination.

Each task supports a different subset of formats, shown above. An exported
artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or
`.engine` file behaves like a checkpoint and returns the same `Results`.
[Export](/docs/v1.5.0/export) lists the arguments every format accepts.

**Semantic**

```python
from libreyolo.models.dinov2.model import LibreDINOv2

model = LibreDINOv2(size="s", task="semantic", nb_classes=19)
model.export(format="onnx")
```

**Classify**

```python
from libreyolo.models.dinov2.model import LibreDINOv2

model = LibreDINOv2(size="s", task="classify", nb_classes=10)
model.export(format="onnx")
```

**Embed**

```python
from libreyolo.models.dinov2.model import LibreDINOv2

model = LibreDINOv2(size="s", task="embed")
model.export(format="tflite")
```

**Use the exported file**

```python
from libreyolo import LibreYOLO, SAMPLE_IMAGE

# The factory routes on the file suffix, so an exported artifact loads
# like any checkpoint and returns the same Results object. Export
# names the file from the task, here LibreDINOv2s-sem.onnx.
model = LibreYOLO("LibreDINOv2s-sem.onnx")
result = model(SAMPLE_IMAGE)
```

## Licensing

Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice.

- Original work: DINOv2, Meta AI (FAIR)
- Upstream license: Apache-2.0
- Upstream source: https://github.com/facebookresearch/dinov2
- LibreYOLO code: MIT
- Weights: Apache-2.0, distributed by their authors. LibreYOLO does not host or mirror them.
- Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. LibreYOLO does not host or republish a DINOv2 checkpoint of its own: LibreDINOv2 downloads the pretrained backbone directly from Meta's facebook/dinov2-with-registers-small repository on Hugging Face the first time it runs, unmodified. The semantic and classification heads start at random initialization until you train them, since LibreYOLO does not publish a trained head for this family.

The "Weights" row above names the license that applies, Apache-2.0, but
nothing is actually republished under the LibreYOLO Hugging Face org for this
family: LibreYOLO hosts no LibreDINOv2 checkpoint of its own. What
`LibreDINOv2(model_path=None)` downloads is Meta's own
`facebook/dinov2-with-registers-small` repository, untouched.

## Citation

```bibtex
@misc{oquab2023dinov2,
  title={DINOv2: Learning Robust Visual Features without Supervision},
  author={Oquab, Maxime and Darcet, Timothée and Moutakanni, Theo and Vo, Huy V. and Szafraniec, Marc and Khalidov, Vasil and Fernandez, Pierre and Haziza, Daniel and Massa, Francisco and El-Nouby, Alaaeldin and Howes, Russell and Huang, Po-Yao and Xu, Hu and Sharma, Vasu and Li, Shang-Wen and Galuba, Wojciech and Rabbat, Mike and Assran, Mido and Ballas, Nicolas and Synnaeve, Gabriel and Misra, Ishan and Jegou, Herve and Mairal, Julien and Labatut, Patrick and Joulin, Armand and Bojanowski, Piotr},
  journal={arXiv:2304.07193},
  year={2023}
}
```

Copied from https://github.com/facebookresearch/dinov2#citing-dinov2


## DINOv2 architecture

- DINOv2 n semantic: [SVG](/diagrams/models/dinov2/n-semantic.svg), [interactive diagram](/diagrams/models/dinov2/n-semantic.html)
- DINOv2 s semantic: [SVG](/diagrams/models/dinov2/s-semantic.svg), [interactive diagram](/diagrams/models/dinov2/s-semantic.html)
- DINOv2 m semantic: [SVG](/diagrams/models/dinov2/m-semantic.svg), [interactive diagram](/diagrams/models/dinov2/m-semantic.html)
- DINOv2 l semantic: [SVG](/diagrams/models/dinov2/l-semantic.svg), [interactive diagram](/diagrams/models/dinov2/l-semantic.html)
- DINOv2 family semantic: [SVG](/diagrams/models/dinov2/family-semantic.svg), [interactive diagram](/diagrams/models/dinov2/family-semantic.html)
- DINOv2 n classify: [SVG](/diagrams/models/dinov2/n-classify.svg), [interactive diagram](/diagrams/models/dinov2/n-classify.html)
- DINOv2 s classify: [SVG](/diagrams/models/dinov2/s-classify.svg), [interactive diagram](/diagrams/models/dinov2/s-classify.html)
- DINOv2 m classify: [SVG](/diagrams/models/dinov2/m-classify.svg), [interactive diagram](/diagrams/models/dinov2/m-classify.html)
- DINOv2 l classify: [SVG](/diagrams/models/dinov2/l-classify.svg), [interactive diagram](/diagrams/models/dinov2/l-classify.html)
- DINOv2 family classify: [SVG](/diagrams/models/dinov2/family-classify.svg), [interactive diagram](/diagrams/models/dinov2/family-classify.html)
- DINOv2 n embed: [SVG](/diagrams/models/dinov2/n-embed.svg), [interactive diagram](/diagrams/models/dinov2/n-embed.html)
- DINOv2 s embed: [SVG](/diagrams/models/dinov2/s-embed.svg), [interactive diagram](/diagrams/models/dinov2/s-embed.html)
- DINOv2 m embed: [SVG](/diagrams/models/dinov2/m-embed.svg), [interactive diagram](/diagrams/models/dinov2/m-embed.html)
- DINOv2 l embed: [SVG](/diagrams/models/dinov2/l-embed.svg), [interactive diagram](/diagrams/models/dinov2/l-embed.html)
- DINOv2 family embed: [SVG](/diagrams/models/dinov2/family-embed.svg), [interactive diagram](/diagrams/models/dinov2/family-embed.html)