# D-FINE
A detection transformer that reformulates box regression as a probability distribution over each box edge, refined across decoder layers. LibreYOLO supports it for detection and instance segmentation.
Tasks: Detection, Instance segmentation. Install: pip install libreyolo.
Verified against LibreYOLO v1.6.0.

## Install

D-FINE needs no optional extra. Everything it imports is in the base install.

```bash
pip install libreyolo
```

Adapter fine-tuning with `lora=True` is the exception, and needs the `lora`
extra.

```bash
pip install "libreyolo[lora]"
```

## Predict

Weights download from Hugging Face on first use and are cached locally.

**Python**

```python
from libreyolo import LibreYOLO, SAMPLE_IMAGE

model = LibreYOLO("LibreDFINEn.pt")
result = model(SAMPLE_IMAGE, save=True)

for box in result.boxes:
    print(box.cls, box.conf, box.xyxy)
```

**CLI**

```bash
libreyolo predict model=LibreDFINEn.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True
```

**Instance segmentation**

```python
from libreyolo import LibreYOLO, SAMPLE_IMAGE

# The -seg suffix in the filename selects the mask head, so no task
# argument is needed here.
model = LibreYOLO("LibreDFINEn-seg.pt")
result = model(SAMPLE_IMAGE, save=True)

print(result.masks.data.shape)
```

The returned `Results` object is the one every family returns, so swapping in a
different detector is a one line change. A `-seg` filename resolves to the
segmentation task on its own, and `result.masks` then carries the instance
masks alongside the boxes. `conf` and `max_det` filter the query selection;
`iou` is accepted for API parity but has no effect, because the decoder is a set
predictor with no NMS step. See [prediction](/docs/predict) for sources,
streaming and result handling.

## Variants

Five sizes. They all run at the same input resolution, so the table separates
them by parameter count and accuracy.

| Checkpoint | Input (px) | mAP 50-95 | Params (M) |
| --- | --- | --- | --- |
| `LibreDFINEl` | 640 | 60.0 | 31.24 |
| `LibreDFINEm` | 640 | 57.8 | 19.59 |
| `LibreDFINEn` | 640 | 45.8 | 3.78 |
| `LibreDFINEs` | 640 | 53.4 | 10.32 |
| `LibreDFINEx` | 640 | 61.4 | 62.62 |

COCO val2017, 500 images. Published on https://www.visionanalysis.org/

Interactive chart: https://www.visionanalysis.org/embed/scatter?highlight=dfine-l%2Cdfine-m%2Cdfine-n%2Cdfine-s%2Cdfine-x&title=D-FINE%20on%20COCO&subtitle=Accuracy%20against%20latency%2C%20every%20benchmarked%20model

Segmentation reuses the detection backbone, encoder and decoder and adds a mask
head, so a `-seg` checkpoint takes the same arguments as its detect sibling.
LibreYOLO's RT-DETRv4 family is written as a subclass of the D-FINE wrapper: it
inherits this decoder line and then pins its task list back to detection,
because it carries no mask head.

## Train

Training starts from a published checkpoint, for both tasks.

**Python**

```python
from libreyolo import LibreYOLO

model = LibreYOLO("LibreDFINEn.pt")
model.train(data="my-dataset.yaml", epochs=50, imgsz=640, batch=8, lr0=2e-4)
```

**CLI**

```bash
libreyolo train model=LibreDFINEn.pt data=my-dataset.yaml \
  epochs=50 imgsz=640 batch=8 lr0=2e-4
```

**Instance segmentation**

```bash
# Continues from published segmentation weights, mask head included.
libreyolo train model=LibreDFINEn-seg.pt data=my-dataset.yaml \
  task=segment epochs=50 imgsz=640
```

**Segmentation from detect weights**

```bash
# Detect weights carry no mask head, so this is an explicit transfer:
# the head starts untrained and is only useful once trained. Asking for
# task=segment here is what authorizes the transfer.
libreyolo train model=LibreDFINEn.pt data=my-dataset.yaml \
  task=segment epochs=50 imgsz=640
```

**LoRA**

```python
from libreyolo import LibreYOLO

model = LibreYOLO("LibreDFINEn.pt")
model.train(data="my-dataset.yaml", epochs=50, lora=True)
```

**Multi-GPU**

```bash
libreyolo train model=LibreDFINEn.pt data=my-dataset.yaml \
  epochs=50 device=0,1 batch=16
```

Left alone, the trainer runs 132 epochs at `lr0=2e-4` with `amp=True` and `amp_dtype="float16"`, a batch
of 16 and early stopping after 50 epochs without improvement. Detect weights are
a legal starting point for segmentation training, but only as an explicit
transfer, since the mask head begins untrained and would otherwise return
meaningless masks. Passing `task=segment` to the CLI is what authorizes it. The
Python route is narrower: `LibreDFINE` has to be constructed directly with
`allow_detect_to_segment_transfer=True`, because the `LibreYOLO()` factory takes
no such argument, and direct construction does not download, so the weights file
must already be on disk.

`lora=True` applies to detection. Segment training rejects it and points at
`freeze='backbone'` instead, because the mask head has not been tested with
adapters. On Apple silicon the trainer moves the whole run to CPU: the backward
pass of the Integral's binned matmul hits a Metal compilation failure. Inference
on MPS is unaffected.

See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers.

## Validate

`val()` returns a dictionary keyed by metric name, and prints per-class results
when `verbose` is left on.

**Python**

```python
from libreyolo import LibreYOLO

model = LibreYOLO("LibreDFINEn.pt")
metrics = model.val(data="my-dataset.yaml")

print(metrics["metrics/mAP50-95"])
print(metrics["metrics/mAP50"])
```

**CLI**

```bash
libreyolo val model=LibreDFINEn.pt data=my-dataset.yaml
```

**Instance segmentation**

```python
from libreyolo import LibreYOLO

model = LibreYOLO("LibreDFINEn-seg.pt")
metrics = model.val(data="my-dataset.yaml")

print(metrics["metrics/mAP50-95(M)"])   # masks
print(metrics["metrics/mAP50-95(B)"])   # boxes
```

Against a `-seg` checkpoint the plain `metrics/mAP50-95` key holds the mask
score, and the same run also reports boxes under `(B)` and masks under `(M)` so
both are available from one pass.

## Export

| Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Detection | yes | yes |  | yes | yes | yes | yes |  |  |  |  | yes |
| Instance segmentation | yes | yes |  | yes | yes |  |  |  |  |  |  |  |

A "yes" means the export is supported. An empty cell means the exporter refuses that combination.

An exported artifact loads back through `LibreYOLO()` on its file suffix, so a
`.onnx` or `.engine` file behaves like a checkpoint and returns the same
`Results`. The OpenVINO, Paddle, MNN and Core AI paths export at a fixed canvas
rather than dynamic shapes. [Export](/docs/export) lists the arguments every
format accepts and the extras a few of them add.

**Python**

```python
from libreyolo import LibreYOLO

model = LibreYOLO("LibreDFINEn.pt")
model.export(format="onnx", imgsz=640)
model.export(format="tensorrt", imgsz=640, half=True)
```

**CLI**

```bash
libreyolo export model=LibreDFINEn.pt format=onnx imgsz=640
libreyolo export model=LibreDFINEn.pt format=tensorrt imgsz=640 half=True
```

**Use the exported file**

```python
from libreyolo import LibreYOLO, SAMPLE_IMAGE

# The factory routes on the file suffix, so an exported artifact loads
# like any checkpoint and returns the same Results object.
model = LibreYOLO("LibreDFINEn.onnx")
result = model(SAMPLE_IMAGE)

print(result.boxes.xyxy)
```

## Checkpoints

Every published weight file for this family.

| File | Input (px) | Task | Weights license |
| --- | --- | --- | --- |
| `LibreDFINEn.pt` | 640 | Detection | apache-2.0 |
| `LibreDFINEs.pt` | 640 | Detection | apache-2.0 |
| `LibreDFINEm.pt` | 640 | Detection | apache-2.0 |
| `LibreDFINEl.pt` | 640 | Detection | apache-2.0 |
| `LibreDFINEx.pt` | 640 | Detection | apache-2.0 |
| `LibreDFINEn-seg.pt` | 640 | Instance segmentation | apache-2.0 |
| `LibreDFINEs-seg.pt` | 640 | Instance segmentation | apache-2.0 |
| `LibreDFINEm-seg.pt` | 640 | Instance segmentation | apache-2.0 |
| `LibreDFINEl-seg.pt` | 640 | Instance segmentation | apache-2.0 |
| `LibreDFINEx-seg.pt` | 640 | Instance segmentation | apache-2.0 |

## Licensing

Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice.

- Original work: D-FINE, University of Science and Technology of China
- Upstream license: Apache-2.0
- Upstream source: https://github.com/Peterande/D-FINE
- LibreYOLO code: Apache-2.0
- Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO
- Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours. The segmentation weights carry a second Apache-2.0 upstream, ArgoHA/D-FINE-seg, under the same terms.

The segmentation weights have a second upstream: their mask decoder, mask
matching and mask loss come from ArgoHA/D-FINE-seg, also Apache-2.0, whose
maintainer approved reuse with attribution.

## Citation

```bibtex
@misc{peng2024dfine,
      title={D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement},
      author={Yansong Peng and Hebei Li and Peixi Wu and Yueyi Zhang and Xiaoyan Sun and Feng Wu},
      year={2024},
      eprint={2410.13842},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}
```

Copied from https://github.com/Peterande/D-FINE#citation