# RTMDet
RTMDet is a single-stage detector that predicts from one point-based prior per grid location, no anchors, through a head whose convolutions are shared across feature levels. LibreYOLO supports it for detection and RTMDet-Ins instance segmentation.
Tasks: Detection, Instance segmentation. Install: pip install libreyolo.
Verified against LibreYOLO v1.5.0.

## Install

RTMDet needs no extra beyond the base package.

```bash
pip install libreyolo
```

## Predict

Weights download from Hugging Face on first use and are cached locally.

**Python**

```python
from libreyolo import LibreYOLO, SAMPLE_IMAGE

model = LibreYOLO("LibreRTMDets.pt")
result = model(SAMPLE_IMAGE, save=True)

for box in result.boxes:
    print(box.cls, box.conf, box.xyxy)
```

**CLI**

```bash
libreyolo predict model=LibreRTMDets.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True
```

**Instance segmentation**

```python
from libreyolo import LibreYOLO, SAMPLE_IMAGE

# The -seg suffix in the filename selects the RTMDet-Ins mask head,
# so no task argument is needed here.
model = LibreYOLO("LibreRTMDets-seg.pt")
result = model(SAMPLE_IMAGE, save=True)

print(result.masks.data.shape)
```

The returned `Results` object is the one every family returns, so swapping in a
different detector is a one line change. A `-seg` filename resolves to the
RTMDet-Ins task on its own, and `result.masks` then carries the instance
masks alongside the boxes. `conf` sets the confidence threshold and `iou` the
NMS threshold. See [prediction](/docs/predict) for sources, streaming and
result handling.

## Variants

Five sizes, `t` through `x`, share one architecture at a common input
resolution. This family carries no benchmark table here: compare sizes by
checkpoint file size in the table below.

## Train

**Python**

```python
from libreyolo import LibreYOLO

model = LibreYOLO("LibreRTMDets.pt")
model.train(
    data="my-dataset.yaml",
    epochs=300, imgsz=640, batch=16, lr0=0.004,
)
```

**CLI**

```bash
libreyolo train model=LibreRTMDets.pt data=my-dataset.yaml imgsz=640 epochs=300 batch=16 lr0=0.004
```

Detection trains through `train()`. The QualityFocalLoss, GIoU and
DynamicSoftLabelAssigner components are ported from upstream mmdetection, and
the forward pass and ONNX export are bit-equivalent to it, with postprocessing
matching mmdet's output within 0.001 mAP on val2017 subsets.

What has not been checked, per `train()`'s own docstring: small-dataset
fine-tune convergence, from-scratch paper parity, multi-GPU behavior, cached
Mosaic and MixUp throughput, the strict upstream two-stage pipeline switch,
and the paramwise weight-decay overrides that zero decay on norm and bias
parameters.

RTMDet-Ins has no training path. Calling `train()` on a `-seg` checkpoint, or
with `task="segment"`, raises `NotImplementedError`; instance segmentation
supports inference and validation only.

`train()` also accepts a `pretrained` argument, but the value is never read
inside the method: training always continues from whatever weights the model
was constructed with, so `pretrained=False` does not reinitialize the
network.

Left alone otherwise, the trainer runs 300 epochs with AdamW at `lr0=0.004`
and `weight_decay=0.05`, a 1-epoch warmup on a cosine schedule, and Mosaic and
MixUp switched off for the final 20 epochs.

See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers.

## Validate

`val()` returns a dictionary of `metrics/` keys covering precision, recall,
mAP 50 and mAP 50-95, measured against any dataset in the format you trained on.

**Python**

```python
from libreyolo import LibreYOLO

model = LibreYOLO("LibreRTMDets.pt")
metrics = model.val(data="my-dataset.yaml")

print(metrics["metrics/mAP50-95"])
print(metrics["metrics/mAP50"])
```

**CLI**

```bash
libreyolo val model=LibreRTMDets.pt data=my-dataset.yaml
```

**Instance segmentation**

```python
from libreyolo import LibreYOLO

model = LibreYOLO("LibreRTMDets-seg.pt")
metrics = model.val(data="my-dataset.yaml")

print(metrics["metrics/mAP50-95(M)"])   # masks
print(metrics["metrics/mAP50-95(B)"])   # boxes
```

Against a `-seg` checkpoint the plain `metrics/mAP50-95` key holds the mask
score, and the same run also reports boxes under `(B)` and masks under `(M)`
so both are available from one pass.

## Export

| Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Detection | yes | yes | yes | yes | yes |  |  |  |  |  |  | yes |
| Instance segmentation |  |  |  |  |  |  |  |  |  |  |  |  |

A "yes" means the export is supported. An empty cell means the exporter refuses that combination.

Detection exports to most formats; instance segmentation currently exports to
none of them; the matrix above reflects that split. An exported detection
artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or
`.engine` file behaves like a checkpoint and returns the same `Results`.
Running the graph in a bare runtime, with no LibreYOLO installed, is also
supported, but then preprocessing and postprocessing are yours to write.

**Python**

```python
from libreyolo import LibreYOLO

model = LibreYOLO("LibreRTMDets.pt")
model.export(format="onnx", imgsz=640)
model.export(format="tensorrt", imgsz=640, half=True)
```

**CLI**

```bash
libreyolo export model=LibreRTMDets.pt format=onnx imgsz=640
libreyolo export model=LibreRTMDets.pt format=tensorrt imgsz=640 half=True
```

**Use the exported file**

```python
from libreyolo import LibreYOLO, SAMPLE_IMAGE

# The factory routes on the file suffix, so an exported artifact loads
# like any checkpoint and returns the same Results object.
model = LibreYOLO("LibreRTMDets.onnx")
result = model(SAMPLE_IMAGE)

print(result.boxes.xyxy)
```

## Checkpoints

Every published weight file for this family.

| File | Input (px) | Task | Weights license |
| --- | --- | --- | --- |
| `LibreRTMDett.pt` | 640 | Detection | apache-2.0 |
| `LibreRTMDets.pt` | 640 | Detection | apache-2.0 |
| `LibreRTMDetm.pt` | 640 | Detection | apache-2.0 |
| `LibreRTMDetl.pt` | 640 | Detection | apache-2.0 |
| `LibreRTMDetx.pt` | 640 | Detection | apache-2.0 |
| `LibreRTMDett-seg.pt` | 640 | Instance segmentation | apache-2.0 |
| `LibreRTMDets-seg.pt` | 640 | Instance segmentation | apache-2.0 |
| `LibreRTMDetm-seg.pt` | 640 | Instance segmentation | apache-2.0 |
| `LibreRTMDetl-seg.pt` | 640 | Instance segmentation | apache-2.0 |
| `LibreRTMDetx-seg.pt` | 640 | Instance segmentation | apache-2.0 |

## Licensing

Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice.

- Original work: RTMDet, OpenMMLab
- Upstream license: Apache-2.0
- Upstream source: https://github.com/open-mmlab/mmdetection
- LibreYOLO code: Apache-2.0
- Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO
- Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours. The published RTMDet and RTMDet-Ins checkpoints are converted from mmdetection's own COCO weights, trained by OpenMMLab under the same license.

## Citation

```bibtex
@misc{lyu2022rtmdet,
      title={RTMDet: An Empirical Study of Designing Real-Time Object Detectors},
      author={Chengqi Lyu and Wenwei Zhang and Haian Huang and Yue Zhou and Yudong Wang and Yanyi Liu and Shilong Zhang and Kai Chen},
      year={2022},
      eprint={2212.07784},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}
```

Copied from https://github.com/open-mmlab/mmdetection/tree/main/configs/rtmdet#citation


## RTMDet architecture

- RTMDet-T: [SVG](/diagrams/models/rtmdet/t.svg), [interactive diagram](/diagrams/models/rtmdet/t.html)
- RTMDet-S: [SVG](/diagrams/models/rtmdet/s.svg), [interactive diagram](/diagrams/models/rtmdet/s.html)
- RTMDet-M: [SVG](/diagrams/models/rtmdet/m.svg), [interactive diagram](/diagrams/models/rtmdet/m.html)
- RTMDet-L: [SVG](/diagrams/models/rtmdet/l.svg), [interactive diagram](/diagrams/models/rtmdet/l.html)
- RTMDet-X: [SVG](/diagrams/models/rtmdet/x.svg), [interactive diagram](/diagrams/models/rtmdet/x.html)
- RTMDet Ins-T: [SVG](/diagrams/models/rtmdet/t-seg.svg), [interactive diagram](/diagrams/models/rtmdet/t-seg.html)
- RTMDet Ins-S: [SVG](/diagrams/models/rtmdet/s-seg.svg), [interactive diagram](/diagrams/models/rtmdet/s-seg.html)
- RTMDet Ins-M: [SVG](/diagrams/models/rtmdet/m-seg.svg), [interactive diagram](/diagrams/models/rtmdet/m-seg.html)
- RTMDet Ins-L: [SVG](/diagrams/models/rtmdet/l-seg.svg), [interactive diagram](/diagrams/models/rtmdet/l-seg.html)
- RTMDet Ins-X: [SVG](/diagrams/models/rtmdet/x-seg.svg), [interactive diagram](/diagrams/models/rtmdet/x-seg.html)
- RTMDet TS family: [SVG](/diagrams/models/rtmdet/family-ts.svg), [interactive diagram](/diagrams/models/rtmdet/family-ts.html)
- RTMDet MLX family: [SVG](/diagrams/models/rtmdet/family-mlx.svg), [interactive diagram](/diagrams/models/rtmdet/family-mlx.html)
- RTMDet Ins TSMLX family: [SVG](/diagrams/models/rtmdet/family-tsmlx-seg.svg), [interactive diagram](/diagrams/models/rtmdet/family-tsmlx-seg.html)