# LibreYOLO documentation > The complete LibreYOLO documentation as a single markdown document. > 176 pages. Source: https://www.libreyolo.com/docs > Each page is also available on its own at .md LibreYOLO is an MIT-licensed computer vision library: one Python API across detection, segmentation, pose, depth, OCR and more, with training, validation and export for each. The code is MIT, so what you build with it stays yours. Pretrained weights carry the license of whoever trained them, stated per checkpoint. --- # Changelog A summary of the repository changelog: what 1.5.0 shipped, and what 1.4.0 shipped before it. Verified against LibreYOLO v1.5.0. ## 1.5.0, released 2026-08-09 The largest release so far: 28 new model families, four new tasks, five new export formats, two new serving backends, and an `import libreyolo` that no longer pulls PyTorch. 600 commits and 88 merged pull requests. Summarized from [`CHANGELOG.md`](https://github.com/LibreYOLO/libreyolo/blob/release/CHANGELOG.md), which carries the full entries. Two defaults move numbers and four arguments changed shape: [upgrading to 1.5.0](/docs/upgrade) is the short list of what that asks of you. ### Breaking changes No public model class or function was removed, and `__all__` grew from 101 names to 142. Four things need a code edit: - `allow_experimental=True` is gone from every `.train()` gate. Delete the argument; a call that still passes it raises `TypeError`. - The export support tier `"experimental"` is gone. `Tier` is now `validated`, `available` or `blocked`. - `pretrained=False` combined with `resume` raises `ValueError` instead of proceeding incoherently. - CLI `--imgsz` widened from int to str so it can carry `480x640`. Typing `--imgsz 640` in a shell and calling `model.predict(imgsz=640)` are both unaffected; only direct Python calls into the CLI command functions need a string. Three changes move metrics at default settings: the COCO backend, YOLOX BatchNorm eps on checkpoints trained before this release, and D-FINE's per-size multi-scale recipe. All of it, with before and after code, is on [upgrading to 1.5.0](/docs/upgrade). ### New tasks The task vocabulary went from 13 entries to 17, with nothing removed. Each new task ships an original-canvas result payload, visualization, dataset schema and filename suffix. [`edge`](/docs/tasks/edge-detection) and [`normal`](/docs/tasks/surface-normals) are dense-prediction contracts, with validators for edge ODS/OIS and normal angular error. [`embed`](/docs/tasks/face-recognition) produces L2-normalized image and region embeddings. It ships `LibreFaceEmbedder`, a `Gallery` and `FaceGallery` API, and 1:N identification from `libreyolo predict` through `--gallery` and `--gallery-threshold`. The task is deliberately general: identity vectors are one use, region embeddings and re-identification are others. [`mesh`](/docs/tasks/body-mesh) is body mesh recovery. `LibreSAM3DBody` is an optional model gated on the `sam_3d_body` dependency, so it is not exported from the top-level package. ### New model families 28 new user-facing classes, exported under 29 names. All are inference-only unless stated. Dome-DETR (`domedetr`) is the trainable addition: a tiny-object detector for aerial, drone and remote-sensing imagery, ported from the Dome-DETR release. It is D-FINE plus a density head (DeFE), encoder attention restricted to occupied windows (MWAS), and a query count set by local density rather than a fixed 300 (PAQI). Sizes s, m and l at 800x800, with a maximum absolute difference of 0.0 against upstream on all six published checkpoints. Training is wired against upstream's full objective, though the published 160-epoch schedule has not been reproduced, so the paper's AP numbers are unverified. Its advantage narrows as objects grow, so it sits beside D-FINE rather than replacing it. There is no COCO checkpoint upstream, only AI-TOD-V2 with 9 classes and VisDrone with 12, so every canonical filename carries a dataset suffix. Export raises instead of emitting a graph, because PAQI's per-image query count makes a traced graph valid only for the image it was traced on. Weights are not rehosted: the upstream card claims Apache-2.0 while also restricting use to academic research. [LingBot-Vision](/docs/models/lingbot-vision) is the other trainable arrival, a ViT semantic-segmentation family at 512 px pairing boundary-centric self-supervised backbones with a 1x1 dense head. Sizes s, b and l are published; `g` is the 1.1B teacher and has no LibreYOLO-hosted checkpoint, so requesting it raises with the reason. The rest are inference-only ports of earlier published work, each checked against its pinned upstream source: - Detection: [DETR](/docs/models/detr), [Deformable DETR](/docs/models/deformable-detr), [DINO-DETR](/docs/models/dino-detr), [LW-DETR](/docs/models/lw-detr), [Faster R-CNN](/docs/models/faster-rcnn), [Mask R-CNN](/docs/models/mask-rcnn), [FCOS](/docs/models/fcos), [RetinaNet](/docs/models/retinanet), [SSD300](/docs/models/ssd), [CenterNet](/docs/models/centernet) and [EfficientDet](/docs/models/efficientdet). - Classification: [AlexNet](/docs/models/alexnet), [VGG](/docs/models/vgg), [ViT](/docs/models/vit), [DeiT](/docs/models/deit) and [Swin](/docs/models/swin). - Semantic segmentation: [FCN](/docs/models/fcn) and [DeepLabv3](/docs/models/deeplabv3). - Depth: [MiDaS](/docs/models/midas). Pose: [HRNet](/docs/models/hrnet), top-down on COCO-17 person crops. - Surface normals: [MoGe-2](/docs/models/moge-2) at 518 px, a DINOv2 patch grid with letterbox preprocessing. - Edge: [TEED](/docs/models/teed) and [DexiNed](/docs/models/dexined), native MIT-licensed architectures with local checkpoint converters. Their upstream BIPED-trained checkpoints are not bundled, mirrored or auto-downloaded, because that dataset's terms are non-commercial. - Embeddings: [LibreFaceEmbedder](/docs/models/librefacerec), ONNX Runtime only. [LibreMODUS](/docs/models/libremodus) adds 14B-A7B analysis-only inference for depth, normals, edges and COCO detection from one checkpoint, plus phrase grounding and image-conditioned `any2any()` chaining. [LibreFeyNobg](/docs/models/feynobg) adds a matte family at 1024 px, code and weights Apache-2.0. [RT-DETRv2](/docs/models/rt-detr) gained [oriented-object detection](/docs/tasks/oriented-detection) inference for the official DOTA 1.0 checkpoints in sizes n, s, m, l and x, with aspect-preserving preprocessing, native `Results.obb` output, validation, and validated ONNX and TorchScript export. ### Torch-free ONNX inference `import libreyolo` no longer pulls torch. Model and results names resolve through a lazy `__getattr__`, and a new top-level `libreyolo.preprocess` package provides numpy-native preprocessing for deim, deimv2, dfine, ec, rfdetr, rtdetr, yolo9, yolonas and yolox, so an ONNX Runtime install needs no torch at all. See [lightweight install](/docs/lightweight-install). ### Export and serving Five new export formats: [RKNN](/docs/export/rknn) for Rockchip, with `--name` for the target platform and `--verify` for PC-simulator versus ONNX Runtime parity; [MNN](/docs/export/mnn); [Paddle](/docs/export/paddle); [ExecuTorch](/docs/export/executorch); and [Core AI](/docs/export/coreai) on macOS. [DeepStream](/docs/export/deepstream) sidecar configs are written by `deepstream=True` on [ONNX](/docs/export/onnx) export. It is an ONNX-only option rather than a `format=` key, and is mutually exclusive with `nms=True`. ExecuTorch and MNN exports now require a checkpoint sidecar, `.pte.json` and `.mnn.json`. Two new inference backends: [Triton](/docs/export/triton), with `create_triton_config` for NVIDIA Triton Inference Server, and Paddle Inference. ### New CLI commands `libreyolo enroll` builds an embedding gallery from a folder-per-person tree, and `libreyolo compare`, aliased as `libreyolo verify`, checks two images by cosine similarity. See the [CLI reference](/docs/cli). ### Speed CUDA graph capture of the training step went from 2 families to 24, across detect, classify, semantic, point and restore. Measured on an RTX 5070 Ti under AMP: 3.63x on FOMO, 2.74x on MobileNetV4, 1.99x on YOLO9-t, and 1.04x to 1.26x for the rest, with the win tracking how much of a step is network rather than loss. End to end on a 20-epoch YOLO9-t fine-tune of 406 images, dataloader and validation included, 428 s became 368 s with identical mAP50-95 and per-epoch losses. Capture at prediction time reaches 39 families, spanning detect, segment, pose, point, classify, semantic, depth, restore, matte and OCR. Every enabled family is verified to replay bit-identically against two probe inputs. A family that cannot be captured whole is split at a verified seam, and the remainder runs eagerly with identical numbers. `pip install libreyolo[hub-kernels]` opts every Deformable-DETR-lineage family into a compiled Apache-2.0 CUDA kernel for multi-scale deformable attention, pinned to an audited revision. It applies to eager CUDA fp32 only; exports keep the portable path. Fused scaled dot product attention now runs across the transformer families using stock torch, with no optional dependency. Measured on an RTX 5070 Ti under fp16 autocast, roughly 1.8x on Swin window attention and 3.7x on OWLv2 vision attention. Families held to a byte-exact parity bar keep manual attention by default and opt in explicitly, and export graphs keep the primitive-op equation either way. COCO metrics moved to faster-coco-eval by default: 15.6x faster overall and 56x on detection-dense datasets, decided on measured parity across all 100 RF100-VL test splits, where 1381 of 1400 metric values were bit-identical to pycocotools, the maximum deviation was 2.22e-16, and headline deltas were exactly 0. `--no-faster-coco-eval` opts out, and pycocotools stays the automatic fallback when the package is missing. ### Training and validation Training from scratch (`pretrained=False`) now works for every g0, g1 and g2 family through a seeded random init. Previously only yolo9, rfdetr and dfine had a scratch path, and the other families silently loaded the pretrained checkpoint anyway. Rectangular input arrived: `imgsz=480x640` for prediction and validation everywhere it makes sense, and rectangular *training* for the seven CNN detect families, yolo9, yolo9_e2e, yolo9_p2, yolox, yolo7, rtmdet and picodet. Transformer families and non-detect tasks raise a clear error, and both dimensions must divide the family stride. Autobatch and DDP spawn honor the rectangular size when probing. `amp_dtype` selects bfloat16 for [training](/docs/train) and [validation](/docs/train/validation), on `TrainConfig`, `ValidationConfig` and as `--amp-dtype` on `train`, `val` and `profile`. GradScaler is skipped for bf16. Validation caps are configurable too: `max_det` defaults to 300, and `eval_max_det` decouples the COCO evaluator cap from NMS. `val_loss=True` reached every trainable family that can support it, across detect, classify, semantic and restore rather than the flagships alone. It moved from `YOLO9Config` and `RFDETRConfig` to `TrainConfig`, so a family that has not implemented it now raises a clear error instead of ignoring the flag. Denoising terms are never included, because validation forwards without ground truth. Comet, ClearML, Neptune and DVCLive joined the built-in training [loggers](/docs/train/loggers), with the same canonical metrics and failure-isolation contract as the existing TensorBoard, MLflow and Weights & Biases integrations. The kernel registry moved to `libreyolo/kernels/`, organized by purpose. `LIBREYOLO_KERNELS` replaces `LIBREYOLO_QUANT_KERNELS`, which is still honored, and `libreyolo.quant.kernels` remains a working alias. ### Predict sources Webcams by index, RTSP and HTTP streams, screen capture, and `s3://` and `gs://` URLs, with `--stream`, `--stream-buffer`, `--vid-stride` and `--show`. Live sources implicitly enable streaming, which emits one JSON record per frame. See [prediction sources](/docs/predict/sources). ### Fixes `train(..., cuda_graph=True)` had been a silent no-op for D-FINE, DEIM, DEIMv2, RT-DETRv4 and EC, whose trainers called the eager forward instead of the routed one, so capture never engaged. The end-to-end suite now asserts that capture actually engages per family. YOLOX applies its BatchNorm `eps=1e-3` and `momentum=0.03` at construction instead of as a fixup afterwards, so the values survive the class-count rebuild that `train()` performs when a dataset's class count differs from the checkpoint. A fine-tune previously trained and reported in-training validation at torch's default `eps=1e-5` but was reloaded for inference at `1e-3`. On RF100-VL `ball`, the same nano checkpoint scores 0.566 mAP50-95 evaluated at its trained eps and 0.151 after a stock reload. Checkpoints trained before this fix carry the old semantics and need the override described on [upgrading](/docs/upgrade). Non-strict checkpoint loads now warn with the counts and first names of missing and unexpected state-dict keys. Shape mismatches always raised, but name mismatches let a partially matching checkpoint load and then predict with freshly initialized tensors, leaving no trace. Auto-conversion no longer risks the file it writes: the new `.pt` is staged and atomically renamed rather than written in place, and the original file mode is preserved instead of collapsing to owner-only. Interrupted weight downloads record a validator for `If-Range` resume and take a cross-process lock, so a changed remote file or a second process can no longer produce corrupt weights. CUDA graph capture no longer races with DataLoader pin-memory threads, which had killed runs with a capture error from the pin-memory thread, and graphs are captured and replayed on the right device. D-FINE training now applies upstream's per-size multi-scale recipe instead of a hardcoded `base_size_repeat=3`. Several families silently produced wrong geometry and no longer do: PicoDet clipped boxes against swapped axes, RTMDet segmentation resized masks to a square derived from width only, Faster R-CNN ONNX exports placed boxes wrong on non-square images, LW-DETR could drop real detections by running top-K over unmapped COCO columns, and an exported MoGe-2 model run at an aspect ratio other than its export canvas stretched the image into wrong normal directions and now raises instead. ### Contributors **juni3227**, first-time contributor: rectangular input resolution for the convolution-based detectors, and a fix for training with YOLO-format datasets at rectangular model sizes (#649, #658). **Xuban Ceccon**, maintainer: everything else. ### Release stats 600 commits, 902 files changed, +125,701 / -3,315 lines, 88 merged pull requests, 8 issues closed. 174 new test modules, 279 to 453. 38 new documentation pages. ## 1.4.0, released 2026-07-24 The release notes summarize it as 15 new model families, 3 new tasks, a quantization stack, two new trackers, and a multi-GPU training correctness overhaul. The new families were SegFormer, SwinIR, Real-ESRGAN, BiRefNet, ZipDepth, Depth Anything 3, PP-OCRv5, SigLIP 2, YOLOv1, SAM 3, EdgeTAM, PicoSAM3, OMDet-Turbo, OV-DEIM and SenseNova-Vision. The new tasks were `panoptic`, `matte` and `ocr`, each with its result type and validator, and EoMT gained instance and panoptic segmentation. Quantization arrived as the `libreyolo quantize` command and `model.quantize()`, with fp16, bf16, fp8, int8, w4a16, w4a8, nvfp4 and mxfp4 recipes, and quantization-aware training through `train()` on a quantized checkpoint, for yolo9 and rfdetr. Tracking gained BoT-SORT and Deep OC-SORT with an OSNet-AIN re-identification embedder. The multi-GPU work was about correctness. DDP now shards correctly for DEIM, D-FINE and YOLO-NAS-pose, where every rank had been training the full dataset at the full batch, and loss normalizers are globally all-reduced to match single-GPU gradients. SyncBatchNorm defaults on for the CNN detectors, and a non-divisible global batch or a custom loader that does not shard now raises at setup instead of running wrong. Several training fixes changed results outright: RTMDet fine-tune collapse from missing head init took an nc=1 rebuild from 0.26 to 0.709 mAP50-95, YOLOv7 training color-space and fp16 overflow bugs took a fine-tune from 0.0 to 0.92 mAP50, and PicoDet's and DEIM's learning-rate defaults were lowered because the old ones destroyed pretrained weights. Segmentation training no longer exhausts RAM on COCO-scale datasets with multiple workers. One compatibility note from that release: checkpoints written with the new task strings or with finalized quantization state cannot be loaded by 1.3.1. ## Earlier releases Releases before 1.4.0 are documented in the [GitHub releases](https://github.com/LibreYOLO/libreyolo/releases) only, not in `CHANGELOG.md`. Their documentation is still online, listed on [versions](/docs/versions). --- # Citation A complete LibreYOLO citation has two parts: the library, and the published work behind the model family that produced the result. Verified against LibreYOLO v1.5.0. ## Citing LibreYOLO The repository publishes its citation metadata as [`CITATION.cff`](https://github.com/LibreYOLO/libreyolo/blob/release/CITATION.cff), not as a BibTeX block. GitHub reads that file and offers a Cite this repository button on the repository page, which generates APA and BibTeX from it. Take the entry from there rather than typing one. The file in full: ```yaml cff-version: 1.2.0 message: "If you use LibreYOLO in your research or software, please cite it as below." title: "LibreYOLO" type: software authors: - family-names: Ceccon given-names: Xuban - name: "The LibreYOLO contributors" license: MIT url: "https://github.com/LibreYOLO/libreyolo" repository-code: "https://github.com/LibreYOLO/libreyolo" ``` It carries no version and no release date on purpose. [`RELEASING.md`](https://github.com/LibreYOLO/libreyolo/blob/release/RELEASING.md) tells maintainers never to bump, date or retitle `CITATION.cff` or `.zenodo.json` during a release, so that every citation lands on one record instead of scattering across versions. Report the version you ran in your own text, and leave the citation alone. ## Citing the model family LibreYOLO is a port. Running `LibreRFDETRm.pt` means running RF-DETR, and the people who wrote RF-DETR are the ones a reviewer expects to see credited. Citing the library on its own attributes their work to the wrong project. Everything needed sits on the family's page. The Upstream row in the header names the original work and the organization behind it, and links the paper and the source repository. The Citation section further down holds the BibTeX. That BibTeX is copied verbatim from the authors' own citation block, normally the Citation section of the upstream README or a `CITATION.cff`, and it renders with a link back to the block it came from so you can check it against the source. It is never assembled from paper metadata. An entry rebuilt by hand fails quietly and expensively: a dropped coauthor, the wrong venue, the wrong entry type, a year belonging to the preprint. Preprints also get accepted, so an entry may be an `@inproceedings` even when the version you read was on arXiv. Copy the block as it stands. If your bibliography style needs a different entry type, convert the entry rather than retyping it, and keep the author list in its original order. ## What a methods section needs Three things make a LibreYOLO result reproducible and correctly attributed: - The library, cited from `CITATION.cff`, together with the version you ran. `libreyolo version` prints it, along with the Python, torch and CUDA versions it is running against. - The upstream work, cited from the Citation section of the family's page. - The exact checkpoint filename, such as `LibreRFDETRm.pt`. Sizes within a family behave differently, and several families publish checkpoints trained on different datasets under the same prefix, so the family name alone does not identify what ran. Attribution is also a license term for much of what LibreYOLO publishes. Apache-2.0 and the CC BY family both require the notice to travel with the weights you redistribute, which is a separate obligation from citing a paper. See [licensing](/docs/licensing) for which terms apply to which checkpoint. --- # libreyolo doctor Runs a set of health checks over a detection dataset and reports what would hurt a training run: missing files, broken labels, corrupt images, split leakage and class imbalance. Verified against LibreYOLO v1.5.0. ## Synopsis ```bash libreyolo doctor [key=value ...] ``` The dataset is positional, and `data=` is accepted as an alternative. Giving both with different values exits with `config_conflict`. Everything else is a `key=value` pair, and POSIX form works too, so `imgsz=1024` and `--imgsz 1024` are the same argument. ## Arguments | Argument | Default | Meaning | |---|---|---| | `data` | | Positional. Dataset YAML in YOLO detection format, e.g. `coco8.yaml`. Required | | `imgsz` | `640` | Training image size used for pixel-based checks such as tiny objects | | `fast` | `false` | Skip image decoding, which drops the corruption, duplicate and leakage checks | | `skip` | | Comma-separated check ids or families to skip, e.g. `images,labels.tiny_object` | | `only` | | Comma-separated check ids or families to run exclusively | | `strict` | `false` | Warnings also fail the exit code, for CI gates | | `download` | `false` | Allow URL-based dataset download if missing. Never scripts | | `json` | `false` | JSON output to stdout | | `quiet` | `false` | Suppress stderr | | `help_json` | `false` | Dump command schema as JSON and exit | ### Check families `skip` and `only` accept either a full check id or a family prefix, so `images` selects every `images.*` check. | Family | Covers | |---|---| | `config` | The dataset YAML itself: missing `names`, `nc` against `names`, missing splits, unresolvable `path`, duplicate class names | | `files` | Image and label pairing: missing labels, missing images, orphan labels, unsupported extensions, case collisions | | `labels` | Label content: syntax, polygon lines, class ids out of range, coordinates out of range, degenerate boxes, tiny objects, huge boxes, extreme aspect ratios, duplicate boxes, crowded images, identical files | | `images` | Pixel data: corrupt files, EXIF orientation, unusual color modes, tiny or extreme dimensions, uniform images, exact and near duplicates | | `splits` | Leakage between splits, exact and near | | `balance` | Class distribution: classes with zero or few instances, imbalance, split coverage, background ratio, split skew | ## Examples **Basic** ```bash # download=true lets the bundled coco8.yaml fetch its images if missing. libreyolo doctor coco8.yaml download=true ``` **Fast pass, no image decoding** ```bash libreyolo doctor coco8.yaml download=true fast=true ``` **CI gate on selected checks** ```bash libreyolo doctor coco8.yaml download=true strict=true json=true \ only=labels,files,config ``` ## Notes ### Exit codes `0` when no errors were found, `1` when any finding is an error. With `strict=true`, warnings raise the exit code to `1` as well, which is the setting a CI gate wants. Usage problems have their own codes: `2` for an unknown check id or family in `skip` or `only`, `3` when the dataset cannot be found, and `3` when the dataset is not detection shaped. ### Selection resolves before the scan `skip` and `only` are resolved against the check registry before anything is read from disk, so a typo fails immediately rather than after a long image pass. A selector that matches nothing is an error, and the message lists the known families. If the combination of `skip`, `only` and `fast` leaves no checks to run, that is also an error rather than a silent pass. ### Downloads The dataset is not fetched unless `download=true`, and only URL downloads are ever performed. An embedded Python download script in a dataset YAML is never executed by this command, whatever the flag. ### Scope The checks are written for detection datasets. A dataset whose labels are pose, segmentation or oriented-box shaped is detected and refused with `data_invalid` rather than scored against the wrong rules. ### Output The human report goes to stdout, and `json=true` replaces it with a structured object carrying the summary counts, the dataset statistics, every finding, and the list of checks that were skipped. Related: [`libreyolo train`](/docs/cli/train), the run this command is meant to be run before. --- # libreyolo export Converts one checkpoint into one deployment format and writes the artifact under weights/. The format decides which of the arguments below apply. Verified against LibreYOLO v1.5.0. ## Synopsis ```bash libreyolo export model= [format=] [key=value ...] ``` Arguments are `key=value` pairs, and POSIX form works too, so `format=onnx` and `--format onnx` are the same argument. ## Arguments | Argument | Default | Meaning | |---|---|---| | `model` | | Model weights `.pt`. Required | | `format` | `onnx` | Export format: `onnx`, `torchscript`, `executorch`, `tensorrt`, `openvino`, `paddle`, `mnn`, `rknn`, `ncnn`, `tflite`, `coreml`, `coreai` | | `name` | | RKNN target platform, currently `rk3588` only. Rejected with any other format | | `imgsz` | | Input image size: `640` or `480x640` (HxW). `480,640` is also accepted. The model's own size when unset | | `batch` | `1` | Export batch size | | `half` | `false` | FP16 precision | | `int8` | `false` | INT8 quantization | | `dynamic` | `false` | Dynamic input shapes (ONNX) | | `simplify` | `true` | ONNX graph simplification | | `nms` | `false` | Embed NMS in the model. ONNX and CoreML only | | `conf` | `0.25` | Confidence threshold for embedded NMS | | `iou` | `0.45` | IoU threshold for embedded NMS | | `max_det` | `300` | Maximum detections for ONNX embedded NMS | | `opset` | | ONNX opset version. Chosen automatically when unset | | `data` | | Calibration data for INT8 | | `fraction` | `1.0` | Fraction of calibration data to use | | `device` | `auto` | Device for tracing | | `allow_download_scripts` | `false` | Allow embedded Python in dataset YAML download blocks | | `json` | `false` | JSON output to stdout | | `quiet` | `false` | Suppress stderr | | `verbose` | `false` | Verbose export logging | | `verify` | `false` | Run the RKNN Toolkit2 PC simulator and compare against ONNX Runtime. RKNN only | | `help_json` | `false` | Dump command schema as JSON and exit | `engine` is an alias for `tensorrt` and `litert` an alias for `tflite`. Both resolve to the canonical name before anything is written, so the JSON output and the log line always report `tensorrt` or `tflite`. ## Examples **Basic** ```bash # Writes weights/LibreYOLO9s.onnx libreyolo export model=LibreYOLO9s.pt format=onnx imgsz=640 ``` **NMS inside the graph** ```bash libreyolo export model=LibreYOLO9s.pt format=onnx \ nms=true conf=0.25 iou=0.45 max_det=300 ``` **Run the artifact** ```bash libreyolo export model=LibreYOLO9s.pt format=onnx imgsz=640 # The factory routes on the file suffix, so the export loads like a checkpoint. libreyolo predict model=weights/LibreYOLO9s.onnx \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` ## Notes ### Where the file lands The command takes no output path. The artifact is written to `weights/`, named after the source checkpoint's stem plus the format's suffix, with `_fp16` or `_int8` inserted when one of those precisions was requested. `LibreYOLO9s.pt` exported to ONNX at FP16 becomes `weights/LibreYOLO9s_fp16.onnx`. The JSON result carries the resolved `output_path`, the file size in MB, and the input shape as `[batch, 3, height, width]`. ### Combinations that are refused `nms=true` is accepted for ONNX and CoreML and refused for every other format with `nms_unsupported_format`. On ONNX it forces `dynamic` off, since the embedded graph is fixed at batch 1, and says so on stderr. On CoreML it takes `conf` and `iou` but not `max_det`, so a non-default `max_det` alongside `format=coreml nms=true` exits with `config_unsupported`. `half=true` together with `int8=true` is not an error. INT8 wins, `half` is dropped, and a warning goes to stderr. `name` and `verify` are RKNN options today. Passing either with another format exits with `config_unsupported` rather than being ignored. ### Which formats a family supports Support is per family and per task, not global. `libreyolo formats family= task=` prints the tier for each format for that combination, with the reason and any constraint attached. See [`libreyolo formats`](/docs/cli/utilities) for the arguments. Some formats need an optional install and some need a toolchain. A missing Python dependency exits with `export_dep_missing`; a precision the format cannot produce exits with `format_precision_unsupported`. ### Running what you exported Exported artifacts load through the same model factory as checkpoints, keyed on the file suffix, so `libreyolo predict model=weights/LibreYOLO9s.onnx` works without any further conversion. Three prediction options are the exception and are refused on runtime backends: `tiling`, `overlap_ratio` and `output_file_format`. Two deployment targets have pages of their own: [NVIDIA DeepStream](/docs/export/deepstream) and [NVIDIA Jetson](/docs/export/jetson). ### Output and exit codes stdout carries the result; progress goes to stderr. The exit code is `0` on success, `2` for a usage or configuration error, `4` when the model cannot be loaded, `5` for an unknown format, a missing export dependency, an unsupported precision or a refused embedded-NMS request, and `1` for other runtime failures. Related: [`libreyolo quantize`](/docs/cli/quantize), which stays in PyTorch and writes a checkpoint rather than a deployment artifact. --- # libreyolo label Starts a local web tool for drawing and editing bounding boxes. It writes LibreYOLO-native label files, so a dataset annotated here trains with no conversion step. Verified against LibreYOLO v1.5.0. ## Synopsis ```bash libreyolo label [data=] [key=value ...] ``` Arguments are `key=value` pairs, and POSIX form works too, so `port=9200` and `--port 9200` are the same argument. ## Arguments | Argument | Default | Meaning | |---|---|---| | `data` | | Dataset YAML or folder to open directly. Starts on the project home when unset | | `host` | `127.0.0.1` | Host or interface to bind | | `port` | `8000` | Port to bind. Bumps to the next free one if taken | | `device` | `auto` | Device for AI auto-label: `0`, `cpu`, `mps`, `auto` | | `no_assist` | `false` | Disable AI auto-label, leaving a manual labeler | | `no_browser` | `false` | Do not auto-open the browser | | `share` | `false` | Bind on `0.0.0.0` so teammates on your network can join | | `json` | `false` | JSON output to stdout | | `quiet` | `false` | Suppress stderr | | `verbose` | `false` | Verbose stderr output | ## Examples **Basic** ```bash # Opens the project home; pick or create a dataset in the browser. libreyolo label ``` **Manual only, fixed port** ```bash libreyolo label no_assist=true port=9200 no_browser=true ``` **Let teammates join** ```bash libreyolo label share=true ``` ## Notes ### What it writes Boxes are saved as LibreYOLO-native `labels/*.txt` files, which is the format `libreyolo train` reads, so nothing has to be converted afterwards. This version handles bounding boxes only. Edits save as you move between images. ### Opening a dataset With no `data`, the tool starts on the project home and a dataset is chosen or created from the browser. Passing `data=path/to/data.yaml` opens that dataset straight away, and the startup line reports the image count, the class count, and whether the dataset is writable. A read-only dataset still opens and says why it cannot be written to. ### Sharing, and what `host` does `share=true` binds the wildcard address, which lets other machines on your network reach the tool while administrative actions, switching or deleting projects and starting compute, stay on this machine. Setting `host` to a specific interface does something different and less safe: the host becomes indistinguishable from a network client, so every client gets administrative rights. The command prints a warning on stderr when you do it. Prefer `share=true`. ### Ports and shutdown An occupied port moves to the next one, up to twenty past the request. Failing all twenty exits with `io_error`. The URL printed on stdout is the port that was actually bound. With `share=true`, the result also carries `lan_url`, the address teammates should open. The command serves in the foreground until Ctrl+C. Related: [`libreyolo doctor`](/docs/cli/doctor) to check the labeled dataset before training, and [`libreyolo train`](/docs/cli/train) to train on it. --- # libreyolo monitor Serves a web dashboard for training runs, reading the artifacts a run writes to disk. It never attaches to the training process, so live, finished and crashed runs all display. Verified against LibreYOLO v1.5.0. ## Synopsis ```bash libreyolo monitor [] [key=value ...] ``` The directory is positional. Everything else is a `key=value` pair, and POSIX form works too, so `port=9100` and `--port 9100` are the same argument. ## Arguments | Argument | Default | Meaning | |---|---|---| | `run_dir` | `runs` | Positional. A runs root to watch, or a single run directory to open directly. Either way every run under the root is listed | | `host` | `127.0.0.1` | Host or interface to bind | | `port` | `8420` | Port to bind. Bumps to the next free one if taken | | `no_browser` | `false` | Do not auto-open the browser | | `json` | `false` | JSON output to stdout | | `quiet` | `false` | Suppress stderr | | `verbose` | `false` | Verbose stderr output | ## Examples **Basic** ```bash # Watches runs/ and lists every run under it. libreyolo monitor ``` **A different runs root** ```bash libreyolo monitor experiments/ ``` **One run, fixed port, no browser** ```bash libreyolo monitor runs/train/exp port=9100 no_browser=true ``` ## Notes ### One server, many runs The server watches a runs root rather than a single run, and addresses each run by URL, so several runs on one machine share one port. Open the root URL for the index, or one tab per run; the `?run=` parameter in each URL identifies which. Pointing the command at a single run directory roots the server at that directory's parent, so sibling runs still appear in the index, and deep-links straight to the one named. ### What it reads The dashboard is built from the files `libreyolo train` writes: `status.json`, `metrics.jsonl`, `train.log` and the run's images. Nothing is read from the training process itself, so a run that has finished, or died, displays exactly as a live one does. ### Preconditions and ports At least one run must already exist. With no argument and no `runs/` directory, the command exits with `source_not_found`; the same happens when the directory given holds no runs. An occupied port moves to the next one, up to twenty past the request. Failing all twenty exits with `io_error`. The URL printed on stdout is the port that was actually bound. The command serves in the foreground until Ctrl+C. `json=true` prints the URL, the root being watched and the number of runs found, as one object with `schema_version`. Related: [`libreyolo train`](/docs/cli/train), whose `project` and `name` arguments decide where these run directories go. --- # libreyolo predict Runs a loaded model over one source and prints the predictions. The source may be an image, a directory, a video, a URL or a live stream; the model may be a checkpoint or an exported artifact. Verified against LibreYOLO v1.5.0. ## Synopsis ```bash libreyolo predict source= [model=] [key=value ...] ``` Arguments are `key=value` pairs. The same command also accepts POSIX form, so `conf=0.4` and `--conf 0.4` are interchangeable, and a boolean written `save=true` becomes `--save`. Names with an underscore accept either spelling: `max_det=50` and `--max-det 50` reach the same option. `libreyolo detect predict ...` is accepted and behaves identically; the task word is stripped before parsing. ## Arguments | Argument | Default | Meaning | |---|---|---| | `source` | | Image path, directory, or URL. Required | | `model` | `yolox-s` | Model name or path | | `conf` | `0.25` | Confidence threshold | | `iou` | `0.45` | NMS IoU threshold | | `imgsz` | | Input image size: `640` (square) or `480x640` (HxW). The model's own input size when unset | | `classes` | | Filter by class IDs, e.g. `[0,2,5]`. A bare integer is accepted | | `max_det` | `300` | Max detections per image | | `half` | `false` | FP16 inference (CUDA only, requires model support) | | `save` | `false` | Save annotated images | | `batch` | `1` | Images per forward pass for directory sources. Above 1 runs true batched inference on models that support it | | `stream` | `false` | Yield results incrementally. Turned on automatically for webcams and live streams | | `stream_buffer` | `false` | Buffer every live frame instead of keeping only the newest | | `vid_stride` | `1` | Process every N-th video or live frame | | `show` | `false` | Display video and live results; `q` stops | | `tiling` | `false` | Tiled inference for large images | | `overlap_ratio` | `0.2` | Tile overlap ratio | | `output_path` | | Explicit output path. Otherwise `project/name` when `save=true` | | `color_format` | `auto` | Input color: `auto`, `rgb`, `bgr` | | `output_file_format` | | Output format: `jpg`, `png`, `webp` | | `device` | `auto` | Device: `0`, `cpu`, `mps`, `auto` | | `face_detector` | | Face detector model (path or CLI name). Required for gaze models | | `gallery` | | Face gallery `.npz` from `libreyolo enroll` to identify faces against. Face-embedding models only | | `gallery_threshold` | `0.4` | Cosine threshold for a gallery identity match | | `project` | `runs/detect` | Output directory root | | `name` | `predict` | Experiment name | | `exist_ok` | `false` | Reuse existing output directory | | `json` | `false` | JSON output to stdout | | `quiet` | `false` | Suppress stderr | | `verbose` | `false` | Verbose stderr output | | `help_json` | `false` | Dump command schema as JSON and exit | ## Examples **Basic** ```bash libreyolo predict model=LibreYOLO9s.pt \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` **Save annotated images** ```bash libreyolo predict model=LibreYOLO9s.pt save=true \ project=runs/detect name=parkour exist_ok=true \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` **Filtered classes, JSON on stdout** ```bash # class 0 is person in the COCO class list the checkpoint ships with. libreyolo predict model=LibreYOLO9s.pt classes="[0]" conf=0.4 max_det=50 \ json=true quiet=true \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` ## Notes An exported artifact loads the same way a checkpoint does, so `model=weights/LibreYOLO9s.onnx` and `model=weights/LibreYOLO9s.engine` are valid values for `model`. Three options are refused on those runtimes rather than ignored: `tiling`, `overlap_ratio` and `output_file_format` exit with `config_unsupported` when a runtime backend cannot honor them. `half` goes the other way. Exported runtimes receive it and run in FP16; native PyTorch inference logs that it was ignored and continues in FP32. Gaze models are two stage and have no detector of their own, so `face_detector` is required for them. `gallery` applies only to models whose task is `embed`; passing it to anything else exits with `config_unsupported`. stdout carries results and nothing else; progress, warnings and errors go to stderr. `json=true` prints one JSON object per invocation, or one per frame when streaming, each carrying `schema_version`. `quiet=true` silences stderr. Both together give a machine reader a clean stdout stream. The exit code is `0` on success, `2` for a usage or configuration error, `3` when the source cannot be found, `4` when the model cannot be loaded, and `1` for other runtime failures. `help_json=true` prints the command's parameters, types, defaults and flags as JSON without running anything, which is the reliable way to read this table back from an installed version. Related: [`libreyolo val`](/docs/cli/val) for measured metrics on a dataset, [`libreyolo export`](/docs/cli/export) to produce the runtime artifacts named above. --- # libreyolo profile A command group that measures where time goes in a training step or an inference call, writes a self-contained profile, and reads that profile back through several lenses. Verified against LibreYOLO v1.5.0. ## Synopsis ```bash libreyolo profile [] [--flag value ...] ``` This group does not take `key=value` arguments. Its subcommands use positional arguments and POSIX flags, so it is `--weights LibreYOLO9t.pt`, not `weights=LibreYOLO9t.pt`. Running `libreyolo profile` with no subcommand prints the list. Two subcommands measure and write a profile; the rest read one. `run` and `infer` both emit the same self-contained `profile.json`, so every reading subcommand works on either. ## profile run Runs a short profiled training and writes a profile. ```bash libreyolo profile run [--flag value ...] ``` | Argument | Default | Meaning | |---|---|---| | `data` | | Positional. Dataset YAML or name, e.g. `coco128`. Required | | `--weights` | `LibreYOLO9t.pt` | Model weights file or name | | `--size` | `t` | Model size variant | | `--batch` | `16` | Micro-batch. `-1` auto-fits about 70% of VRAM | | `--imgsz` | `640` | Training image size | | `--workers` | `8` | Dataloader workers | | `--amp` | `true` | Use the family's AMP path. `--no-amp` disables it | | `--steps` | `20` | Profiled, that is measured, steps | | `--warmup` | `5` | Warmup steps before measuring | | `--repeat` | `1` | Repeat N times for a mean and standard deviation | | `--device` | `0` | Device | | `--project` | `runs/profile` | Output directory root | | `--json` | `false` | JSON output to stdout | The measured window is `--warmup` plus `--steps` iterations. A dataset too small to fill it produces no profile and the command exits with code `3`, naming the three ways out: a larger dataset, fewer steps, or a smaller batch. `--repeat` above 1 writes an aggregated `runs/profile/profile_repeat.json` whose scalar metrics are averaged across trials, while the kernel lists come from the final trial. It is also the prerequisite for a significance verdict in `compare`: a single run cannot supply one. ## profile infer Profiles the inference path and writes a profile. ```bash libreyolo profile infer [] [--flag value ...] ``` | Argument | Default | Meaning | |---|---|---| | `source` | | Positional. Image or directory. The bundled sample image when omitted | | `--weights` | `LibreYOLO9t.pt` | Model weights file or name | | `--size` | `t` | Model size variant | | `--batch` | `1` | Images per forward pass | | `--imgsz` | `640` | Input image size | | `--half` | `false` | Autocast forward, CUDA only. `--no-half` disables it | | `--amp-dtype` | `float16` | CUDA autocast dtype: `float16` or `bfloat16` | | `--warmup` | `20` | Warmup iterations before measuring | | `--runs` | `100` | Measured iterations | | `--repeat` | `1` | Repeat N times for a mean and standard deviation | | `--conf` | `0.25` | Confidence threshold, which changes how much work NMS does | | `--iou` | `0.45` | NMS IoU threshold | | `--max-det` | `300` | Max detections per image, which changes how much work NMS does | | `--device` | `0` | Device | | `--trace` | `true` | Emit a Chrome trace for kernel and op drill-down. `--no-trace` skips it | | `--project` | `runs/profile` | Output directory root | | `--json` | `false` | JSON output to stdout | Reports latency at p50, p90 and p99, throughput in images per second, and the stage split across preprocess, forward and postprocess. The three threshold arguments are here because they move the postprocess number. ## profile summary ```bash libreyolo profile summary [--json] ``` | Argument | Default | Meaning | |---|---|---| | `trace` | | Positional. Path to a `profile.json` or `profile_trace.json`. Required | | `--json` | `false` | JSON output to stdout | The high-level read: step time, throughput, GPU utilization, Tensor Core share, peak VRAM, host overhead, kernel launches per step, the bottleneck verdict with its reason, the kernel mix by category, and the top kernels per step. On an inference profile it also prints the latency percentiles and the stage split. A profile taken under VRAM thrash is marked, because utilization and throughput measured there cannot be trusted. ## profile get ```bash libreyolo profile get [] [--json] ``` | Argument | Default | Meaning | |---|---|---| | `trace` | | Positional. Path to a profile. Required | | `field` | | Positional. Metric name. Omit to list the available metrics | | `--json` | `false` | JSON output to stdout | Prints one metric and nothing else, for scripted loops. An unknown field exits with code `2` and points at the listing form. ## profile phases ```bash libreyolo profile phases [--json] ``` | Argument | Default | Meaning | |---|---|---| | `trace` | | Positional. Path to a profile. Required | | `--json` | `false` | JSON output to stdout | GPU milliseconds, wall milliseconds, kernel count and op count per phase: forward, backward, dataload, to_device, optimizer. ## profile kernels ```bash libreyolo profile kernels [--flag value ...] ``` | Argument | Default | Meaning | |---|---|---| | `trace` | | Positional. Path to a profile. Required | | `--top` | `20` | Show top N by GPU time | | `--category` | | Filter by category substring: `gemm`, `layout`, `norm`, `elementwise` | | `--grep` | | Filter by kernel-name regular expression | | `--tensorcore` | `false` | Only Tensor Core kernels | | `--sort` | `time` | `time`, `count` or `name` | | `--phase` | | Restrict to one phase: `forward`, `backward`, `dataload`, `to_device`, `optimizer` | | `--json` | `false` | JSON output to stdout | The bottom of the analysis: individual GPU kernels with their share of GPU time, milliseconds per step, invocations per step and category. An unknown `--phase` exits with code `2` and lists the phases the profile has. ## profile ops ```bash libreyolo profile ops [--flag value ...] ``` | Argument | Default | Meaning | |---|---|---| | `trace` | | Positional. Path to a profile. Required | | `--top` | `20` | Show top N by CPU time | | `--phase` | | Restrict to one phase | | `--json` | `false` | JSON output to stdout | The framework view rather than the device view: `aten` and autograd ops ranked by CPU time, which is where host-launch cost shows up. ## profile compare ```bash libreyolo profile compare [--json] ``` | Argument | Default | Meaning | |---|---|---| | `before` | | Positional. Baseline profile. Required | | `after` | | Positional. New profile. Required | | `--json` | `false` | JSON output to stdout | Diffs throughput, milliseconds per image, GPU utilization, host overhead, kernel launches per step and the bottleneck verdict. The significance call needs both sides measured with `--repeat` of at least 2. Given that, a difference counts as significant when it exceeds twice the combined standard error, and the output prints the comparison it made. Without it, the line reads that a single run cannot support the call. ## profile what-if ```bash libreyolo profile what-if [--flag value ...] ``` | Argument | Default | Meaning | |---|---|---| | `trace` | | Positional. Path to a profile. Required | | `--remove-category` | | Project removing a kernel category: `gemm`, `layout`, `norm`, `elementwise` | | `--remove-launches` | | Project removing N kernel launches per step, for example an op-fusion win | | `--json` | `false` | JSON output to stdout | Estimates what a change would buy before the change is written. One of the two options is required; neither exits with code `2`. The projection follows the profile's own verdict. Below 80% GPU utilization it models the saving as fewer launches times the measured per-launch host cost; above it, as less GPU work. The result carries a caveat field, because the per-launch cost is an approximation and the only proof is a second measurement. ## Examples **Measure inference** ```bash # No source argument means the bundled sample image. libreyolo profile infer --device cpu --warmup 5 --runs 20 ``` **Read the verdict** ```bash libreyolo profile summary runs/profile/infer/profile.json ``` **Compare two measurements** ```bash libreyolo profile infer --device cpu --warmup 5 --runs 20 --project runs/profile/a libreyolo profile infer --device cpu --warmup 5 --runs 20 --batch 4 --project runs/profile/b libreyolo profile compare runs/profile/a/infer/profile.json \ runs/profile/b/infer/profile.json ``` ## Notes The profiler measures and reports. It changes nothing: reading the verdict, editing the configuration or the code, re-running, and comparing is the loop it is built for. `--device` defaults to `0`, which is CUDA device 0. Passing `--device cpu` measures on the CPU and produces a profile the reading subcommands still accept, without the GPU kernel detail. Every subcommand supports `--json`, and the reading ones print to stdout only, which is what makes the group usable from a script. Exit codes here are the group's own: `2` for a file that does not exist or an argument that does not resolve, `3` when `run` produced no profile, and `1` when a trace cannot be analyzed. Related: [`libreyolo train`](/docs/cli/train), whose arguments are what a training profile is usually taken to tune. --- # libreyolo quantize Replaces a model's float modules with quantized ones, calibrates them on unlabeled images where the recipe needs statistics, and saves the result as a PyTorch checkpoint. Verified against LibreYOLO v1.5.0. ## Synopsis ```bash libreyolo quantize model= [recipe=] [key=value ...] ``` Arguments are `key=value` pairs, and POSIX form works too, so `recipe=int8` and `--recipe int8` are the same argument. ## Arguments | Argument | Default | Meaning | |---|---|---| | `model` | | Model weights `.pt`. Required | | `recipe` | `int8` | Quantization recipe: `fp16`, `bf16`, `fp8`, `int8`, `w4a16`, `w4a8`, `nvfp4`, `mxfp4`, `int2` | | `calib` | `coco128.yaml` | Calibration images: a data YAML or a built-in dataset name. Unlabeled, forward only. `none` skips calibration | | `samples` | `128` | Maximum calibration images | | `batch` | `8` | Calibration batch size | | `algorithm` | `auto` | Activation range estimation: `auto`, which selects minmax, or `minmax`, or `percentile` | | `out` | | Output checkpoint path. Defaults to the source path with `-` before the suffix | | `device` | `auto` | Device | | `allow_download_scripts` | `false` | Allow embedded Python in dataset YAML download blocks | | `json` | `false` | JSON output to stdout | | `quiet` | `false` | Suppress stderr | | `help_json` | `false` | Dump command schema as JSON and exit | ## Examples **Basic** ```bash # Calibrates on coco128 and writes LibreYOLO9s-int8.pt libreyolo quantize model=LibreYOLO9s.pt recipe=int8 ``` **Cast only, no calibration** ```bash libreyolo quantize model=LibreYOLO9s.pt recipe=fp16 calib=none \ out=weights/LibreYOLO9s-fp16.pt ``` **Wider calibration, then heal** ```bash libreyolo quantize model=LibreYOLO9s.pt recipe=int8 \ calib=coco128.yaml samples=256 batch=16 algorithm=minmax # Quantization-aware training on the quantized checkpoint recovers accuracy. libreyolo train model=LibreYOLO9s-int8.pt data=coco8.yaml epochs=10 lr0=0.001 ``` ## Notes ### Which families accept it Quantization covers four families: `yolo9`, `rfdetr`, `birefnet` and `feynobg`. Any other family exits with `quantize_failed` carrying the list. ### What each recipe touches `fp16` and `bf16` are casts. They change dtype only, need no calibration, and `calib=none` is the right setting for them. `int8` and `fp8` quantize `Conv2d` and `Linear` modules, which is why they suit the convolutional families. `w4a16`, `w4a8`, `nvfp4`, `mxfp4` and `int2` quantize `nn.Linear` only, so they target the transformer families. Asking for one of them on `yolo9` is refused with an explanation rather than silently producing an unquantized model, since sub-8-bit acceleration there is GEMM only and the convolutions would stay in higher precision. `int8`, `fp8`, `w4a8` and `int2` need calibration statistics for their activations. `int2` also needs training to heal afterwards, so it is refused on `birefnet` and `feynobg`, which have no trainer. Each family keeps a set of modules in float regardless of recipe: first layers, prediction heads, and on YOLOv9 the DFL convolution, which is a fixed integral expectation operator that must not be quantized. ### Calibration data is not training data `calib` points at a small unlabeled image set, used forward only, to derive activation ranges. It is not evaluated against and its labels are never read. The default `coco128.yaml` downloads on first use from a URL, so it needs no extra permission; a YAML with an embedded Python download script needs `allow_download_scripts=true`. `algorithm=percentile` is available and can reduce accuracy on transformer families, which is why `auto` selects minmax. ### Recovering accuracy The output is a normal PyTorch checkpoint, so [`libreyolo train`](/docs/cli/train) accepts it directly. Training a quantized checkpoint is quantization-aware training; adding `distill_model=` makes it quantization-aware distillation. ### Output and exit codes The result prints the saved path, the recipe, the execution mode, whether calibration ran, and the count of modules swapped per kind. The exit code is `0` on success, `4` when the model cannot be loaded, `5` when quantization or the save fails, and `1` for other runtime failures. Related: [`libreyolo export`](/docs/cli/export), which leaves PyTorch and writes a deployment artifact instead. --- # libreyolo train Trains one model on one dataset and writes checkpoints, metrics and logs into a run directory. Every argument below has a default from the command definition, which a model family's own training config may replace. Verified against LibreYOLO v1.5.0. ## Synopsis ```bash libreyolo train data= [model=] [key=value ...] ``` Arguments are `key=value` pairs, and POSIX form works too, so `epochs=50` and `--epochs 50` are the same argument. Booleans accept `true` and `false`: `amp=false` becomes `--no-amp` where the flag has a negative form. ## Arguments ### Model and data | Argument | Default | Meaning | |---|---|---| | `data` | | Path to dataset YAML (YOLO format, e.g. `coco8.yaml`). Required | | `model` | `yolox-s` | Model name or path to weights | | `task` | | Explicit task override: `detect`, `segment`, `semantic`, `pose`, `classify`, `gaze`, `obb`, `point`, `depth` | | `pretrained` | `true` | Use pretrained weights. `false` builds the architecture and trains from scratch | | `allow_download_scripts` | `false` | Allow embedded Python in dataset YAML download blocks | ### Training loop | Argument | Default | Meaning | |---|---|---| | `epochs` | `300` | Training epochs | | `batch` | `16` | Batch size per device | | `imgsz` | `640` | Training image size: `640` (square) or `480x640` (HxW) | | `device` | `auto` | Device: `0`, `cpu`, `mps`, `auto` | | `workers` | `4` | Dataloader workers | | `cache` | `false` | Cache images to speed dataloading: `ram`, `disk`, `true`, `false` | | `seed` | `0` | Random seed | | `resume` | | Resume training: `true`, or a path to a checkpoint | | `amp` | `true` | Automatic Mixed Precision | | `amp_dtype` | `float16` | CUDA AMP dtype: `float16` or `bfloat16` | | `cuda_graph` | `false` | Capture the training forward and backward into CUDA graphs. Single GPU, supported families only; the rest run eager | | `lora` | `false` | LoRA fine-tuning, for the transformer families listed under Notes | | `freeze` | | Freeze layers: an integer count, a list of indices, or module names | ### Distillation | Argument | Default | Meaning | |---|---|---| | `distill_model` | | Teacher: a detector checkpoint, or a foundation-teacher id such as `dinov2` for backbone feature distillation | | `dis` | | Distillation loss weight. The published default for the loss type when unset | | `distill_loss_type` | `mgd` | Feature loss for detector teachers: `mgd`, `cwd`. Foundation teachers always use `feat_mse` | ### Optimizer | Argument | Default | Meaning | |---|---|---| | `optimizer` | `sgd` | Optimizer: `sgd`, `adam`, `adamw` | | `lr0` | `0.01` | Initial learning rate | | `momentum` | `0.937` | SGD momentum, and the first-moment coefficient for the Adam optimizers | | `weight_decay` | `0.0005` | L2 regularization | | `nesterov` | `true` | Nesterov momentum | ### Scheduler | Argument | Default | Meaning | |---|---|---| | `scheduler` | `yoloxwarmcos` | LR schedule type | | `warmup_epochs` | `5` | Warmup duration | | `warmup_lr_start` | `0.0` | Initial warmup LR | | `min_lr_ratio` | `0.05` | Minimum LR ratio | | `lr_drop` | `100` | RF-DETR step LR drop epoch | ### Augmentation | Argument | Default | Meaning | |---|---|---| | `mosaic` | `1.0` | Mosaic probability | | `mixup` | `1.0` | Mixup probability | | `hsv_prob` | `1.0` | HSV jitter probability | | `flip_prob` | `0.5` | Horizontal flip probability | | `degrees` | `10.0` | Rotation range, plus and minus, in degrees | | `translate` | `0.1` | Translation ratio | | `shear` | `2.0` | Shear angle | | `mosaic_scale` | `(0.1,2.0)` | Mosaic scale range | | `mixup_scale` | `(0.5,1.5)` | Mixup scale range | | `no_aug_epochs` | `15` | Disable augmentation for the final N epochs | ### EMA | Argument | Default | Meaning | |---|---|---| | `ema` | `true` | Exponential Moving Average | | `ema_decay` | `0.9998` | EMA decay factor | ### Validation during training | Argument | Default | Meaning | |---|---|---| | `val` | `true` | Validate during training | | `eval_interval` | `10` | Validate every N epochs | | `max_det` | `300` | Maximum predictions per image after validation NMS | | `eval_max_det` | | COCO evaluator cap. The pycocotools AP@100 convention when unset | | `faster_coco_eval` | `true` | Use the faster-coco-eval C++ backend for COCO metrics when installed; falls back to pycocotools | | `save_plots` | `false` | Save final validation plots during training | | `patience` | `50` | Early stopping patience. `0` disables it | ### Output | Argument | Default | Meaning | |---|---|---| | `project` | `runs/train` | Output directory root | | `name` | `exp` | Experiment name | | `exist_ok` | `false` | Reuse existing output directory | | `save_period` | `10` | Save checkpoint every N epochs | | `log_interval` | `10` | Log loss every N batches | ### Agent flags | Argument | Default | Meaning | |---|---|---| | `json` | `false` | JSON output to stdout | | `quiet` | `false` | Suppress stderr | | `dry_run` | `false` | Resolve and print the config without executing | | `help_json` | `false` | Dump command schema as JSON and exit | ## Examples **Basic** ```bash # coco8.yaml ships with the package and downloads its 8 images on first use. libreyolo train model=LibreYOLO9s.pt data=coco8.yaml epochs=10 imgsz=640 batch=8 ``` **Check the resolved config first** ```bash # Prints what the run would use, including family defaults, and exits # without training or loading data. libreyolo train model=LibreDFINEn.pt data=coco8.yaml epochs=10 dry_run=true ``` **Named run with an explicit recipe** ```bash libreyolo train model=LibreYOLO9s.pt data=coco8.yaml \ epochs=50 batch=8 optimizer=adamw lr0=0.001 weight_decay=0.0001 \ patience=20 save_period=5 project=runs/train name=yolo9s-coco8 exist_ok=true ``` ## Notes ### The defaults above are not always the values used Every model family carries its own training config, and where that config differs from the base one, its value replaces the command default for any argument you did not set explicitly. Setting the argument yourself always wins. `libreyolo cfg` prints the base defaults and the per-family overrides, which is the way to see what a given family will actually use. `imgsz` is the argument this matters most for. The command default is `640`, which is not every checkpoint's native input: the published RF-DETR detection sizes are 384, 512, 576 and 704, and the YOLOX `n` and `t` checkpoints are 416. RF-DETR and DEIMv2 are handled by only forwarding `imgsz` when it was set explicitly, so their own size stays in force otherwise. Other families are handed the value as given and train at it. FOMO is the strict one: each size accepts only its native input (96, 192 and 224), so a FOMO run needs `imgsz` set to match or it stops with an error. RF-DETR also requires the value to divide by its patch size times its window count, and reports the two nearest legal sizes when it does not. ### Arguments a family ignores Not every family reads every argument, and the augmentation ones are where that shows. RF-DETR, D-FINE, DEIM, DEIMv2, RT-DETRv4 and DINOv2 train through pass-through pipelines with no mosaic, no mixup and no affine warp, so `mosaic`, `mixup`, `hsv_prob`, `degrees`, `translate`, `shear`, `mosaic_scale` and `mixup_scale` reach nothing there. EC shares that pipeline but does read `hsv_prob`, `degrees` and `translate` when its task is pose. The classification families, SegFormer and NAFNet ignore that whole set and `flip_prob` with it, because their flip runs at a fixed probability rather than a configurable one. YOLO-NAS ignores `mosaic` alone, since it augments with an always-on per-sample affine instead. RF-DETR ignores three more on top of that list: `optimizer`, `momentum` and `nesterov`. Setting one of these is not an error. The run logs a line to stderr naming the family and the arguments it will ignore, then trains, and that line is the authoritative list for the version installed. It is also the only signal, so a scripted run with `quiet=true` suppresses the warning along with everything else on stderr. `val=false` is a related case. It sets `eval_interval` to `0` for most families; RF-DETR cannot disable validation that way and logs that it ignored the request. ### Other behavior worth knowing `lora=true` is accepted by RF-DETR, D-FINE, DEIM, DEIMv2, RT-DETR v1, v2 and v4, EC and ConvNeXt. Any other family exits with `config_unsupported` rather than training without it. `pretrained=false` combined with `resume` is refused for the families that support scratch training, since the two ask for opposite things. `mosaic` and `mixup` are the command-line spellings of the `mosaic_prob` and `mixup_prob` config fields. On families whose mixup only applies to mosaic samples, `mixup` above zero with `mosaic` at zero never fires, and the run says so. `dry_run=true` resolves the model reference, applies family defaults, and prints the config it would train with. It does not load the dataset, so it is the cheap way to confirm an argument reached the value you expected. stdout carries the final result object; progress and warnings go to stderr. The exit code is `0` on success, `2` for a usage or configuration error, `3` when the dataset cannot be found or read, `4` when the model cannot be loaded, and `1` for other runtime failures. Related: [`libreyolo doctor`](/docs/cli/doctor) to check a dataset before committing to a run, [`libreyolo monitor`](/docs/cli/monitor) to watch a run in the browser, [`libreyolo val`](/docs/cli/val) to measure the result. --- # libreyolo ui Starts a local web server that accepts dropped or pasted images, runs a chosen model on them, and shows the results in the browser. Verified against LibreYOLO v1.5.0. ## Synopsis ```bash libreyolo ui [key=value ...] ``` Arguments are `key=value` pairs, and POSIX form works too, so `port=9000` and `--port 9000` are the same argument. ## Arguments | Argument | Default | Meaning | |---|---|---| | `host` | `127.0.0.1` | Host or interface to bind | | `port` | `8000` | Port to bind. Bumps to the next free one if taken | | `device` | `auto` | Device: `0`, `cpu`, `mps`, `auto` | | `no_browser` | `false` | Do not auto-open the browser | | `json` | `false` | JSON output to stdout | | `quiet` | `false` | Suppress stderr | | `verbose` | `false` | Verbose stderr output | ## Examples **Basic** ```bash libreyolo ui ``` **Fixed port, no browser** ```bash libreyolo ui port=9000 no_browser=true ``` **On the CPU, machine readable** ```bash libreyolo ui device=cpu json=true ``` ## Notes The default bind is loopback, so the UI is reachable from this machine only. If the requested port is in use, the command tries the next one and keeps going up to twenty ports past the request. Failing all twenty exits with `io_error` and the suggestion to pass a different port. The URL printed on stdout is the port that was actually bound, so read it rather than assuming the one you asked for. Unless `no_browser=true`, a browser tab opens at that URL shortly after the bind. The command then serves in the foreground until Ctrl+C, which shuts the server down cleanly. There is no detached mode; background it with your shell if you want the terminal back. `json=true` prints the URL and device as one object with `schema_version` before the server starts, which is how a script picks up the bound port. Related: [`libreyolo label`](/docs/cli/label) for drawing boxes and saving labels, [`libreyolo monitor`](/docs/cli/monitor) for watching training runs. Both are local web servers with the same port and browser behavior. --- # libreyolo utilities Nine commands that report or inspect rather than compute. They print environment facts, the model and format inventory, resolved defaults, checkpoint details, and they build and query a face gallery. Verified against LibreYOLO v1.5.0. ## Synopsis ```bash libreyolo [key=value ...] ``` Arguments are `key=value` pairs, and POSIX form works too, so `model=x` and `--model x` are the same argument. Every command here writes results to stdout and accepts `json=true` and `quiet=true`. The root command carries one flag of its own, `libreyolo --version`, which prints the version string and exits. That is a smaller output than the `version` command below. ## version Prints the LibreYOLO version plus the Python, torch and CUDA versions it is running against. ```bash libreyolo version ``` | Argument | Default | Meaning | |---|---|---| | `json` | `false` | JSON output to stdout | | `quiet` | `false` | Suppress stderr | ## checks Prints the environment in more detail: Python, torch, CUDA, cuDNN, every detected GPU with its name and memory, and the installed version of each optional package the export paths use. ```bash libreyolo checks ``` | Argument | Default | Meaning | |---|---|---| | `json` | `false` | JSON output to stdout | | `quiet` | `false` | Suppress stderr | The package list covers `onnx`, `onnxruntime`, `tensorrt`, `openvino`, `paddlepaddle`, `x2paddle`, `mnn`, `ncnn`, `onnx2tf`, `ai-edge-litert`, `transformers` and `scipy`. A package that is not installed reports as such rather than being omitted, so a failed export can be traced to a missing dependency from this one command. ## models Lists every model family with its tasks, sizes, the CLI names that resolve to its checkpoints, and each size's input resolution. ```bash libreyolo models ``` | Argument | Default | Meaning | |---|---|---| | `json` | `false` | JSON output to stdout | | `quiet` | `false` | Suppress stderr | A family whose optional dependency is not installed is listed as unavailable together with the `pip install` line that would make it available. The CLI names are what `model=` accepts as a shorthand: `yolox-s` resolves to `LibreYOLOXs.pt`, and non-detection tasks carry their task suffix. ## formats Lists the export formats the installed environment can produce, with each format's file extension and whether it supports FP16 and INT8. ```bash libreyolo formats [family=] [task=] ``` | Argument | Default | Meaning | |---|---|---| | `family` | | Show tiers for one model family. `model=` is accepted as the same option | | `task` | | Canonical model task. The family's default task when unset | | `json` | `false` | JSON output to stdout | | `quiet` | `false` | Suppress stderr | Without `family`, the output is the format inventory alone. With it, each format gains the support tier for that family and task, the reason behind the tier, and any constraint attached to it. An unknown family, or a task the family does not support, is a usage error. Format aliases appear next to their canonical name: `engine` for `tensorrt`, `litert` for `tflite`. ## cfg Prints the resolved default configuration: the train defaults, the validation defaults, the prediction defaults, and the per-family overrides. ```bash libreyolo cfg ``` | Argument | Default | Meaning | |---|---|---| | `json` | `false` | JSON output to stdout | | `quiet` | `false` | Suppress stderr | The values are read from the configuration dataclasses, not from a copy, so this is the authority on what a training run will use when you do not pass an argument. `family_overrides` is the section that answers why a family trained at settings you did not ask for. See [`libreyolo train`](/docs/cli/train) for how those overrides are applied. ## info Loads a model on the CPU and reports its family, size, parameter count, classes, and the export tier for each format. ```bash libreyolo info model= ``` | Argument | Default | Meaning | |---|---|---| | `model` | | Model name or path to weights. Required | | `detailed` | `false` | Include per-parameter details | | `json` | `false` | JSON output to stdout | | `quiet` | `false` | Suppress stderr | ## metadata Reads a checkpoint's metadata without constructing a model, and validates it against the LibreYOLO checkpoint schema. ```bash libreyolo metadata path= ``` | Argument | Default | Meaning | |---|---|---| | `path` | | Path to a `.pt` checkpoint. Required | | `json` | `false` | JSON output to stdout | | `quiet` | `false` | Suppress stderr | Large tensor-bearing entries are summarized rather than printed, so the output stays readable on a full training checkpoint. A checkpoint that does not exist exits with `checkpoint_not_found`, and one whose metadata fails validation prints the errors and exits `1`. ## enroll Builds a face gallery from a folder-per-person tree, so later predictions can name the faces they find. ```bash libreyolo enroll model= source= gallery= ``` | Argument | Default | Meaning | |---|---|---| | `model` | | Face-embedding model, path or name. Required | | `source` | | Folder-per-person tree, `source//*.jpg`. Required | | `gallery` | | Output gallery file `.npz`. Extended in place if it exists. Required | | `face_detector` | | Face detector: a YuNet `.onnx` or a LibreYOLO detector. The family's default detector when unset | | `device` | `auto` | Device: `0`, `cpu`, `mps`, `auto` | | `json` | `false` | JSON output to stdout | | `quiet` | `false` | Suppress stderr | ```bash # people/ holds one folder per identity; the folder name becomes the identity. libreyolo enroll model=librefacerec-l.onnx source=people/ gallery=people.npz ``` The subfolder name is the identity. A reference image with no detectable face is skipped with a line on stderr and the rest continue; a source with no identity subfolders, or one where no face was found at all, is an error. Pass the resulting file to [`libreyolo predict`](/docs/cli/predict) as `gallery=people.npz` to have detections carry an identity and a match score. ## compare Reports the cosine similarity between two face images and whether it clears the same-identity threshold. ```bash libreyolo compare model= source= source2= ``` | Argument | Default | Meaning | |---|---|---| | `model` | | Face-embedding model, path or name. Required | | `source` | | First image. Required | | `source2` | | Second image to compare against. Required | | `face_detector` | | Face detector: a YuNet `.onnx` or a LibreYOLO detector | | `threshold` | `0.4` | Cosine-similarity threshold for the same-identity decision | | `device` | `auto` | Device: `0`, `cpu`, `mps`, `auto` | | `json` | `false` | JSON output to stdout | | `quiet` | `false` | Suppress stderr | ```bash libreyolo compare model=librefacerec-l.onnx source=a.jpg source2=b.jpg ``` `libreyolo verify` is registered as a second name for this command and takes the same arguments. Both `compare` and `enroll` need a model whose task is face embedding. Anything else exits with `config_unsupported`. Local image paths and `http` or `https` URLs are both accepted as sources. ## Examples **Environment** ```bash libreyolo version libreyolo checks ``` **What is available** ```bash libreyolo models libreyolo formats family=yolo9 task=detect ``` **Inspect a checkpoint** ```bash libreyolo info model=LibreYOLO9s.pt libreyolo metadata path=weights/LibreYOLO9s.pt ``` ## Notes stdout carries the result; progress and warnings go to stderr. `json=true` prints one object with `schema_version`, which is the form to read from a script. Text output is the default and is meant to be read by a person. Exit codes follow the same map as the rest of the CLI: `0` on success, `2` for a usage or configuration error, `3` when a source cannot be found, `4` when a model or checkpoint cannot be loaded, and `1` for other runtime failures. Related: [`libreyolo doctor`](/docs/cli/doctor), which is the dataset-side inspection command, and [`libreyolo profile`](/docs/cli/profile), the performance-side one. --- # libreyolo val Evaluates one model against one dataset split and prints the metrics. The metric set depends on the model's task, and the numbers are the ones a benchmark row is built from. Verified against LibreYOLO v1.5.0. ## Synopsis ```bash libreyolo val model= data= [key=value ...] ``` Arguments are `key=value` pairs, and POSIX form works too, so `batch=8` and `--batch 8` are the same argument. ## Arguments | Argument | Default | Meaning | |---|---|---| | `model` | | Model weights path or CLI name. Required | | `data` | | Path to dataset YAML (YOLO format, e.g. `coco8.yaml`). Required | | `data_dir` | | Direct dataset directory, bypassing the path in the YAML | | `split` | `val` | Dataset split: `val`, `test`, `train` | | `batch` | `16` | Batch size | | `imgsz` | | Image size: `640` (square) or `480x640` (HxW). The model's own input size when unset | | `conf` | `0.001` | Confidence threshold | | `iou` | `0.6` | NMS IoU threshold | | `max_det` | `300` | Max predictions per image after NMS | | `eval_max_det` | | COCO evaluator cap. The pycocotools AP@100 convention when unset | | `faster_coco_eval` | `true` | Use the faster-coco-eval C++ backend for COCO metrics when installed; falls back to pycocotools | | `half` | `false` | FP16 inference | | `amp_dtype` | `float16` | CUDA autocast dtype when `half=true`: `float16` or `bfloat16` | | `save_json` | `false` | Save COCO-format JSON results | | `save_plots` | `false` | Save validation plots: metrics, per-class AP, confusion matrix, samples | | `workers` | `4` | Dataloader workers | | `device` | `auto` | Device | | `project` | `runs/val` | Output directory root | | `name` | `exp` | Experiment name | | `exist_ok` | `false` | Reuse output directory | | `allow_download_scripts` | `false` | Allow embedded Python in dataset YAML download blocks | | `json` | `false` | JSON output to stdout | | `quiet` | `false` | Suppress stderr | | `verbose` | `true` | Verbose output | | `help_json` | `false` | Dump command schema as JSON and exit | ## Examples **Basic** ```bash libreyolo val model=LibreYOLO9s.pt data=coco8.yaml ``` **Plots and COCO JSON** ```bash libreyolo val model=LibreYOLO9s.pt data=coco8.yaml \ imgsz=640 batch=8 save_json=true save_plots=true \ project=runs/val name=yolo9s-coco8 exist_ok=true ``` **Machine readable** ```bash libreyolo val model=LibreYOLO9s.pt data=coco8.yaml json=true quiet=true ``` ## Notes ### What the metrics are The printed set follows the model's task, and the JSON output uses the same keys. Detection, segmentation and oriented boxes report `mAP50`, `mAP50_95`, `precision` and `recall`. Where a model predicts more than one output kind, the per-kind groups appear alongside as `box_metrics`, `mask_metrics` and `obb_metrics`, each carrying the same four keys. Classification reports `accuracy_top1` and `accuracy_top5`. Point detection reports `precision`, `recall`, `f1`, `MLE`, `MAE`, `RMSE` and `mAP_sweep`. Depth reports `abs_rel`, `rmse`, `delta1`, `delta2` and `delta3`. Semantic segmentation reports `mIoU` and `pixel_accuracy`. Restoration reports `PSNR` and `SSIM`. The JSON result also carries `eval_backend`, naming the COCO evaluation library and version that produced the numbers, so two runs can be compared knowing whether the same backend scored both. ### Thresholds The defaults here are evaluation defaults, not prediction defaults: `conf` is `0.001` and `iou` is `0.6`, where [`libreyolo predict`](/docs/cli/predict) uses `0.25` and `0.45`. Raising `conf` to a display threshold lowers recall and with it the mAP, so a number produced that way is not comparable to a published one. `imgsz` is unset by default, which means the model's own input size. Setting it evaluates at the size given, which is how a checkpoint gets measured away from its native resolution. ### Datasets that download A dataset YAML whose `download` field is a URL fetches on first use with no extra permission. One that carries an embedded Python download script needs `allow_download_scripts=true`, and the command warns on stderr that local code execution was enabled. The bundled `coco8.yaml` and `coco128.yaml` are URL based, so they need nothing. ### Output and exit codes stdout carries the metrics; progress goes to stderr. `json=true` prints one object with `schema_version`, and `quiet=true` silences stderr. The exit code is `0` on success, `2` for a usage or configuration error, `3` when the dataset cannot be found, `4` when the model cannot be loaded, and `1` for other runtime failures. Related: [`libreyolo train`](/docs/cli/train), which runs this same evaluation on its own schedule through `eval_interval`. --- # Core concepts Four ideas describe every model in LibreYOLO: the task it performs, the family it belongs to, the size within that family, and the support tier the family sits in. The checkpoint filename encodes the first three. Verified against LibreYOLO v1.5.0. ## Tasks A task is what a model returns. LibreYOLO has seventeen canonical task names, and each one names the field on the `Results` object that carries its output. | Task | Returns | |---|---| | `detect` | Axis-aligned boxes with a class and a confidence | | `segment` | Per-instance masks, one mask per detected object | | `semantic` | One class label per pixel, with no instance separation | | `panoptic` | One non-overlapping label per pixel, merging countable things with amorphous stuff | | `pose` | Per-instance keypoints, rows aligned with the boxes | | `classify` | A probability over a label set for the whole image | | `obb` | Oriented boxes, with a rotation angle | | `point` | One image coordinate per detection, rather than a box | | `depth` | A dense relative inverse-depth map | | `normal` | A dense unit-vector surface-normal field | | `edge` | A dense edge-probability map | | `restore` | A restored RGB image, for deblurring, denoising or super-resolution | | `matte` | A soft foreground map from 0 to 1, for background removal | | `ocr` | Text quads with transcripts, in reading order | | `embed` | An L2-normalized vector whose dot product measures agreement | | `gaze` | A gaze direction per detected face | | `mesh` | A posed 3D body per detected person | Those are the names that appear in checkpoint metadata and in filenames. Familiar aliases are accepted wherever a task is passed and normalized before anything else happens: `detection` and `det` become `detect`, `keypoints` becomes `pose`, `cls` becomes `classify`, `deblur`, `denoise` and `super-resolution` all become `restore`, `face-recognition` and `reid` become `embed`. An unrecognized name raises rather than silently defaulting. `segment`, `semantic` and `panoptic` are three different tasks, not three words for one. Instance masks, per-pixel labels and the merged thing-plus-stuff map have different ground truth, different metrics and different result fields. ## Model families A family is one architecture lineage with its own loading, preprocessing and postprocessing code. Every family declares a `FAMILY` identifier such as `yolo9`, `rfdetr` or `dfine`, the tasks it supports, and the input resolution for each size it ships. `LibreYOLO()` is a factory rather than a class. Given a path it loads the file, identifies the family from the checkpoint's metadata or, failing that, from the tensor keys themselves, and returns an instance of that family's model. This is why swapping detectors is a one-line change: the object that comes back exposes the same `predict`, `train`, `val` and `export` surface and returns the same `Results` type. **List families** ```bash # Tasks, sizes and input resolutions for every registered family. libreyolo models ``` **One model** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") print(model.family, model.size, model.task) print(model.input_size) print(model.nb_classes, model.names[0]) ``` **Pick a task** ```python from libreyolo import LibreYOLO # Aliases normalize at the API boundary: "keypoints" resolves to # "pose", "det" to "detect", "semantic-segmentation" to "semantic". model = LibreYOLO("LibreYOLO9t.pt", task="det") print(model.task) ``` A family that serves more than one task usually publishes a separate checkpoint per task, often with a different set of sizes for each; a few share one artifact between two runtime tasks instead. Either way the supported tasks are a fixed list, and asking for one outside it raises with the supported list in the message rather than loading something approximate. The full list, with per-family benchmarks and published weights, is at [all models](/docs/models). ## Sizes A size is a variant within a family, written as a lowercase code attached directly to the family prefix. The common letters are `n` for nano, `t` for tiny, `s` for small, `m` for medium, `l` for large and `x` for xlarge, but the codes are family-specific and several families use something else entirely: backbone-named codes such as `r50` or `r101` where the size is a ResNet depth, compound-scaling codes such as `b0` through `b3`, or a name that identifies the one released checkpoint. YOLOv9 uses `c` for compact where other families use `l`. Size also fixes the input resolution, and for families with several tasks the resolution can differ per task. Both are read from the family, never assumed; `libreyolo models` prints them. ## Checkpoint filenames Every published weight file follows one schema: ```text Libre[-].pt ``` The family prefix is a fixed string per family, the size is lowercase and attached with no separator, and the task suffix is hyphen-prefixed. Detection carries no suffix, following the convention YOLO checkpoints have always used, so `LibreYOLO9t.pt` is a detector and `LibreRFDETRn-seg.pt` is a segmentation model of the same family. | Task | Suffix | |---|---| | `detect` | | | `segment` | `-seg` | | `semantic` | `-sem` | | `panoptic` | `-panoptic` | | `pose` | `-pose` | | `classify` | `-cls` | | `gaze` | `-gaze` | | `obb` | `-obb` | | `point` | `-point` | | `depth` | `-depth` | | `edge` | `-edge` | | `normal` | `-normal` | | `restore` | `-restore` | | `matte` | `-matte` | | `ocr` | `-ocr` | | `embed` | `-embed` | | `mesh` | `-mesh` | A family with no suffixless task can require the suffix, so that a name without one is not accepted as a valid checkpoint for it. A family that publishes weights trained on a dataset other than its default appends the dataset name as a further suffix, and that variant stays part of the repository name the file is downloaded from. Three tiers stand outside this schema. The promptable segmentation families, the vision-language families and the open-vocabulary detectors are not registered into the checkpoint factory and emit no `Libre.pt` file. Their prefix names a downloaded Hugging Face snapshot or a promptable checkpoint instead, and upstream brand casing is preserved there on purpose. ## How the task is decided When several signals could name a task, they are consulted in a fixed order and the first one that is present wins: the `task` argument you passed, then the task recorded in the checkpoint metadata, then the task suffix in the filename, then the family's default task. The result is checked against the family's supported tasks before the model is built, so a mismatch fails at load time rather than producing wrong output later. ## Support tiers Families are enrolled in exactly one tier. A tier is a statement about engineering attention, not about accuracy: it tells you where a new feature lands first and what is kept green. | Tier | What it means | |---|---| | Flagship | Features are designed and fully GPU-validated here first | | Core | Core trainable detectors. Features follow the flagships in the same release wave | | Supported | Supporting trainable families. Kept green in CI, features land opportunistically | | Inference only | Predict, validate and export. Training features do not apply | | Museum | A frozen exhibit. Bug fixes only | | Sibling tier | A separate product surface with its own factory and contract | Each model page carries its family's tier in the header. The two flagship families are [YOLOv9](/docs/models/yolov9) for the CNN detectors and [RF-DETR](/docs/models/rf-detr) for the transformer detectors; start there unless you have a reason not to. Inference only says what is missing, which is a training loop in LibreYOLO. Predict, validate and, where the family supports it, export all work. Calling `train()` on such a family raises `NotImplementedError` naming the reason. --- # Core AI Core AI is Apple's on-device inference stack. LibreYOLO captures the model with torch.export, lowers it through the Core AI converter, and writes a .aimodel asset carrying the model metadata and the exported output names. Verified against LibreYOLO v1.5.0. ## Install This format is macOS only. The `coreai-torch` requirement carries a `sys_platform == 'darwin'` marker, and the toolchain neither converts nor runs anywhere else. **Install, on macOS** ```bash # Kept out of every aggregate extra on purpose: coreai-torch pins torch # to 2.11.x and would drag the whole environment onto that version. pip install "libreyolo[coreai]" ``` The extra sits outside every aggregate extra, including `libreyolo[all]`, because `coreai-torch` pins torch to the 2.11 series. Install it into an environment you are willing to constrain to that pair. ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") # Writes weights/LibreYOLO9t.aimodel path = model.export(format="coreai", imgsz=640) print(path) ``` **CLI** ```bash libreyolo export --model LibreYOLO9t.pt --format coreai --imgsz 640 ``` **Arguments** ```python model.export( format="coreai", imgsz=640, # int, or (height, width); this is the run canvas batch=1, output_path=None, # None writes weights/.aimodel ) # dynamic=True raises NotImplementedError. # half=True and int8=True are rejected during validation. ``` Capture is `torch.export`, a real graph capture with guards, rather than a single recorded trace. That is stricter than the Core ML path: host scalar reads and data-dependent control flow are rejected instead of being silently baked in, which is why a few families are blocked here with a capture failure recorded against them. Three preparation steps run inside a scope that restores the caller's live model whether the export succeeds or fails. Darknet-derived families get their inference batch normalization folded exactly into the preceding convolutions, because Core AI 0.4.1 does not preserve Darknet's epsilon-after-square-root formula. Grid and anchor families get their anchors frozen for the fixed canvas. RF-DETR gets its position embedding rebaked for the requested canvas by re-running the model's own baking path, because the converter has no lowering for `aten._upsample_bicubic2d_aa`. Lowering folds PyTorch's reference decomposition for `aten.grid_sampler_2d` into the decomposition table, since the Core AI converter has no lowering for the deformable-attention sampler the DETR families use. Assets declare a minimum OS of v27, which is the only value the toolchain offers. That gates deployment, not conversion: conversion and Python-side execution work on earlier macOS through the runtime inside the wheel, but numerics differ between OS versions, so recorded parity is measured on macOS 27. ## Run the artifact There is no Core AI entry in `libreyolo/backends`, so `LibreYOLO()` does not load a `.aimodel`. Consumers use the Core AI runtime directly, and preprocessing, decoding, NMS and coordinate rescaling are theirs. A validated row in the support matrix is a claim that the exported graph computes the same numbers as the reference, not that `predict` will run it. The one thing a consumer cannot re-derive is the output ordering: **Read the output ordering before wiring a consumer** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") model.export(format="coreai", imgsz=640) # The asset metadata records the exported output names, in graph order, # under "coreai_output_names". Map Core AI's returned dictionary by name # using that list; never pair it positionally with the eager tuple. ``` Core AI returns a named dictionary whose key order matches neither the eager forward's tuple order nor anything guessable. The exported names are written into the asset metadata as `coreai_output_names` for exactly this reason. Map by name. ## Constraints Fixed canvas, FP32, batch as exported. `dynamic=True` raises `NotImplementedError`, and `half=True` and `int8=True` are rejected during validation. Coverage is wide on the conversion side. Validated combinations include the YOLO9 families, YOLOX, YOLO7, the four Darknet-era detectors, YOLO-NAS, PicoDet, RTMDet, RT-DETR, RT-DETRv2, RT-DETRv4, D-FINE, DEIM, DEIMv2, EC and RF-DETR detection; the four CNN classification families plus frozen-class CLIP and SigLIP2; Depth Anything V2 and ZipDepth; NAFNet and Real-ESRGAN restoration; PIDNet and LingBotVision semantic segmentation; and FOMO point detection. Each carries its own recorded context, which `libreyolo formats` prints. Blocked, with the reason recorded per combination: | Combination | Why | |---|---| | EoMT semantic segmentation | Strict capture fails with `GuardOnDataDependentSymNode`: something in the mask path reads a value off a tensor and branches on it | | SegFormer semantic segmentation | The capture path has not been assessed, and its published weights are non-commercial regardless of format | | L2CS gaze | The model itself supports ONNX, TorchScript, ExecuTorch, TensorRT and OpenVINO only, which is a model-side decision | | Depth Anything 3 depth | The family rejects export for every format | RF-DETR carries one caveat worth reading before comparing artifacts. Its parity is recorded against the graph the Core AI exporter itself prepares, not against ONNX, and at a 640 canvas the RF-DETR ONNX artifact disagrees with that prepared graph. The Core AI rebake preserves the antialiased resize the eager model performs, while the ONNX path disables antialiasing. ONNX is therefore not a valid reference for that family at a non-native canvas. For Apple's earlier format, see [Core ML](/docs/export/coreml). For the full family and task grid, see [the export matrix](/docs/reference/export-matrix). For one combination: **Check one family and task before exporting** ```bash libreyolo formats --family yolo9 --task detect ``` --- # Core ML Core ML is Apple's on-device model format. LibreYOLO traces the detector behind a per-family preprocessing wrapper so the converted graph always takes a canonical RGB image input, then writes an ML Program .mlpackage with the model metadata attached. Verified against LibreYOLO v1.5.0. ## Install **Install** ```bash pip install "libreyolo[coreml]" ``` Prediction needs macOS. `LibreYOLO()` refuses a `.mlpackage` on any other platform with a message naming the current one, and the support matrix records these combinations as available on the grounds that runtime parity needs a macOS runner. ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") # Writes the bundle weights/LibreYOLO9t.mlpackage path = model.export(format="coreml") print(path) ``` **CLI** ```bash libreyolo export --model LibreYOLO9t.pt --format coreml ``` **Arguments** ```python model.export( format="coreml", imgsz=640, batch=1, half=False, # True converts with FLOAT16 compute precision compute_units="all", # all | cpu_and_gpu | cpu_and_ne | cpu_only output_path=None, # None writes weights/.mlpackage ) # dynamic is accepted but the input is a fixed-shape ct.ImageType, # and the embedded metadata records dynamic=False either way. ``` The bundle is written to `weights/` under the checkpoint's stem, with `_fp16` appended when `half=True`. A `.mlpackage` is a directory, so copy the whole tree. Every family is traced behind a preprocessing wrapper, so the converted graph takes one canonical input: RGB, `scale=1/255`, no bias, declared as `ct.ImageType`. The wrapper absorbs the family's own convention, which is BGR in the range 0 to 255 for YOLOX, ImageNet mean and standard deviation for RF-DETR, and identity for YOLO9 and RT-DETR. That is why a Core ML consumer feeds an ordinary image rather than a family-specific tensor. Conversion targets ML Program with a minimum deployment target of iOS 15. `compute_units` is stored on the converted model and can be overridden again when the artifact is loaded. Model metadata goes into `user_defined_metadata` as strings, which is where the backend reads the family, task, class names, input size and pose schema. ### Embedded NMS **Embed Apple's NMS layer** ```python from libreyolo import LibreYOLO # YOLOX and YOLO9 detection only, batch 1. LibreYOLO("LibreYOLO9t.pt").export( format="coreml", nms=True, conf=0.25, iou=0.45, ) ``` **CLI** ```bash libreyolo export --model LibreYOLO9t.pt --format coreml --nms \ --conf 0.25 --iou 0.45 ``` `nms=True` wraps the model in a Core ML pipeline that ends in Apple's `NonMaximumSuppression` layer. The result has two outputs: `confidence`, shaped `N` by the class count, and `coordinates`, shaped `N` by 4 as normalized `xywh`. It applies to YOLOX and YOLO9 detection only, and it requires batch 1. The DETR-style families are refused by name, because set prediction takes a top-k over queries and classes with no IoU step and cannot use that layer. `max_det` is not exposed here either; when the detection cap matters, use [ONNX embedded NMS](/docs/export/onnx) instead. ## Run the artifact **Through LibreYOLO, on macOS** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO( "weights/LibreYOLO9t.mlpackage", compute_units="all", # or cpu_and_ne to pin the Neural Engine ) result = model.predict(SAMPLE_IMAGE) print(result.boxes.xyxy[:3]) ``` **Bare coremltools** ```python import coremltools as ct from PIL import Image mlmodel = ct.models.MLModel("weights/LibreYOLO9t.mlpackage") print(mlmodel.user_defined_metadata["model_family"]) print(mlmodel.user_defined_metadata["names"]) # The input is an image named "image" at the fixed export size. image = Image.open(SAMPLE_IMAGE).convert("RGB").resize((640, 640)) out = mlmodel.predict({"image": image}) print({name: value.shape for name, value in out.items()}) # Letterboxing and postprocessing are yours on this path. ``` `LibreYOLO()` recognizes a directory with the `.mlpackage` suffix and returns the same `Results` object as the checkpoint. `compute_units` is the one argument the factory passes through for this format, and it accepts `all`, `cpu_and_gpu`, `cpu_and_ne` and `cpu_only`. The `device` argument is ignored, because Core ML routes through compute units instead. The second snippet is the bare-runtime path. Letterboxing, decoding, NMS and coordinate rescaling become yours there, and the class names live in `user_defined_metadata`. ## Constraints Four families, detection only: `yolox`, `yolo9`, `rtdetr` and `rfdetr`. Anything else is refused in preflight, because the family-aware preprocessing wrapper is what makes the fixed image input contract correct, and a family outside it would convert with the wrong normalization. The error names ONNX and TorchScript as the alternatives. The input shape is hard-fixed by `ct.ImageType`, so `dynamic=True` changes nothing and the metadata records `dynamic=False`. Export a second bundle for a second resolution. `half=True` converts with FP16 compute precision. There is no INT8 path from this exporter. For the full family and task grid, see [the export matrix](/docs/reference/export-matrix). For Apple's newer on-device format, see [Core AI](/docs/export/coreai). For one combination: **Check one family and task before exporting** ```bash libreyolo formats --family yolo9 --task detect ``` --- # NVIDIA DeepStream NVIDIA DeepStream runs inference through its nvinfer element, which needs an ONNX graph, a matching config file and a bounding-box parser. Setting deepstream=True on the ONNX export writes the first two and wires them to the third. ## Availability DeepStream export ships in v1.5.0. It merged into `dev` on 2026-08-08 in pull request 728, so a current install has it and no branch pin is needed. **Install** ```bash pip install "libreyolo[onnx]" ``` If you cloned the `deepstream-export` branch before 2026-08-08, replace it. That branch was rebased and force-pushed, and the older history is missing the fix that lets these exports run on a CUDA machine at all. ## What the export writes `model.export(format="onnx", deepstream=True)` writes three files side by side. For `libreyolo9s.pt`: - `libreyolo9s.onnx`, the detection graph, one output tensor of shape `(batch, num_detections, 6)`, each row `[x1, y1, x2, y2, score, class_id]` in network-input pixel coordinates. - `config_infer_primary_libreyolo9s.txt`, an `nvinfer` configuration carrying the family's preprocessing constants, class count, thresholds and parser wiring. - `libreyolo9s_labels.txt`, one class name per line. A labels file appears whenever the checkpoint carries class names. Depth models have none, so they get neither the file nor a `labelfile-path` key. LibreYOLO does not emit a `.so`. The `.so` that DeepStream loads is the bounding-box parser from `marcoslucianops/DeepStream-Yolo`, compiled once per device, and it is the same binary whichever LibreYOLO detector you point it at. The model is the ONNX. Classification and semantic segmentation need no parser at all, because `nvinfer` post-processes those itself. ## Export the model **Python** ```python from libreyolo import LibreYOLO9, LibreDFINE # Writes libreyolo9s.onnx, config_infer_primary_libreyolo9s.txt # and libreyolo9s_labels.txt into the working directory. LibreYOLO9("libreyolo9s.pt", size="s").export(format="onnx", deepstream=True) # Keep each detection model in its own directory: every detection # config names the same engine cache file. See "Known traps". LibreDFINE("LibreDFINEs.pt", size="s").export(format="onnx", deepstream=True) ``` **Arguments** ```python model.export( format="onnx", # deepstream=True is rejected for every other format deepstream=True, conf=0.25, # seeds pre-cluster-threshold (and classifier-threshold, # segmentation-threshold on those tasks) iou=0.45, # seeds nms-iou-threshold, omitted at cluster-mode=4 batch=1, # seeds batch-size and the engine cache filename half=False, # True marks the config network-mode=2 (fp16 build) int8=False, # True marks the config network-mode=1 dynamic=True, # dynamic batch axis in the ONNX graph imgsz=640, # seeds infer-dims=3;H;W ) # deepstream=True and nms=True are mutually exclusive: DeepStream runs # suppression in its clustering stage, so nothing is embedded in the graph. ``` **Fetch D-FINE weights first** ```bash curl -L -o LibreDFINEs.pt \ https://huggingface.co/LibreYOLO/LibreDFINEs/resolve/main/LibreDFINEs.pt ``` `LibreDFINE._load_weights` raises `FileNotFoundError` when the file is not already on disk, without attempting a download, so fetch `LibreDFINEs.pt` yourself first. That gap is tracked as [issue #727](https://github.com/LibreYOLO/libreyolo/issues/727). YOLO9 weights download on first use. The flag is Python only. `libreyolo export` on this branch has no `deepstream` option, and the CLI builds its export arguments from a fixed list rather than passing unknown keys through. ## Build the bounding-box parser Detection needs the parser library, instance segmentation needs a different one, and the remaining tasks need none. Two things on the DeepStream 8.0 image break the documented build command, and both are environmental rather than LibreYOLO problems. The image ships `cuda`, `cuda-12`, `cuda-12.5`, `cuda-12.8` and `cuda-12.9` under `/usr/local`. Only `cuda-12.5` has a complete toolkit. It also ships `libcublas.so.12` and `libcublas.so.12.8.4.1` but not the unversioned `libcublas.so` that `-lcublas` resolves against. The script below works around both. **build_parser.sh, run inside the DeepStream container** ```bash set -e git clone --depth 1 https://github.com/marcoslucianops/DeepStream-Yolo.git # /usr/local/cuda-12 on this image is a stub and the build dies on it with # "fatal error: crt/host_defines.h: No such file or directory". Resolve a # toolkit that actually carries the header; on the 8.0 image that is cuda-12.5. CUDA_DIR=$(readlink -f /usr/local/cuda) [ -f "$CUDA_DIR/include/crt/host_defines.h" ] || \ CUDA_DIR=$(ls -d /usr/local/cuda-*.* | sort -Vr | \ while read d; do [ -f "$d/include/crt/host_defines.h" ] && echo "$d" && break; done) # The image ships libcublas.so.12 and libcublas.so.12.8.4.1 but not the # unversioned libcublas.so that -lcublas needs, so the link step fails with # "/usr/bin/ld: cannot find -lcublas". Give the linker the names it wants. mkdir -p /tmp/cudalibs for lib in cublas cublasLt cudart; do real=$(find /usr/local -name "lib${lib}.so.1*" | grep -v stubs | sort -V | tail -1) ln -sf "$real" "/tmp/cudalibs/lib${lib}.so" done export LIBRARY_PATH="/tmp/cudalibs:$LIBRARY_PATH" make -C DeepStream-Yolo/nvdsinfer_custom_impl_Yolo CUDA_VER="${CUDA_DIR##*/cuda-}" ``` **Instance segmentation uses a different parser** ```bash git clone --depth 1 https://github.com/marcoslucianops/DeepStream-Yolo-Seg.git make -C DeepStream-Yolo-Seg/nvdsinfer_custom_impl_Yolo_seg \ CUDA_VER="${CUDA_DIR##*/cuda-}" ``` Then point `custom-lib-path` in the generated config at the built `libnvdsinfer_custom_impl_Yolo.so`. The generated value is the relative path `nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so`, which resolves when `deepstream-app` runs from the `DeepStream-Yolo` checkout and needs editing otherwise. ## Run the pipeline Check that the container can see the GPU before spending time on anything else. This is the check the validation run made first, on a Blackwell card under WSL2. **Confirm GPU passthrough before anything else** ```bash docker run --rm --gpus all nvcr.io/nvidia/tritonserver:26.04-py3 \ nvidia-smi --query-gpu=name,driver_version,compute_cap --format=csv ``` Output: ``` name, driver_version, compute_cap NVIDIA GeForce RTX 5070 Ti, 591.86, 12.0 ``` The validation run drove `deepstream-app` with one file source, no display sink, the on-screen display on, and `gie-kitti-output-dir` set so every frame's detections landed on disk as KITTI text. A config with those settings: **deepstream_app_config.txt** ```text [application] enable-perf-measurement=1 perf-measurement-interval-sec=5 gie-kitti-output-dir=kitti [tiled-display] enable=0 [source0] enable=1 type=3 uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 num-sources=1 gpu-id=0 [streammux] gpu-id=0 batch-size=1 batched-push-timeout=40000 width=1920 height=1080 live-source=0 [primary-gie] enable=1 gpu-id=0 gie-unique-id=1 config-file=config_infer_primary_libreyolo9s.txt [osd] enable=1 border-width=2 text-size=15 [sink0] enable=1 type=1 sync=0 [tests] file-loop=0 ``` **Run it** ```bash deepstream-app -c deepstream_app_config.txt ``` Output: ``` App run successful ``` **Both steps in one container** ```bash docker run --rm --gpus all -v "$PWD:/work" -w /work \ nvcr.io/nvidia/deepstream:8.0-samples-multiarch \ bash -c "bash build_parser.sh && deepstream-app -c deepstream_app_config.txt" ``` `nvinfer` builds the TensorRT engine from the ONNX on first run and caches it next to the model, so the first run pays for the engine build and later ones load the cache. ## The generated config Both configs below were written by the exporter for the validation run, not edited afterwards. | Key | YOLO9-s | D-FINE-s | |---|---|---| | `net-scale-factor` | 0.003921568627 | 0.003921568627 | | `model-color-format` | 0 | 0 | | `infer-dims` | 3;640;640 | 3;640;640 | | `maintain-aspect-ratio` | 1 | 0 | | `symmetric-padding` | 0 | 0 | | `network-type` | 0 | 0 | | `num-detected-classes` | 80 | 80 | | `cluster-mode` | 2 | 4 | | `parse-bbox-func-name` | NvDsInferParseYolo | NvDsInferParseYolo | | `pre-cluster-threshold` | 0.25 | 0.25 | | `nms-iou-threshold` | 0.45 | | | `topk` | 300 | 300 | The two configs differ in three places: `maintain-aspect-ratio`, `cluster-mode`, and whether `nms-iou-threshold` is present at all. D-FINE's config omits that key entirely, which is what `cluster-mode=4` calls for. Heads that emit at most one prediction per object get `cluster-mode=4`, so DeepStream runs no clustering over them; clustering would merge genuinely distinct detections. That covers `rfdetr`, `dfine`, `deim`, `deimv2`, `ec`, `rtdetr`, `rtdetrv2`, `rtdetrv4` and `yolo9_e2e`. Grid and anchor heads get `cluster-mode=2` plus `nms-iou-threshold`. Detection configs also carry `engine-create-func-name=NvDsInferYoloCudaEngineGet`, which hands engine building to the parser library. That is what fixes the engine cache filename, and it is the source of the collision described under known traps. ## Supported tasks and families Forty-three family and task combinations export. `deepstream_supported_tasks()` and `deepstream_supported_families(task)` in `libreyolo/export/deepstream.py` return the same lists at runtime. | Task | `network-type` | Parser library | Families | |---|---|---|---| | Detection | 0 | DeepStream-Yolo | yolo9, yolo9_p2, yolo9_e2e, yolo1, yolo2, yolo3, yolo4, yolo7, yolox, yolonas, rtmdet, picodet, rfdetr, dfine, deim, deimv2, ec, rtdetr, rtdetrv2, rtdetrv4 | | Classification | 1 | None needed | mobilenetv4, convnext, efficientnetv2, resnet, dinov2 | | Semantic segmentation | 2 | None needed | pidnet, eomt, dinov2, lingbotvision | | Instance segmentation | 3 | DeepStream-Yolo-Seg | rfdetr, dfine, ec | | Pose | 100 | None needed | yolo9, yolonas, rfdetr, ec | | Depth | 100 | None needed | depth_anything, zipdepth | | Restoration | 100 | None needed | nafnet, realesrgan, swinir | | Matting | 100 | None needed | birefnet | | Gaze | 100 | None needed | l2cs | `network-type=100` means DeepStream has no post-processor for the task. Those configs set `output-tensor-meta=1`, the graph's native outputs pass through untouched, and the application decodes them from the tensor metadata. Multi-output graphs are fine there: every output layer reaches the metadata with the same output names and dynamic axes as a plain ONNX export. Instance segmentation rows are the detection row followed by that instance's mask, flattened at `(netH / 4, netW / 4)`, which is the resolution the seg parser hardcodes, as probabilities for `segmentation-threshold`. Classification and gaze run as secondary inference. Set `process-mode=2` and `operate-on-gie-id` in the generated config to put a classifier behind a detector. Gaze is a head-only contract, one face crop per input, so it needs a face detector in front of it. Three families are absent on purpose. `segformer` is not wired to the shared semantic export contract and cannot export to ONNX in any format. RTMDet-Ins and YOLO9 have their instance segmentation export blocked in LibreYOLO itself. `depth_anything3` has no export implementation. Two rows in the table have checkpoint gaps behind them. Only the `l` EoMT semantic checkpoint is published, and DINOv2 classification has no published checkpoint at all, so that combination needs your own fine-tuned weights. ## Preprocessing differences `nvinfer` computes `net-scale-factor * (x - offsets)` per channel with a scalar scale, which cannot express per-channel standard deviation. Families that need one (`rfdetr`, `ec`, the DINO-backboned `deimv2` sizes, `rtmdet`, `picodet`, and every classification family) have the normalization baked into the exported graph, and the generated config feeds the graph the matching raw input space. The geometry is where LibreYOLO's own Python pipelines and `nvinfer` still diverge: - Letterbox families (`yolo9`, `yolox`, `yolonas`, `rtmdet`, `yolo2`, `yolo3`, `yolo4`, `yolo7`) pad with gray natively. `nvinfer` pads black. - `yolonas` detection natively resizes the longest side to 636 inside its 640 canvas. `nvinfer`'s `maintain-aspect-ratio` uses the full 640. - Classification natively resizes the shortest side then center-crops. `nvinfer` stretches the frame or object ROI to the network input, so tightly cropped subjects differ. - EoMT natively runs sliding-window tiles for semantic segmentation. The exported graph is a single stretched canvas, which is faster and less accurate. - `pidnet` emits a class map at 1/8 of the input resolution and `lingbotvision` at 1/16. DeepStream upsamples the class map for display. The ONNX parity gate feeds already-preprocessed tensors, so it checks graph outputs and cannot catch a wrong color order or padding policy in the config. Validate on your own data before deploying an exact-parity workload. ## Known traps ### Two detection models in one directory load each other's engine Every detection config carries the same line: ```ini model-engine-file=model_b1_gpu0_fp32.engine ``` The parser's engine builder requires that basename and it does not vary by model. Export a second detection model into the same directory and the second run loads the first model's cached engine. Nothing crashes; the boxes are just wrong. Give each detection model its own directory. The validation run had to isolate D-FINE into one before it could be tested at all. ### A box can only carry one class `nvinfer`'s row format is `[x1, y1, x2, y2, score, class_id]`, one class per box, so the export collapses class scores to their argmax. A box that `predict` reports under two classes survives under one. Measured case: LibreYOLO reports `vase 0.773` and `bottle 0.383` on the same box, and the DeepStream graph keeps `vase`. This follows from the parser's row format and cannot be changed without leaving that contract, so it is expected behavior rather than a regression. ## Validated `deepstream-app` ran to EOS with `App run successful` on both detector head types, over NVIDIA's bundled `sample_1080p_h264.mp4` (1443 frames), with per-frame KITTI dumps enabled. | | YOLO9-s | D-FINE-s | |---|---|---| | Head type | grid | one-to-one | | `cluster-mode` | 2 | 4 | | `maintain-aspect-ratio` | 1 | 0 | | Frames with detections | 1443 | 1443 | | Total detections | 18031 | 71105 | Class histograms over all 1443 frames put cars first and people second for both models, which is right for a street scene. The four-fold gap in detection count is the `cluster-mode` difference doing its job: D-FINE at `cluster-mode=4` runs no clustering, so every query above threshold survives, near-duplicates included. Two independently trained models put the dominant object in the same place: ```text YOLO9 bus [706.72, 0.82, 1916.34, 1062.97] conf 0.965 D-FINE bus [702.73, 2.93, 1916.24, 1069.32] conf 0.965 ``` That run establishes five things: TensorRT builds an engine from the exported ONNX on sm_120, `nvinfer` accepts every key in the generated config, `NvDsInferParseYolo` reads the tensor layout correctly, boxes land in source-resolution 1920x1080 coordinates, and labels resolve against the generated labels file. The environment it ran in: | Component | Value | |---|---| | Host OS | Windows 11 Pro 26200 | | GPU | NVIDIA GeForce RTX 5070 Ti, 16 GB | | Driver | 591.86 | | Compute capability | 12.0 (Blackwell, sm_120) | | Container runtime | Docker Desktop 29.4.3, WSL2 backend | | DeepStream image | `nvcr.io/nvidia/deepstream:8.0-samples-multiarch` | | DeepStream version | 8.0.0 | | Container CUDA | 12.8.1 | | Parser | `marcoslucianops/DeepStream-Yolo` at HEAD | Alongside the pipeline run, `tests/unit/test_deepstream_export.py` covers the graph adapters and the generated config keys, and its 35 tests pass on this commit. ## Not validated Stated so the scope above is not read wider than it is. - Jetson and aarch64. The export contract does not depend on the architecture, but the pipeline has only been run on an x86 discrete GPU. - Forty-one of the 43 combinations. Only detection with `yolo9` and detection with `dfine` went through DeepStream. Classification, semantic segmentation, instance segmentation and the raw-tensor tasks are covered by unit tests and ONNX parity checks, not by a pipeline run. - FP16 and INT8. Only `network-mode=0` was exercised. - Multi-stream and batching. One source, `batch-size=1`. - Accuracy against a ground-truth dataset. Detections were checked for semantic plausibility and cross-model agreement, not scored as mAP through DeepStream. --- # ExecuTorch ExecuTorch runs PyTorch programs on edge targets. LibreYOLO captures the model with torch.export in strict mode, lowers it to XNNPACK, and commits the .pte program together with a JSON metadata sidecar as one unit. Verified against LibreYOLO v1.5.0. ## Install **Install** ```bash # Kept out of libreyolo[all] on purpose: ExecuTorch constrains which # Torch version it can be paired with. pip install "libreyolo[executorch]" ``` This extra is deliberately outside `libreyolo[all]`, because ExecuTorch pins which Torch version it works with and installing it would drag the whole environment onto that pair. Install it into an environment you are willing to constrain. On Windows the lowering step calls the `flatc` executable that ships with ExecuTorch. If it is not on `PATH` the export raises a `RuntimeError` saying so, and running from a Visual Studio 2022 Developer PowerShell is the fix. ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") # Writes weights/LibreYOLO9t.pte and weights/LibreYOLO9t.pte.json path = model.export(format="executorch", imgsz=640) print(path) ``` **CLI** ```bash libreyolo export --model LibreYOLO9t.pt --format executorch --imgsz 640 ``` **Arguments** ```python model.export( format="executorch", imgsz=640, # int, or (height, width) batch=1, # any other value raises ValueError dynamic=False, # True raises ValueError delegate="xnnpack", # the only accepted value device="cpu", # any other device raises ValueError output_path=None, # None writes weights/.pte ) ``` Capture is `torch.export.export(..., strict=True)`, which is a real graph capture with guards rather than a recorded trace. Host scalar reads and data-dependent control flow are rejected instead of being silently baked in, so several families fail here that trace successfully elsewhere; the reasons are recorded per combination in the support matrix. Lowering runs `to_edge_transform_and_lower` with the XNNPACK partitioner. If the result contains zero delegate partitions the export raises rather than labeling a portable-kernel-only program as XNNPACK. The program and the sidecar are committed together. Both are staged, both are swapped in, and a failure rolls back to whatever was there before, so a partial pair never reaches disk. ## Run the artifact **Through LibreYOLO** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("weights/LibreYOLO9t.pte") result = model.predict(SAMPLE_IMAGE) print(result.boxes.xyxy[:3]) ``` **Bare ExecuTorch runtime** ```python import json from pathlib import Path import torch from executorch.runtime import Runtime runtime = Runtime.get() print(runtime.backend_registry.is_available("XnnpackBackend")) program = runtime.load_program(Path("weights/LibreYOLO9t.pte").read_bytes()) method = program.load_method("forward") # Preprocessing and postprocessing are yours on this path. outputs = method.execute((torch.zeros(1, 3, 640, 640),)) print([tensor.shape for tensor in outputs]) meta = json.load(open("weights/LibreYOLO9t.pte.json")) print(meta["model_family"], meta["task"], meta["executorch_delegate"]) ``` `LibreYOLO()` dispatches on the `.pte` suffix and returns the same `Results` object as the checkpoint. The sidecar is mandatory on load: without `.pte.json` the backend raises `FileNotFoundError`, because the program carries no class names, task or input size of its own. The backend also checks that the installed runtime provides `XnnpackBackend` before loading, and reads the program from bytes rather than mapping the file, which avoids holding a Windows file lock for the backend's lifetime. The second snippet is the bare-runtime path. Preprocessing, decoding, NMS and coordinate rescaling become yours there. ## Constraints Batch 1, fixed shape, FP32, CPU. `batch != 1` and `dynamic=True` both raise `ValueError` before the export mutates anything, `half=True` and `int8=True` are rejected during validation, and a device other than CPU is refused. `delegate` accepts `"xnnpack"` and nothing else in this version. Classification exports carry two extra metadata keys, `crop_pct` and `interpolation`, so the runtime can reproduce the family's resize and center-crop policy. The blocked entries name the concrete failure rather than a category. D-FINE detection and segmentation reach an unsupported `ContextVar` read in deformable attention under strict capture, and forcing the manual grid-sample path serializes but then fails at run time on an invalid delegated tensor dimension order. DEIM and DEIMv2 capture, lower and serialize, then fail during execution. EoMT semantic segmentation fails on a data-dependent symbolic expression in the mask path. BiRefNet matting captures at 1024 by 1024 but has no out variant for `torchvision::deform_conv2d`. SwinIR restoration reloads and then fails in `aten::alias_copy.out` on mismatched dimension orders. For the full family and task grid, see [the export matrix](/docs/reference/export-matrix). For one combination: **Check one family and task before exporting** ```bash libreyolo formats --family yolo9 --task detect ``` --- # Hailo Hailo accelerators are compiled with the Hailo Dataflow Compiler, a proprietary SDK distributed through Hailo's Developer Zone. LibreYOLO's part of the flow is a plain static ONNX export; parsing, quantization and compilation to a HEF happen in the DFC afterwards. Verified against LibreYOLO v1.5.0. ## Install There is no `format="hef"` in LibreYOLO and there will not be one. The Hailo Dataflow Compiler is a proprietary SDK distributed as a private wheel behind Developer Zone registration, so it cannot be a dependency or an extra. Deployment is two stages: LibreYOLO writes a static ONNX file, and you run the DFC over it. ```text Libre.pt -> ONNX -> HAR (parse) -> HAR (quantize INT8) -> HEF [libreyolo] [Hailo DFC, installed by you] ``` **LibreYOLO side** ```bash pip install "libreyolo[onnx]" ``` **Hailo side, installed by you** ```text Prerequisites, none of them installable from PyPI: - A Linux x86_64 machine. WSL2 Ubuntu 22.04 works. The Raspberry Pi is a runtime target, never the compile host. - The Dataflow Compiler wheel (hailo_sdk_client) from the Hailo Developer Zone, which is free to register for. - For Hailo-8 and Hailo-8L, the Hailo Model Zoo v2.x line, for its recipes and NMS configurations. - A GPU on the compile host is strongly recommended: the quantization step takes hours without one. ``` ## Export **Python** ```python from libreyolo import LibreYOLO # Hailo needs batch 1, a fixed resolution and no dynamic axes. # The Python API defaults to dynamic=True, so turn it off explicitly. model = LibreYOLO("LibreYOLOXs.pt") model.export(format="onnx", imgsz=640, dynamic=False, simplify=True) ``` **CLI** ```bash # The CLI already defaults to static shapes. libreyolo export --model LibreYOLOXs.pt --format onnx --imgsz 640 ``` **Confirm the graph is static before compiling** ```python import onnx graph = onnx.load("weights/LibreYOLOXs.onnx").graph shape = graph.input[0].type.tensor_type.shape print([d.dim_value or d.dim_param for d in shape.dim]) ``` Do not pass `half=True`. The DFC ingests FP32 ONNX and does its own INT8 quantization. Do not pass `nms=True` either: Hailo either owns NMS through `nms_postprocess` or the application does, and an NMS subgraph is dead weight past the end nodes. The default opset works; if the DFC parser objects, re-export with `opset=11`. The DFC cuts the graph at the end nodes you supply, which are the detection-head convolutions, and discards everything downstream. LibreYOLO's ordinary decoded ONNX is therefore acceptable input: the decode tail is simply ignored by the parser. ## Compile **Parse, quantize and compile** ```python from pathlib import Path import numpy as np from hailo_sdk_client import ClientRunner from PIL import Image ONNX = "weights/LibreYOLOXs.onnx" HW_ARCH = "hailo8" # hailo8 | hailo8l | hailo10h IMGSZ = 640 runner = ClientRunner(hw_arch=HW_ARCH) # For YOLOX, translate once without end_node_names: the DFC log prints # the end nodes it suggests. Re-run with those. runner.translate_onnx_model(ONNX) # Normalization must match LibreYOLO preprocessing. YOLOX and YOLO9 # need no mean or standard deviation, only the 0-255 to 0-1 scale. script = "normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])\n" # Optional: let Hailo own NMS. The configuration is specific to both the # class count and the input size, so a COCO-80 config is wrong for a # fine-tuned three-class model. Without this line the HEF emits raw head # tensors and the application decodes them. # script += 'nms_postprocess("yolox_nms_config.json", meta_arch=yolox, engine=cpu)\n' runner.load_model_script(script) # Calibration images must be representative of deployment data. # Random images compile and silently destroy accuracy. calib_paths = sorted(Path("calib_images").glob("*.jpg"))[:128] calib = np.stack([ np.asarray( Image.open(p).convert("RGB").resize((IMGSZ, IMGSZ)), dtype=np.float32, ) for p in calib_paths ]) runner.optimize(calib) Path("libreyoloxs.hef").write_bytes(runner.compile()) ``` **YOLO9 end nodes** ```python # LibreYOLO graphs use a "/head/..." prefix, not the "model.N" prefix # seen in configurations written for other exports. A copied config will # not match. Confirm the names in your own graph if parsing fails. END_NODES = [ "/head/cv2.0/cv2.0.2/Conv", "/head/cv3.0/cv3.0.2/Conv", "/head/cv2.1/cv2.1.2/Conv", "/head/cv3.1/cv3.1.2/Conv", "/head/cv2.2/cv2.2.2/Conv", "/head/cv3.2/cv3.2.2/Conv", ] runner.translate_onnx_model(ONNX, end_node_names=END_NODES) ``` Pick `hw_arch` for the target: `hailo8` for Hailo-8, the 26 TOPS AI HAT+ and the M.2 and PCIe modules; `hailo8l` for Hailo-8L, the Raspberry Pi AI Kit and the 13 TOPS AI HAT+; `hailo10h` for Hailo-10H, which needs a matching newer DFC and Model Zoo. `hailortcli fw-control identify` on the device answers the question when you are unsure. Two families map onto a HailoRT NMS meta-architecture, so Hailo can own suppression inside the compiled pipeline: YOLOX through `meta_arch=yolox`, and YOLO9 through Hailo's decoupled-head meta-architecture, whose head layout is identical. Take the matching `nms_postprocess` configuration from the Hailo Model Zoo and adjust it for your class count and input size. Every other convolutional detector compiles as a graph with no matching meta-architecture: the HEF emits raw head tensors and the application runs decode and NMS on the CPU. Keep the compile log when something fails. Every fix hinges on the exact failing layer or operator name. ## Run the artifact **Raspberry Pi 5 with the AI Kit or AI HAT+** ```bash sudo apt install dkms hailo-all hailortcli fw-control identify # device check, and it names the arch hailortcli run libreyoloxs.hef # smoke test and throughput ``` Application inference uses the `hailo_platform` Python API. With `nms_postprocess` compiled in, the output is `(batch, num_classes, max_dets, 5)` carrying `[y1, x1, y2, x2, score]` in model coordinates, which you scale back to the source image yourself. LibreYOLO's `Results` pipeline is not involved at run time; the HEF is a standalone artifact, and preprocessing and postprocessing are the application's. ## Constraints Whether a model can target Hailo-8 or Hailo-8L is a property of its architecture, not its name, so the rule below applies to families added after this page was written. A model will not compile if it contains any of these: - Attention of any kind, self, cross, deformable or windowed. That rules out every DETR-style detector, every open-vocabulary or text-conditioned detector, every ViT backbone, and every language or vision-language tower. Hailo's own zoo ships a few hand-tuned transformer HEFs; that is bespoke vendor work and is not evidence that an arbitrary attention graph compiles. - Dynamic shapes or data-dependent control flow. The DFC compiles one fixed input shape and a static graph, so variable query counts, text prompts, dynamic top-k, `NonZero`, `Gather` or `TopK` with dynamic indices, and `grid_sample` are all out. - A LayerNorm-dominated or GELU-dominated design. BatchNorm folds into convolutions cleanly; LayerNorm support is poor and GELU is not a native activation, so a ConvNeXt-style stack is a bad fit even though it is nominally convolutional. - Native-resolution image-to-image work. Restoration models run at full input resolution and exceed practical Hailo SRAM budgets. A family is a candidate when it is convolution only, uses BatchNorm with ReLU or SiLU, and has a fixed input size. In this library that means the CNN single-stage detectors, with YOLOX and YOLO9 as the primary targets; other convolutional detectors such as PicoDet, YOLO-NAS and RTMDet, with application-side decode; the CNN classifiers ResNet, MobileNetV4-conv and EfficientNetV2, of which ResNet is best supported because Hailo's Model Zoo ships recipes for it; and small convolutional task heads such as FOMO point detection and L2CS gaze on a ResNet backbone, which are compilable in principle but have no Hailo recipe. One status caveat, which is the reason nothing on this page is presented as supported: no LibreYOLO family has been taken end to end through the DFC to a running HEF. The rules above predict compilability from architecture. Parser behavior, quantization and accuracy remain unproven until a HEF is compiled and measured, so treat every candidate as requiring its own recorded evidence: a compiled HEF from the exact checkpoint with DFC, Model Zoo and HailoRT versions recorded, documented calibration, and an on-device accuracy comparison against the FP32 baseline rather than a throughput number. If the model is disqualified, the alternatives are the runtimes with recorded parity: [ONNX](/docs/export/onnx), [TensorRT](/docs/export/tensorrt) and [OpenVINO](/docs/export/openvino). --- # NVIDIA Jetson NVIDIA Jetson boards run LibreYOLO on the standard aarch64 PyTorch wheels. No Jetson-specific torch build is involved, but JetPack omits four libraries that torch links against, and the install has to supply them. Verified against LibreYOLO v1.4.0. ## What this page records This page records one configuration that was verified end to end, not a support matrix. The board was a Jetson Orin Nano Super Developer Kit with 8 GB of memory running JetPack 7.2 (L4T R39.2, Ubuntu 24.04, CUDA 13, Python 3.12.3), and the stack that came up on it was `libreyolo 1.4.0` with `torch 2.13.0+cu130`, OpenCV 5.0.0 and NumPy 2.5.1. `torch.cuda.is_available()` returned `True` and the GPU reported itself as `Orin`. Other JetPack releases, other Jetson boards and other CUDA versions were not tested. The recipe below is the one that worked on that combination. That run was on 2026-07-27 against LibreYOLO 1.4.0, and it has not been repeated on 1.5.0 hardware: this is the one page in the 1.5.0 tree still carrying a 1.4.0 verification, which is why its front matter says `last_verified: "1.4.0"`. Nothing in the 1.5.0 changes touches the install path, the four missing libraries or the export flags described here, so the commands are expected to hold, but the version numbers in the outputs below are what 1.4.0 printed, not a 1.5.0 measurement. Two things about it run against what most Jetson guides say. The wheels are the ordinary aarch64 builds published for CUDA 13, so no Jetson-specific torch build is needed. And JetPack does not ship four libraries that those wheels link against, so `import torch` fails one library at a time until all four are installed. ## Install JetPack images arrive without pip and without the `venv` module, so both come first. **System packages and a virtual environment** ```bash # JetPack does not preinstall pip or the venv module. sudo apt update sudo apt install -y python3.12-venv python3-pip python3 -m venv ~/libreyolo source ~/libreyolo/bin/activate pip install -U pip wheel setuptools ``` An 8 GB board is tight for larger checkpoints. Adding swap on the NVMe before loading them avoids an out-of-memory kill mid-run. Then PyTorch. The CUDA 13 index carries the aarch64 wheels; the extra index supplies the pure-Python dependencies from PyPI. **PyTorch, from the CUDA 13 wheel index** ```bash pip install torch torchvision \ --index-url https://download.pytorch.org/whl/cu130 \ --extra-index-url https://pypi.org/simple ``` **The four libraries JetPack does not ship** ```bash pip install nvidia-cudnn-cu13 nvidia-nccl-cu13 \ nvidia-cusparselt-cu13 nvidia-nvshmem-cu13 ``` **If pip demands cuda-toolkit 13.0.3, install with --no-deps** ```bash # --no-deps means torch's Python dependencies are named by hand too. pip install --no-deps \ torch torchvision \ nvidia-cudnn-cu13 nvidia-nccl-cu13 \ nvidia-cusparselt-cu13 nvidia-nvshmem-cu13 \ filelock typing_extensions sympy networkx jinja2 markupsafe mpmath \ fsspec numpy pillow ``` The four `nvidia-*-cu13` wheels are the part that is easy to miss. JetPack provides the GPU driver, not cuDNN, NCCL, cuSPARSELt or NVSHMEM, and torch refuses to import without them. Installing all four at once is faster than discovering them one exception at a time. The third snippet covers a specific failure: torch's dependency metadata for the CUDA 13 build asks for `cuda-toolkit==13.0.3`, which has no aarch64 wheel on PyPI, so resolution fails before anything downloads. `--no-deps` skips the resolver, which means every dependency has to be named on the command line. LibreYOLO goes in last. Installing it first lets pip choose its own torch, which on this platform is not the CUDA build. **Install LibreYOLO after torch, not before** ```bash # torch is already satisfied, so pip leaves the CUDA build in place. pip install libreyolo # The ONNX extra is only needed to export. A TensorRT export runs # through ONNX, so add it before the export section below. pip install "libreyolo[onnx]" ``` Every remaining dependency resolves to a prebuilt aarch64 wheel, including OpenCV, NumPy, SciPy, pycocotools and safetensors. Nothing compiles from source. ## Check that CUDA works **Versions and device** ```python import cv2 import numpy import torch import libreyolo print("torch", torch.__version__, "cuda", torch.cuda.is_available()) print("gpu", torch.cuda.get_device_name(0)) print("libreyolo", libreyolo.__version__) print("cv2", cv2.__version__, "numpy", numpy.__version__) ``` Output: ``` torch 2.13.0+cu130 cuda True gpu Orin libreyolo 1.4.0 cv2 5.0.0 numpy 2.5.1 ``` **Then run a real kernel** ```python import torch x = torch.rand(2000, 2000, device="cuda") print(float((x @ x).sum())) ``` The second snippet matters as much as the first. A wheel built for the wrong GPU architecture still reports `torch.cuda.is_available() == True` and then fails on the first real operation with `CUDA error: no kernel image is available for execution on the device`. A matrix multiply on the device is the check that catches it. ## Run a prediction **Python** ```python from libreyolo import LibreYOLO9, SAMPLE_IMAGE # Downloads the checkpoint on first use. model = LibreYOLO9("libreyolo9s.pt", size="s") result = model.predict(SAMPLE_IMAGE) print(result.boxes) ``` **CLI** ```bash libreyolo predict --source https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg --model libreyolo9s.pt --save ``` `predict` returns the same `Results` object as on any other platform, so model pages apply unchanged. ## Export to TensorRT On this board, TensorRT was faster than both PyTorch and ONNX Runtime for all 55 models that were measured in every runtime. **Python** ```python from libreyolo import LibreYOLO, LibreYOLO9, SAMPLE_IMAGE # Writes libreyolo9s.onnx, then builds libreyolo9s.engine from it. LibreYOLO9("libreyolo9s.pt", size="s").export(format="tensorrt", half=True) # The engine loads back through the same entry point. result = LibreYOLO("libreyolo9s.engine").predict(SAMPLE_IMAGE) ``` **CLI** ```bash libreyolo export --model libreyolo9s.pt --format tensorrt --half ``` `format="tensorrt"` writes an ONNX graph first and builds the engine from it, so the `onnx` extra has to be installed. `LibreYOLO()` dispatches on the file suffix, so a `.engine` file loads through the same call as a `.pt` checkpoint. Do not use the `tensorrt` pip extra on a Jetson. It pins `tensorrt-cu12`, a CUDA 12 build, against a CUDA 13 platform. Use the TensorRT that JetPack installs instead. If `import tensorrt` fails inside the virtual environment while it works outside, recreate the environment with `--system-site-packages` so the system module is visible. Serialized TensorRT engines are tied to the device, the GPU architecture and the TensorRT version that built them. An engine built on a workstation will not load on a Jetson, so the build step runs on the board. ## Measured on this board Latency per image, batch size 1, end to end including preprocessing and postprocessing, on COCO val2017 (500-image subset) at `conf=0.001` and `max_det=300`. Five models out of the 58 measured: | Model | Input (px) | PyTorch FP32 (ms) | ONNX FP32 (ms) | TensorRT FP32 (ms) | TensorRT FP16 (ms) | mAP 50-95 | |---|---:|---:|---:|---:|---:|---:| | DEIMv2-Atto | 320 | 64.9 | 22.8 | 12.3 | 11.2 | 27.49 | | YOLOX-Tiny | 416 | 49.2 | 31.8 | 23.0 | 19.4 | 35.45 | | YOLO9-t | 640 | 101.2 | 53.8 | 36.0 | 29.1 | 41.78 | | RT-DETR-r18 | 640 | 98.3 | 103.7 | 45.3 | 25.7 | 49.72 | | D-FINE-s | 640 | 96.8 | 96.1 | 44.7 | 33.1 | 53.45 | The mAP column is the TensorRT FP16 run's own score. Across the 55 models measured in all four runtimes, the largest gap between the PyTorch FP32 score and the TensorRT FP16 score was 0.59 points, on DEIMv2-X. The runtimes differ in speed, not in accuracy. TensorRT FP32 was faster than both PyTorch and ONNX Runtime for all 55 of those models. TensorRT FP16 was faster than PyTorch FP32 for all 55 as well, by 1.68x to 6.22x, with a median of 3.39x. ONNX Runtime is the one that varies: it was slower than PyTorch on 23 of the 55, the RT-DETR-r18 row among them. Conditions behind every number: `libreyolo 1.2.0.dev0`, `torch 2.12.0+cu130`, Python 3.12.3, CUDA 13, driver 595.78, ONNX Runtime 1.24.0, measured June 2026. Latency on a Jetson also depends on the active power mode, which the benchmark records do not carry. **Power mode and clocks** ```bash sudo nvpmodel -q # which modes this board exposes, and the active one sudo nvpmodel -m 0 # highest mode on the board tested here sudo jetson_clocks tegrastats # live load; nvidia-smi is limited on Tegra ``` All 223 runs, including the other 53 models and the full accuracy columns, are published on [the Jetson Orin page at Vision Analysis](https://www.visionanalysis.org/hardware/jetson_orin). ## Troubleshooting ### import torch fails naming a shared library One of the four libraries above is missing. Rather than guessing which, read it off the binary: **Name the next missing library instead of guessing** ```bash ldd "$VIRTUAL_ENV/lib/python3.12/site-packages/torch/lib/libtorch_cuda.so" \ | grep "not found" # Everything still missing across all of torch's libraries, in one pass: ldd "$VIRTUAL_ENV"/lib/python3.12/site-packages/torch/lib/*.so 2>/dev/null \ | grep "not found" | sort -u ``` Each missing entry maps to one wheel: | Missing library | Wheel | |---|---| | cuDNN | `nvidia-cudnn-cu13` | | NCCL | `nvidia-nccl-cu13` | | cuSPARSELt | `nvidia-cusparselt-cu13` | | NVSHMEM | `nvidia-nvshmem-cu13` | ### torch warns that no build supports this GPU The first CUDA call on the working configuration prints this: ```text UserWarning: Found GPU0 Orin which is of compute capability (CC) 8.7. The following list shows the CCs this version of PyTorch was built for and the hardware CCs it supports: - 8.0 which supports hardware CC >=8.0,<9.0 except {8.7} - 9.0 which supports hardware CC >=9.0,<10.0 - 10.0 which supports hardware CC >=10.0,<11.0 except {10.1} - 11.0 which supports hardware CC >=11.0,<12.0 - 12.0 which supports hardware CC >=12.0,<13.0 No published PyTorch CUDA builds for release 2.13.0+cu130 support this GPU. ``` The warning is cosmetic on this board. The wheel carries `sm_80` kernels and the Orin executes them. The same warning appeared on the earlier wheel from that index, the one that produced every benchmark row above. Confirm with the matrix multiply from the CUDA check rather than trusting or distrusting the message. ### CUDA error: no kernel image is available for execution on the device The installed wheel was built for a different GPU architecture. This is what happens with wheels from NVIDIA's `sbsa` index, which target server ARM GPUs rather than Jetson silicon. Reinstall from the CUDA 13 index in the install section. ### pip cannot find cuda-toolkit 13.0.3 There is no aarch64 wheel for it. Use the `--no-deps` form in the install section and name torch's dependencies explicitly. ### libnvpl_lapack_lp64_gomp.so.0: cannot open shared object file The aarch64 torch wheel links NVIDIA Performance Libraries for CPU math. Install them and put them on the library path: ```bash pip install nvpl-lapack nvpl-blas --index-url https://pypi.jetson-ai-lab.io/sbsa/cu130/ export LD_LIBRARY_PATH="$VIRTUAL_ENV/lib/python3.12/site-packages/nvpl/lib:$LD_LIBRARY_PATH" ``` That index is fine for these two CPU libraries. Its torch builds are the ones that produce the "no kernel image" failure above. ### Wheel sources that do not fit JetPack 7.2 | Source | Result on the Orin Nano Super | |---|---| | `pypi.jetson-ai-lab.io/sbsa/cu130` torch | Built for server ARM GPUs. Imports, reports CUDA available, then fails with "no kernel image is available for execution on the device". | | `pypi.jetson-ai-lab.io/jp6/*` torch | CUDA 12 and Python 3.10 builds. They do not install on this image's Python 3.12. | | JetPack 6 PyTorch containers | CUDA initialization fails with error 801 on a JetPack 7 host. | | Building torch from source | Works, but takes hours on an 8 GB board and is unnecessary once the CUDA 13 wheels are installed. | ## DeepStream For a full video pipeline rather than a Python loop, export with `deepstream=True` and run the graph through `nvinfer`. That path has its own page, including the generated `nvinfer` config, the bounding-box parser build and the known traps: [DeepStream](/docs/export/deepstream). The DeepStream pipeline itself was validated on an x86 discrete GPU, not on a Jetson. The export contract does not depend on the architecture, but the pipeline run on aarch64 is still outstanding. ## Not verified - JetPack releases other than 7.2, and L4T releases other than R39.2. - Jetson boards other than the Orin Nano Super 8 GB. - Training on the board. Inference and export were exercised; a training run was not. - INT8 engines. Only FP32 and FP16 rows exist for this board. - Batch sizes above 1. Every measurement above is batch 1. --- # MNN MNN is Alibaba's lightweight inference engine. LibreYOLO exports a static ONNX graph, converts it with the mnnconvert tool shipped by the MNN package, and writes a JSON sidecar recording the input and output names, the fixed input shape and the class names. Verified against LibreYOLO v1.5.0. ## Install **Install** ```bash # The extra includes libreyolo[onnx]: MNN converts from an ONNX intermediate. pip install "libreyolo[mnn]" ``` **Confirm the converter is on the path** ```bash mnnconvert --version ``` The extra includes `libreyolo[onnx]`, because the conversion runs over an ONNX intermediate. It also brings the `mnnconvert` executable, which the exporter looks for next to the active Python interpreter first and on `PATH` second. A missing converter raises an `ImportError` naming the install command rather than failing mid-conversion. ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") # Writes weights/LibreYOLO9t.mnn and weights/LibreYOLO9t.mnn.json path = model.export(format="mnn", imgsz=640) print(path) ``` **CLI** ```bash libreyolo export --model LibreYOLO9t.pt --format mnn --imgsz 640 ``` **Arguments** ```python model.export( format="mnn", imgsz=640, # int, or (height, width) batch=1, # baked into the artifact simplify=True, # onnxsim over the ONNX intermediate output_path=None, # None writes weights/.mnn verbose=False, # True streams the mnnconvert log ) # dynamic=True raises ValueError. half=True and int8=True are rejected. ``` Before handing the graph over, the exporter reads the ONNX input contract and refuses anything it cannot express: more than one image input, or an input shape with a symbolic dimension. MNN in this version requires a fully fixed NCHW shape, and `batch` is baked into the artifact rather than negotiated at load time. The sidecar is not optional bookkeeping. `weights/LibreYOLO9t.mnn.json` records the input and output names, the fixed input shape, the batch, the class names, the MNN version used, and the backend the artifact was built for, and the runtime validates every one of those fields on load. On Windows, MNN 3.6.1 sometimes completes the conversion and then terminates during process teardown with an access violation or a fail-fast status. The exporter recognizes those specific exit codes and treats the conversion as successful when the output file is present. ## Run the artifact **Through LibreYOLO** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("weights/LibreYOLO9t.mnn") result = model.predict(SAMPLE_IMAGE) print(result.boxes.xyxy[:3]) ``` **Bare MNN** ```python import json import MNN import numpy as np meta = json.load(open("weights/LibreYOLO9t.mnn.json")) print(meta["mnn_input_names"], meta["mnn_output_names"], meta["mnn_input_shape"]) runtime = MNN.nn.create_runtime_manager( ({"backend": 0, "precision": 1, "numThread": 4},) ) module = MNN.nn.load_module_from_file( "weights/LibreYOLO9t.mnn", meta["mnn_input_names"], meta["mnn_output_names"], runtime_manager=runtime, dynamic=False, shape_mutable=False, ) blob = np.zeros(meta["mnn_input_shape"], dtype=np.float32) input_var = MNN.expr.const( blob, list(blob.shape), MNN.expr.NCHW, MNN.expr.float ) outputs = module.forward([input_var]) for out in outputs: print(np.array(MNN.expr.convert(out, MNN.expr.NCHW).read()).shape) # Preprocessing and postprocessing are yours on this path. ``` `LibreYOLO()` dispatches on the `.mnn` suffix and returns the same `Results` object as the checkpoint. The load is strict by design: the sidecar has to declare `format=mnn`, `mnn_backend=cpu`, `dynamic=false`, `precision=fp32`, a size, a detection task, a fixed positive NCHW shape that agrees with the recorded image size, and class names covering every index from 0 to `nc - 1`. Any mismatch raises rather than guessing. Prediction at a different `imgsz` than the artifact was built for raises too, and `device` is ignored with a warning, because MNN exports run on CPU here. The second snippet is the bare-runtime path. Preprocessing, decoding, NMS and coordinate rescaling become yours there, and the input and output names come from the sidecar because MNN's module loader wants them explicitly. ## Constraints Detection only. The backend refuses any other task on load, and the export side matches: outside the recorded combinations, preflight raises with "MNN v1 has no implemented runtime contract for this family and task." FP32, CPU, fixed shape. `dynamic=True` raises `ValueError`, and `half=True` and `int8=True` are rejected during validation. Validated detection families are YOLO9, YOLO9-E2E, YOLO9-P2, RF-DETR, EC, RT-DETR, RT-DETRv2, RT-DETRv4, D-FINE, DEIM and YOLO-NAS, each covered by conversion, a fresh artifact reload, MNN CPU execution, metadata checks and matched post-NMS detection parity against the PyTorch model. DEIMv2 converts, reloads, executes and preserves post-NMS detections, but its intermediate ONNX route has incomplete query-level score parity, so it is recorded as available rather than validated. For the full family and task grid, see [the export matrix](/docs/reference/export-matrix). For one combination: **Check one family and task before exporting** ```bash libreyolo formats --family yolo9 --task detect ``` --- # ncnn ncnn is Tencent's CPU inference library for mobile targets. LibreYOLO converts through PNNX, writing a model.ncnn.param graph beside a model.ncnn.bin weight file and a metadata.yaml that carries the family, task and class names. Verified against LibreYOLO v1.5.0. ## Install **Install** ```bash # pnnx converts, ncnn runs the result. pip install "libreyolo[ncnn]" ``` The extra pulls both halves of the toolchain: `pnnx` performs the conversion and `ncnn` executes the result. Neither goes through ONNX on the primary path. ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") # Writes the directory weights/LibreYOLO9t_ncnn path = model.export(format="ncnn", imgsz=640) print(path) ``` **CLI** ```bash libreyolo export --model LibreYOLO9t.pt --format ncnn --imgsz 640 ``` **Arguments** ```python model.export( format="ncnn", imgsz=640, # int, or (height, width) batch=1, simplify=True, # applies to the ONNX fallback path only opset=None, # auto; applies to the ONNX fallback path only output_path=None, # None writes weights/_ncnn ) # half=True and int8=True are rejected during validation. ``` The artifact is a directory. `weights/LibreYOLO9t_ncnn` holds `model.ncnn.param`, `model.ncnn.bin` and `metadata.yaml`; all three are one artifact and move together. Conversion tries PNNX directly from PyTorch first. If that fails it exports a static ONNX graph to a temporary directory and calls the `pnnx` command line tool on it, and the export only raises when both paths fail, reporting both errors. `opset` and `simplify` therefore only affect the fallback. YOLOX needs one rewrite to convert at all. Its Focus layer uses strided slicing, which PNNX cannot lower, so the export swaps it for `pixel_unshuffle` and permutes the following convolution's input channels to compensate for the different channel ordering. The output is numerically identical, and the original weights are restored after the export. ## Run the artifact **Through LibreYOLO** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("weights/LibreYOLO9t_ncnn") result = model.predict(SAMPLE_IMAGE) print(result.boxes.xyxy[:3]) ``` **Bare ncnn** ```python import ncnn import numpy as np import yaml directory = "weights/LibreYOLO9t_ncnn" net = ncnn.Net() net.load_param(f"{directory}/model.ncnn.param") net.load_model(f"{directory}/model.ncnn.bin") # ncnn takes a single CHW image, not a batch. mat_in = ncnn.Mat(np.zeros((3, 640, 640), dtype=np.float32)) extractor = net.create_extractor() extractor.input("in0", mat_in) ret, mat_out = extractor.extract("out0") print(ret, np.array(mat_out).shape) meta = yaml.safe_load(open(f"{directory}/metadata.yaml")) print(meta["model_family"], meta["task"], meta["names"]) # Preprocessing and postprocessing are yours on this path. ``` `LibreYOLO()` recognizes any directory holding `model.ncnn.param` and `model.ncnn.bin`, reads `metadata.yaml`, and returns the same `Results` object as the checkpoint. The second snippet is the bare-runtime path, and two details differ from every other format here. ncnn works on a single CHW image rather than a batch, so there is no leading batch axis. Blob names come from the `.param` file; PNNX writes `in0` and `out0` by convention, and the backend parses the file rather than assuming them. Preprocessing, decoding, NMS and coordinate rescaling are yours on that path. ## Constraints FP32 on a fixed canvas. `half=True` and `int8=True` are both rejected during validation, and the exported metadata records `dynamic=False` whatever the flag said, so no backend assumes an axis the graph does not have. Every DETR-style family is refused in preflight: `detr`, `deformable_detr`, `dinodetr`, `dfine`, `lwdetr`, `deim`, `deimv2`, `rtdetr`, `rtdetrv2`, `rtdetrv4`, `rfdetr` and `ec`. The message is the same for all of them, that the model needs decoder or sampling operations unavailable in ncnn, and it points at ONNX, OpenVINO, TorchScript or TensorRT instead. What does convert is broad on the convolutional side: YOLO9 and YOLO9-E2E, YOLOX, PicoDet, YOLO-NAS detection and pose, the older YOLO1, YOLO3, YOLO4 and YOLO7 detectors, the four CNN classification families, PIDNet semantic segmentation, FOMO point detection at a fixed 96 by 96, ZipDepth, NAFNet and Real-ESRGAN. Blocked entries name the concrete failure. Transformer graphs generally leave unsupported `pnnx.Expression` nodes behind, which produces a network with no runnable input blob, and that is what stops DINOv2, CLIP, SigLIP2 and SegFormer. BiRefNet needs torchvision deformable convolution, which PNNX cannot lower. YOLO2's converted graph terminates the ncnn runtime on Windows with a native integer divide by zero during output extraction. For the full family and task grid, see [the export matrix](/docs/reference/export-matrix). For one combination: **Check one family and task before exporting** ```bash libreyolo formats --family yolo9 --task detect ``` --- # ONNX ONNX is a portable graph format. LibreYOLO traces the model with torch.onnx.export, optionally simplifies the graph, and writes the family, task, class names and input size into the file's own metadata so any LibreYOLO backend can rebuild the postprocessing. Verified against LibreYOLO v1.5.0. ## Install **Install** ```bash pip install "libreyolo[onnx]" ``` The extra pulls `onnx`, `onnxsim` and `onnxruntime`. `onnx` alone is enough to write the file; `onnxsim` runs the simplification pass and `onnxruntime` runs the artifact and performs INT8 calibration. ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") # Writes weights/LibreYOLO9t.onnx path = model.export(format="onnx") print(path) ``` **CLI** ```bash libreyolo export --model LibreYOLO9t.pt --format onnx ``` **Arguments** ```python model.export( format="onnx", imgsz=640, # int, or (height, width) batch=1, dynamic=True, # Python default; the CLI defaults to False simplify=True, # run onnxsim over the graph opset=None, # None picks 13, or 17 for DETR-style families half=False, # FP16 weights and activations int8=False, # QDQ INT8, YOLO9 detection only data=None, # calibration data.yaml, INT8 only device=None, # trace device; None uses the model's device output_path=None, # None writes weights/.onnx ) ``` Without `output_path`, the file lands in `weights/` under the checkpoint's stem, with `_fp16` or `_int8` appended when that precision was requested. `dynamic` defaults to `True` in Python and `False` on the CLI. When it is on, the batch axis becomes symbolic and a few tasks widen further: semantic segmentation also opens the mask height and width, Real-ESRGAN restoration opens the spatial axes, and the two-stage detectors keep source height and width dynamic because their resize happens inside the graph. `opset` is chosen per family when omitted. DETR-style families (`detr`, `deformable_detr`, `dinodetr`, `dfine`, `deim`, `deimv2`, `ec`, `lwdetr`, `rfdetr`, `rtdetr`, `rtdetrv2`, `rtdetrv4`) plus `deit`, `midas` and `moge2` get opset 17, which is where `aten::scaled_dot_product` lowers. Everything else gets 13. Matting is raised to 19 regardless, because BiRefNet's decoder needs the `DeformConv` operator, which ONNX defines from opset 19. `simplify=True` runs `onnxsim` and keeps the original graph if the pass fails, so a simplification error is a warning rather than an export failure. On macOS arm64 with `onnx` 1.22 or newer and `onnxsim` 0.6.5 or older the pass is skipped entirely, because that pairing can abort the Python process. ### Embedded NMS **Embed NMS in the graph** ```python from libreyolo import LibreYOLO # YOLO9 detection only, batch 1. dynamic is forced to False. LibreYOLO("LibreYOLO9t.pt").export( format="onnx", nms=True, conf=0.25, iou=0.45, max_det=300, ) ``` **CLI** ```bash libreyolo export --model LibreYOLO9t.pt --format onnx --nms \ --conf 0.25 --iou 0.45 --max-det 300 ``` `nms=True` is YOLO9 detection only and requires batch 1; requesting it with `dynamic=True` logs a warning and turns dynamic off. The graph then has two outputs: `output`, shaped `(batch, max_det, 6)`, and `raw`, the undecoded detector tensor that LibreYOLO's own backend uses so postprocessing stays identical to the PyTorch path. ### DeepStream `deepstream=True` is an ONNX-only option. It exports the graph in the layout NVIDIA DeepStream's parser expects and writes two sidecar files next to it, `config_infer_primary_.txt` and `_labels.txt`, so the artifact drops into a pipeline without hand-written configuration. It is mutually exclusive with `nms=True`, and asking for both raises a `ValueError`: DeepStream runs suppression in its own clustering stage. Passing it to any format other than ONNX raises as well. See [DeepStream](/docs/export/deepstream) for the supported family and task grid and the parser build. ### INT8 **INT8 with calibration data** ```python from libreyolo import LibreYOLO LibreYOLO("LibreYOLO9t.pt").export( format="onnx", int8=True, data="coco128.yaml", # a few hundred representative images fraction=1.0, ) ``` `int8=True` runs ONNX Runtime static quantization and writes a QDQ graph with float32 inputs and outputs. Only `Conv` and `Gemm` nodes are quantized. Leaving the detection-head decode in float32 is deliberate: that concatenation mixes pixel-scale box coordinates with class scores in the range 0 to 1, and a single per-tensor activation scale dominated by the box magnitude would drive every score to zero. This flag currently applies to YOLO9 detection only, and anything else raises `NotImplementedError` in preflight. Omitting `data` falls back to `coco8.yaml` with a warning; eight images is not a representative calibration set. A model that was already quantized in PyTorch takes a different route, described on [Quantization](/docs/export/quantization). ## Run the artifact **Through LibreYOLO** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("weights/LibreYOLO9t.onnx") result = model.predict(SAMPLE_IMAGE) print(result.boxes.xyxy[:3]) ``` **Bare ONNX Runtime** ```python import numpy as np import onnx import onnxruntime as ort session = ort.InferenceSession( "weights/LibreYOLO9t.onnx", providers=["CPUExecutionProvider"], ) # Preprocessing and postprocessing are yours on this path. batch = np.zeros((1, 3, 640, 640), dtype=np.float32) outputs = session.run(None, {session.get_inputs()[0].name: batch}) print([out.shape for out in outputs]) # The graph carries the family, task, class names and input size. meta = {p.key: p.value for p in onnx.load("weights/LibreYOLO9t.onnx").metadata_props} print(meta["model_family"], meta["task"], meta["imgsz"]) ``` `LibreYOLO()` dispatches on the `.onnx` suffix and returns the same `Results` object as a `.pt` checkpoint, because the class names, task, input size and pose schema were written into the graph's `metadata_props` at export time. With `device="auto"` the session takes `CUDAExecutionProvider` when ONNX Runtime reports it and falls back to CPU otherwise. The second snippet is for readers with no LibreYOLO installed. Preprocessing, decoding, NMS and coordinate rescaling all become yours on that path; the metadata block is still there to read. ## Constraints Output tensor names are fixed per task, and they are what the metadata-free consumer has to match: | Task | Output names | |---|---| | Detection, grid and anchor heads | `output` | | Detection, DETR-style | `pred_logits`, `pred_boxes` | | Detection, RF-DETR | `dets`, `labels` | | Classification | `output` | | Semantic segmentation | `semantic_logits` | | Depth | `depth` | | Surface normal | `normal` | | Edge | `edges` | | Restoration | `restored` | | Matting | `matte` | | Gaze | `yaw_logits`, `pitch_logits` | RF-DETR is also the one family whose input tensor is named `input` rather than `images`. Several tasks carry a fixed-resolution runtime contract in this version. Depth, surface normal and edge reject `batch != 1` and force `dynamic=False`. Matting forces the native 1024 square, because BiRefNet's Swin relative-position tables are tied to their resolution. Restoration forces a fixed canvas for every family except Real-ESRGAN, whose generator is fully convolutional. Rectangular `imgsz` works for the YOLO9 families, HRNet, NAFNet and Real-ESRGAN. Families with a fixed square contract (`clip`, `deformable_detr`, `detr`, `dinodetr`, `dfine`, `deim`, `deimv2`, `ec`, `lwdetr`, `moge2`, `rtdetr`, `rtdetrv2`, `rtdetrv4`, `rfdetr`, `siglip2`, `ssd`) reject it outright. Two combinations are refused before tracing: YOLO9 segmentation, because YOLO9 is detection only in LibreYOLO, and RTMDet-Ins segmentation, whose dynamic-kernel mask decode has no exported-runtime contract. For the full family and task grid, see [the export matrix](/docs/reference/export-matrix). For one combination, ask the library directly: **Check one family and task before exporting** ```bash libreyolo formats --family yolo9 --task detect ``` --- # OpenVINO OpenVINO IR is Intel's runtime format, a model.xml graph beside a model.bin weight blob. LibreYOLO exports an ONNX intermediate, converts it with ov.convert_model, and writes a metadata.yaml into the same directory. Verified against LibreYOLO v1.5.0. ## Install **Install** ```bash # The IR is converted from an ONNX intermediate, so both extras are needed. pip install "libreyolo[onnx,openvino]" ``` **INT8 additionally needs NNCF** ```bash pip install nncf ``` The conversion goes through an ONNX intermediate, so the `onnx` extra is part of the requirement rather than an optional companion. NNCF is a separate install and is only needed for `int8=True`. ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") # Writes the directory weights/LibreYOLO9t_openvino path = model.export(format="openvino") print(path) ``` **CLI** ```bash libreyolo export --model LibreYOLO9t.pt --format openvino ``` **Arguments** ```python model.export( format="openvino", imgsz=640, batch=1, dynamic=False, # True keeps a dynamic batch axis through the IR half=False, # True stores FP16 weights int8=False, # True runs NNCF post-training quantization data=None, # required when int8=True output_path=None, # None writes weights/_openvino ) ``` The artifact is a directory, not a file. `weights/LibreYOLO9t_openvino` holds `model.xml`, `model.bin` and `metadata.yaml`, and `_fp16` is inserted before the suffix when `half=True`. Move or copy the whole directory; the three files are one artifact. `half=True` sets `compress_to_fp16` on save. That is weight compression in the IR, not a change to the inference precision the device chooses at run time. ### INT8 **INT8 with calibration data** ```python from libreyolo import LibreYOLO LibreYOLO("LibreYOLO9t.pt").export( format="openvino", int8=True, data="coco128.yaml", # required: there is no default for this format fraction=1.0, ) ``` `int8=True` runs NNCF post-training quantization over a LibreYOLO calibration loader with the mixed preset, and `data` is mandatory: this format has no eight-image fallback. Missing NNCF raises an `ImportError` naming the install command. ## Run the artifact **Through LibreYOLO** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("weights/LibreYOLO9t_openvino") result = model.predict(SAMPLE_IMAGE) print(result.boxes.xyxy[:3]) ``` **Select the device** ```python from libreyolo import LibreYOLO # "auto" and "cpu" map to CPU, "gpu" and "cuda" map to GPU, # anything else is passed through uppercased, for example "npu" -> NPU. model = LibreYOLO("weights/LibreYOLO9t_openvino", device="gpu") ``` **Bare OpenVINO** ```python import numpy as np import openvino as ov import yaml core = ov.Core() print(core.available_devices) compiled = core.compile_model("weights/LibreYOLO9t_openvino/model.xml", "CPU") outputs = compiled(np.zeros((1, 3, 640, 640), dtype=np.float32)) print([tensor.shape for tensor in outputs.values()]) # Class names, task and input size live in metadata.yaml beside the IR. meta = yaml.safe_load(open("weights/LibreYOLO9t_openvino/metadata.yaml")) print(meta["model_family"], meta["task"], meta["names"]) # Preprocessing and postprocessing are yours on this path. ``` `LibreYOLO()` recognizes any directory containing `model.xml` and returns the same `Results` object as the checkpoint, reading class names, task, input size and pose schema from `metadata.yaml`. The device string is mapped rather than passed straight through. `auto` and `cpu` both compile for CPU, `gpu` and `cuda` both compile for GPU, and any other value is uppercased and handed to OpenVINO, which is how an NPU target is reached. The third snippet is for readers with no LibreYOLO installed. Preprocessing, decoding, NMS and coordinate rescaling become yours there, and the class names only exist in `metadata.yaml`. ## Constraints An IR without its `metadata.yaml` still loads, but the backend then falls back to 80 classes and the detection task, which is wrong for anything else. Keep the directory intact. Blocked before tracing: YOLO9 segmentation, RTMDet-Ins segmentation, SSD, Faster R-CNN and RetinaNet detection, and BiRefNet or FeyNobg matting, where OpenVINO 2026.2 cannot lower the shared matte decoder's standard ONNX `DeformConv-19` operation. Where a combination is neither validated nor blocked, the converter path is available and the project has not recorded OpenVINO runtime parity for it. Several combinations are validated with an explicit context attached, for example DeepLabV3 semantic segmentation at a fixed 520 by 520 input on OpenVINO 2026.2 with the CPU default inference precision, and L2CS gaze at a fixed 448 by 448 face crop. `libreyolo formats` prints that context per combination. For the full family and task grid, see [the export matrix](/docs/reference/export-matrix). For one combination: **Check one family and task before exporting** ```bash libreyolo formats --family yolo9 --task detect ``` --- # Paddle PaddlePaddle inference models are a model.pdmodel graph beside a model.pdiparams weight file. LibreYOLO exports a static opset-15 ONNX graph, converts it with X2Paddle, and packages the result with a metadata.yaml so it loads through the same factory as every other runtime. Verified against LibreYOLO v1.5.0. ## Install **Install** ```bash # Python 3.10 to 3.12. WSL2 with Ubuntu 22.04 is the validated Windows path. pip install "libreyolo[paddle]" ``` **Confirm the pinned versions** ```bash python -c "from importlib.metadata import version; print(version('paddlepaddle'), version('x2paddle'), version('onnx'))" ``` The extra pins the exact stack the parity work measured: PaddlePaddle 2.6.2, X2Paddle 1.6.0 and ONNX 1.17 or earlier. Those pins are checked at export time, not just at install time, and a different version raises an `ImportError` naming the expected one. Newer Paddle releases reject parts of the static code X2Paddle 1.6.0 generates, so failing early is better than producing an artifact nobody has validated. ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") # Writes the directory weights/LibreYOLO9t_paddle path = model.export(format="paddle") print(path) ``` **CLI** ```bash libreyolo export --model LibreYOLO9t.pt --format paddle ``` **Arguments** ```python model.export( format="paddle", imgsz=640, # int; this family's square canvas batch=1, # any other value raises ValueError dynamic=False, # True raises ValueError simplify=True, # False raises ValueError opset=15, # any other value raises ValueError output_path=None, # None writes weights/_paddle ) ``` Four arguments are fixed rather than defaulted. `dynamic` must be `False`, `batch` must be 1, `simplify` must be `True` for a fully static conversion graph, and `opset` must be 15, which is the ceiling X2Paddle 1.6.0 accepts. Passing anything else raises before tracing. One normalization runs on the intermediate graph. ONNX defines an omitted MaxPool dilation as one, PyTorch writes the explicit all-ones attribute, and X2Paddle 1.6.0 rejects it, so the exporter removes that redundant default and leaves the specified operation unchanged. The artifact is a directory: `model.pdmodel`, `model.pdiparams` and `metadata.yaml`. The Python that X2Paddle generates during conversion is not part of it. ## Run the artifact **Through LibreYOLO** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("weights/LibreYOLO9t_paddle", device="cpu") result = model.predict(SAMPLE_IMAGE) print(result.boxes.xyxy[:3]) ``` **CLI** ```bash libreyolo predict --model weights/LibreYOLO9t_paddle \ --source https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg --device cpu --save ``` **The backend directly** ```python from libreyolo.backends.paddle import PaddleBackend # What LibreYOLO() constructs for a Paddle directory. Same Results # object, no factory routing in between. backend = PaddleBackend("weights/LibreYOLO9t_paddle", device="cpu") result = backend.predict("parkour.jpg") print(result.boxes.xyxy[:3]) ``` **Bare Paddle** ```python import numpy as np import paddle.inference as paddle_infer import yaml directory = "weights/LibreYOLO9t_paddle" config = paddle_infer.Config( f"{directory}/model.pdmodel", f"{directory}/model.pdiparams" ) config.disable_gpu() config.disable_mkldnn() config.switch_ir_optim(False) predictor = paddle_infer.create_predictor(config) handle = predictor.get_input_handle(predictor.get_input_names()[0]) handle.reshape([1, 3, 640, 640]) handle.copy_from_cpu(np.zeros((1, 3, 640, 640), dtype=np.float32)) predictor.run() for name in predictor.get_output_names(): print(name, predictor.get_output_handle(name).copy_to_cpu().shape) meta = yaml.safe_load(open(f"{directory}/metadata.yaml")) print(meta["model_family"], meta["task"], meta["names"]) # Preprocessing and postprocessing are yours on this path. ``` `LibreYOLO()` recognizes any directory holding both `model.pdmodel` and `model.pdiparams`, reads `metadata.yaml`, and returns the same `Results` object as the checkpoint. A device other than `auto` or `cpu` raises: this backend is CPU only. What the factory constructs is `PaddleBackend`, exported from `libreyolo` and importable as `libreyolo.backends.paddle.PaddleBackend`. Construct it yourself when you want the backend without the factory's suffix routing, for example to pass `task=` explicitly for a directory whose `metadata.yaml` you did not write. Its `predict()` takes the same sources and returns the same results. The bare-runtime snippet mirrors what the backend configures, and the three disabled options are deliberate. The Paddle 2.6 CPU fusion pipeline can crash while optimizing the large gather and scatter graphs emitted for deformable attention, so the portable unfused static graph is the one parity was measured against. Preprocessing, decoding, NMS and coordinate rescaling become yours on that path. ## Constraints No dynamic shapes, no FP16, no INT8, no embedded NMS, no GPU runtime. Validated combinations are YOLO9 detection, YOLO9-E2E and YOLO9-P2 detection, EC detection, pose and segmentation, RT-DETRv4, D-FINE, DEIM and DEIMv2 detection, and YOLO-NAS detection and pose. Each is covered by conversion, a CPU runtime reload, raw-output parity and matched public results. Blocked, with the reason recorded per combination: | Combination | Why | |---|---| | RF-DETR, all tasks | Needs ONNX opset 17 and GridSample; X2Paddle 1.6.0 accepts opset 15 or lower and has no GridSample mapper | | RT-DETR and RT-DETRv2 detection | The trained graphs need GridSample at opset 16 or newer | | D-FINE segmentation | Converts and reloads, but mask-logit relative RMS error is 3.52% and minimum matched-mask IoU is 0.582 | | YOLO9 segmentation | YOLO9 is detection only in LibreYOLO | | RTMDet-Ins segmentation | The dynamic-kernel mask decode has no exported-runtime contract | Anything not listed as validated or blocked is refused with the note that it has not been validated through the ONNX-to-Paddle conversion path. For the full family and task grid, see [the export matrix](/docs/reference/export-matrix). For one combination: **Check one family and task before exporting** ```bash libreyolo formats --family yolo9 --task detect ``` --- # Quantization Quantization in LibreYOLO runs entirely in PyTorch: model.quantize() swaps a model's Conv2d and Linear modules for quantized equivalents and calibrates them. The result keeps the ordinary predict, val, train and save contract, so a quantized model is scored by the same validators as a float one. Verified against LibreYOLO v1.5.0. ## Install Quantization needs no extra. The module swap, the calibration pass and the simulated arithmetic all run in PyTorch, so `pip install libreyolo` is the whole requirement. The deployment artifacts need whatever their own format needs, which for the ONNX path is `libreyolo[onnx]`. ## Quantize **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") # Structure swap plus calibration. calib is a small UNLABELED image set, # read forward-only to derive activation ranges and scales. qmodel = model.quantize(recipe="int8", calib="coco128.yaml", samples=128) print(qmodel.quant_info()) qmodel.val(data="coco8.yaml") # same validators as a float model qmodel.save("LibreYOLO9s-int8.pt") # checkpoint carries a quant manifest ``` **CLI** ```bash libreyolo quantize --model LibreYOLO9s.pt --recipe int8 --calib coco128.yaml ``` **Arguments** ```python model.quantize( recipe="int8", calib="coco128.yaml", # data.yaml path or built-in name; None skips calibration samples=128, # maximum calibration images batch=8, # calibration batch size algorithm="auto", # auto and minmax are the same; percentile is the alternative keep_high_precision=None, # None uses the family policy verbose=True, ) ``` `quantize()` transforms the loaded model in place and returns it. No gradients are involved: the swap installs quantized modules and the calibration pass runs forward only. The resulting checkpoint is an ordinary LibreYOLO checkpoint with a `quant` manifest attached, so it reloads with its structure and scales intact: **A quantized checkpoint reloads as one** ```python from libreyolo import LibreYOLO # The quant manifest rebuilds the quantized structure and scales # before the weights are loaded. qmodel = LibreYOLO("LibreYOLO9s-int8.pt") print(qmodel.quant_info()) ``` Trainer checkpoints written during a QAT run carry the manifest too, which means `best.pt` from such a run is itself a quantized checkpoint. ## Recipes Four families are supported: `yolo9`, `rfdetr`, `birefnet` and `feynobg`. | Recipe | What it does | Families | Calibration | |---|---|---|---| | `fp16` | Cast to half precision with a float32 input and output contract. Inference only. | all four | none | | `bf16` | Cast to bfloat16, which keeps float32's exponent range. The fix when fp16 overflows on a DETR-style model. Inference only. | all four | none | | `fp8` | E4M3 weights and activations on `Conv2d` and `Linear`: per-channel weight scales, calibrated per-tensor activation scales. | all four | required | | `int8` | W8A8 on `Conv2d` and `Linear`: per-channel symmetric weights, per-tensor affine activations. | all four | required, or `calib=None` for weights only | | `w4a16` | Grouped symmetric INT4 weights, group 128 along `in_features`, float activations, on `Linear`. | rfdetr, birefnet, feynobg | not needed | | `w4a8` | Grouped INT4 weights plus calibrated INT8 activations, on `Linear`. | rfdetr, birefnet, feynobg | required | | `nvfp4` | W4A4 NVFP4 on `Linear`: E2M1 elements, 16-element blocks, FP8 E4M3 block scales, FP32 tensor scale. Dynamic activation scaling. | rfdetr, birefnet, feynobg | not needed | | `mxfp4` | OCP MXFP4 on `Linear`: E2M1 elements, 32-element blocks, power-of-two E8M0 block scales. Dynamic activation scaling. | rfdetr, birefnet, feynobg | not needed | | `int2` | Research only: grouped 2-bit weights, group 64, plus INT8 activations, on `Linear`. Post-training alone is unusable, so QAT or QAD is required. | rfdetr | required | The sub-8-bit recipes target `nn.Linear` and are rejected for `yolo9` on purpose: that acceleration is GEMM only on current hardware, so convolutions stay in higher precision. YOLO9 uses `int8` or `fp8`. `int2` is rejected for `birefnet` and `feynobg` because those families are inference only, so the QAT healing the recipe depends on is unavailable there. Per-family defaults keep the first layer and the heads in float, and the YOLO9 DFL convolution is never quantized: it is a fixed integral-expectation operator. Override with `keep_high_precision=("head.",)` when you have a reason to. ## Calibration data is not training data `calib=` takes a few hundred images, reads no labels, and runs forward only to estimate activation ranges. `data=` in `train()` and `val()` is the labeled dataset used for gradients and metrics. They are different arguments with different purposes, and the default for `calib` is `coco128.yaml`. `algorithm="minmax"` keeps the absolute extremes seen across calibration batches and is what `"auto"` selects. `"percentile"` uses the mean of per-batch 0.1 and 99.9 percentiles; it was measured to collapse DETR-family accuracy, because transformer activation outliers are load-bearing. What actually fixes small-model INT8 sensitivity is calibrating on enough batches: with the `coco128` default, YOLO9-t lands within about one mAP point of its float score. The chosen algorithm is recorded in the checkpoint manifest. ## Recover accuracy **QAT is plain train() on a quantized model** ```python from libreyolo import LibreYOLO qmodel = LibreYOLO("LibreYOLO9s-int8.pt") # A finetune, not a from-scratch run: use finetune learning rates. qmodel.train(data="coco8.yaml", epochs=5, lr0=1e-4) ``` **QAD adds the existing distillation arguments** ```python qmodel.train( data="coco8.yaml", epochs=5, lr0=1e-4, distill_model="LibreYOLO9m.pt", ) ``` **CLI** ```bash libreyolo train --model LibreYOLO9s-int8.pt --data coco8.yaml --epochs 5 --lr0 1e-4 ``` Quantized modules keep fp32 master weights and apply fake quantization with a straight-through estimator, so gradients reach the masters and the existing trainers work unchanged: EMA, AMP, checkpoint resume and the distillation arguments all compose. QAT is a finetune of an already-trained model. Use finetune learning rates rather than the from-scratch defaults, or a short run will destroy the pretrained weights regardless of quantization. QAD availability follows family distillation support, which today means `yolo9` and `rfdetr`. `fp16`- and `bf16`-quantized models are inference only, and the trainer rejects them with a pointer to `amp=True`. ## Export **Packed PyTorch checkpoint** ```python from libreyolo import LibreYOLO qmodel = LibreYOLO("LibreYOLO9s-int8.pt") # Writes LibreYOLO9s-int8-final.pt: packed low-bit weights and scales, # fp32 masters stripped, the non-quantized remainder cast to fp16. qmodel.export(format="pt") # remainder="fp32" keeps the non-quantized tensors exact. qmodel.export(format="pt", remainder="fp32") ``` **QDQ INT8 ONNX** ```python from libreyolo import LibreYOLO qmodel = LibreYOLO("LibreYOLO9s-int8.pt") # In-graph QuantizeLinear/DequantizeLinear pairs carrying the model's # own calibrated or QAT-trained scales. qmodel.export(format="onnx") ``` **CLI** ```bash libreyolo export --model LibreYOLO9s-int8.pt --format onnx ``` `format="pt"` crystallizes the model. Packed low-bit weights and scales replace the masters, and the non-quantized remainder is cast to fp16 unless `remainder="fp32"` is passed. The packing invariant is that unpacking reproduces the simulation bit for bit on the device you finalized on, so the finalized file scores exactly what you validated. Measured: YOLO9-s int8 goes from 29.5 MB to 9.6 MB, RF-DETR-n nvfp4 from 122 MB to 26 MB. Loading one gives an inference-ready model, and calling `train()` on it reconstructs the masters from the packed weights automatically. `format="onnx"` applies to `int8` models and emits a QDQ graph carrying the model's own calibrated or QAT-trained scales, which ONNX Runtime and TensorRT run with real INT8 kernels. This is a different path from [`export(format="onnx", int8=True)`](/docs/export/onnx) on a float model, where ONNX Runtime derives the scales itself. The cast recipes need no quantized exporter at all: **Back to float, keeping QAT-trained weights** ```python from libreyolo import LibreYOLO qmodel = LibreYOLO("LibreYOLO9s-int8.pt") qmodel.dequantize() # Any float exporter now applies, at any precision it supports. qmodel.export(format="tensorrt", half=True) ``` ## Constraints Quantized arithmetic executes in simulation, which is fake quantization computed in float32 islands even under AMP. Simulation is numerics-true, so a `val()` score on any device is a real claim about the quantized arithmetic. It is not a speed claim. Two exceptions execute natively. `fp16` and `bf16` are ordinary casts. Finalized `fp8` modules run their GEMM directly on packed E4M3 weights through `torch._scaled_mm` on Ada, Hopper and Blackwell class hardware, using the same calibrated activation scales as the simulation; setting `LIBREYOLO_KERNELS=off` restores the exact simulated path everywhere. Deployment coverage is narrower than the recipe list. Only `int8` has a deployable ONNX form here; `fp8` and the sub-8-bit linear recipes execute in PyTorch and crystallize through `format="pt"`. Requesting an ONNX export from them raises with that instruction, as does requesting any non-ONNX format from an `int8` model: build downstream engines from the QDQ graph instead. Exporting an `int8` model whose activations were never calibrated logs a warning and produces a graph carrying weight quantization only. --- # RKNN RKNN is Rockchip's compiled NPU format. LibreYOLO exports an opset-19 ONNX intermediate, compiles it with the RKNN Toolkit2 SDK, and can compare the compiled graph against ONNX Runtime in Toolkit2's host simulator without a board. Verified against LibreYOLO v1.5.0. ## Install Compilation needs Rockchip's RKNN Toolkit2, which is distributed as a vendor SDK under Rockchip's own license and is not a LibreYOLO dependency. There is no `libreyolo[rknn]` extra, and nothing about this format installs from a single line. **LibreYOLO side** ```bash pip install "libreyolo[onnx]" ``` **Vendor SDK, installed by you** ```bash # rknn-toolkit2 is a Rockchip SDK under a separate license. LibreYOLO # neither bundles nor installs it. x86_64 Linux only; on Windows use # WSL2 or a Linux container. # # Toolkit2 2.3.2 needs setuptools<81 and fails on ONNX 1.19 or newer, # whose removal of onnx.mapping its compiler still imports. pip install "setuptools==80.9.0" "onnx==1.18.0" # Then install the matching rknn-toolkit2 wheel from Rockchip's own # wheel repository, and confirm it imports: python -c "import rknn.api; print('rknn-toolkit2 ready')" ``` A board is not needed to compile or to check numerical parity. An RK3588 board is needed for latency, power and thermal measurements, none of which have been recorded. ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") # Writes weights/LibreYOLO9t.rknn and weights/LibreYOLO9t.rknn.metadata.json path = model.export(format="rknn", name="rk3588", imgsz=640, verify=True) print(path) ``` **CLI** ```bash libreyolo export --model LibreYOLO9t.pt --format rknn --name rk3588 \ --imgsz 640 --verify ``` **Arguments** ```python model.export( format="rknn", name="rk3588", # target platform; target= and target_platform= also work imgsz=640, # must match the variant's recorded canvas batch=1, # any other value raises NotImplementedError dynamic=False, # True raises ValueError opset=19, # any other value raises NotImplementedError verify=False, # True runs the PC simulator and gates on parity ) ``` The request is validated against a list of exact model variants before anything is compiled, and the canvas is validated too: passing an `imgsz` other than the one the variant was recorded at raises rather than silently compiling something untested. LibreYOLO writes an opset-19 ONNX intermediate, compiles it, optionally simulates it, and removes the intermediate afterwards. Metadata is a sidecar named `.rknn.metadata.json`, because the RKNN format has no portable metadata field. `verify=True` runs Toolkit2's PC simulator inside the same session that compiled the artifact, compares every output against ONNX Runtime on the same input, and writes `.rknn.parity.json` with per-output error metrics. The gates are cosine similarity of at least 0.9999 and normalized RMSE of at most 0.02, applied to any output that is not already elementwise close; the vendor floating build lowers internal tensors to half precision, so strict `allclose` does not hold even when the decoded boxes are stable. A failing run writes `.rknn.failed.parity.json`, discards the candidate, and leaves any earlier successful export at that path untouched. To compare an ONNX artifact you already have, without exporting again: **Board-free parity against an existing ONNX artifact** ```python import numpy as np from libreyolo.export import verify_rknn_simulator_parity input_tensor = np.random.default_rng(0).standard_normal( (1, 3, 640, 640), dtype=np.float32 ) metrics = verify_rknn_simulator_parity( "weights/LibreYOLO9t.onnx", input_tensor, target_platform="rk3588", rtol=1e-3, atol=1e-4, raise_on_failure=False, ) print(metrics) ``` Toolkit2's simulator runs the in-memory graph produced by `load_onnx` and `build`. It cannot reload a target-specific `.rknn` file without a board, which is why `verify=True` does compilation, export and simulation in one session. ## Run the artifact There is no RKNN entry in `libreyolo/backends`, so `LibreYOLO()` does not load a `.rknn` file. The compiled artifact is deployed to the board and executed by Rockchip's own runtime, and preprocessing, decoding, NMS and coordinate rescaling are the application's responsibility there. `.rknn.metadata.json` carries the class names, input size, task and target platform, which is what an application needs to reproduce LibreYOLO's postprocessing. Ship it alongside the compiled model. For a host-side check that does not need the board, keep an ONNX artifact at the same fixed shape and compare it in the simulator, as above. ## Constraints Four combinations compile, and they are model variants rather than families: | Variant | Task | Canvas | Target | |---|---|---:|---| | YOLO9-t | detect | 640 | RK3588 | | YOLO9-E2E-t | detect | 640 | RK3588 | | PicoDet-s | detect | 320 | RK3588 | | YOLO-NAS-s | detect | 640 | RK3588 | Everything else is refused before compilation, with the message that RKNN in this version is limited to the exact simulator-tested detection variants. Compile-only results for other models exist but are deliberately not presented as support: on the same measurement run, RF-DETR left two decoder `GridSample` nodes unlowered, and D-FINE, RT-DETR, RT-DETRv2, RT-DETRv4, DEIM, DEIMv2 and EC compiled and simulated with decoded outputs that were materially wrong. Batch 1, static shapes, opset 19. `half=True` is rejected, because RKNN does not expose LibreYOLO's `half` contract, and `int8=True` is rejected until representative calibration and task-accuracy results exist. Other Rockchip targets are rejected: `rk3588` is the only validated platform. For the full family and task grid, see [the export matrix](/docs/reference/export-matrix). For one combination: **Check one family and task before compiling** ```bash libreyolo formats --family yolo9 --task detect ``` --- # TensorRT TensorRT compiles a graph into an engine tuned for one GPU. LibreYOLO exports an ONNX intermediate first, parses it with TensorRT's ONNX parser, builds the engine, and writes the model metadata beside it as a JSON sidecar. Verified against LibreYOLO v1.5.0. ## Install Both the build and the run need an NVIDIA GPU with a working CUDA stack. There is no CPU fallback for this format. **Install** ```bash # The engine is built from an ONNX intermediate, so both extras are needed. pip install "libreyolo[onnx,tensorrt]" ``` **Confirm the toolchain before building** ```bash python -c "import tensorrt, torch; print(tensorrt.__version__, torch.cuda.is_available())" ``` The `tensorrt` extra pins `tensorrt-cu12` and `pycuda`, and the marker drops both on macOS. On a Jetson, do not use that extra: it pins a CUDA 12 build against a CUDA 13 platform. Use the TensorRT that JetPack installs instead, as described on [NVIDIA Jetson](/docs/export/jetson). ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") # Writes weights/LibreYOLO9t_fp16.engine and weights/LibreYOLO9t_fp16.engine.json path = model.export(format="tensorrt", half=True) print(path) ``` **CLI** ```bash libreyolo export --model LibreYOLO9t.pt --format tensorrt --half ``` **Arguments** ```python model.export( format="tensorrt", imgsz=640, batch=1, half=False, int8=False, data=None, # required when int8=True dynamic=False, workspace=4.0, # GiB of build-time scratch min_batch=1, # dynamic profile bounds opt_batch=1, max_batch=8, hardware_compatibility="none", # or "ampere_plus" gpu_device=0, # build device on a multi-GPU host verbose=False, ) ``` The export runs in two steps. Step one writes an ONNX intermediate to a temporary path, step two parses it and builds the engine, and the intermediate is removed afterwards. `workspace` is build-time scratch memory in GiB; a larger value lets the builder try more kernels and does not affect inference memory. The metadata sidecar is written next to the engine as `.json` and records the precision the build actually realized. When the GPU lacks fast FP16 or fast INT8 the builder warns and falls back, and the sidecar reports the precision that came out rather than the one that was asked for. Under FP16, a ViT backbone in the graph is detected and its float layers are pinned to FP32. DINOv2-style backbones overflow in FP16 and produce NaN, so the build sets `OBEY_PRECISION_CONSTRAINTS` and reports `FP16 (FP32 ViT backbone)`. The pass is a no-op on CNN backbones. ### Dynamic batch **Dynamic batch engine** ```python from libreyolo import LibreYOLO # The ONNX intermediate needs the dynamic batch axis for the profile # to have anything to bind to. LibreYOLO("LibreYOLO9t.pt").export( format="tensorrt", dynamic=True, min_batch=1, opt_batch=4, max_batch=8, half=True, ) ``` `dynamic=True` adds one optimization profile spanning `min_batch` to `max_batch`, optimized at `opt_batch`, and records those three values in the sidecar. The profile is only added when the ONNX intermediate actually carries a dynamic batch dimension; otherwise the build logs that it is using static optimization and continues. ### INT8 **INT8 with calibration data** ```python from libreyolo import LibreYOLO LibreYOLO("LibreYOLO9t.pt").export( format="tensorrt", int8=True, data="coco128.yaml", # required: there is no default for this format fraction=1.0, ) ``` INT8 uses TensorRT's entropy calibrator over a LibreYOLO calibration loader, and `data` is mandatory: this format has no eight-image fallback. Calibration needs `cuda-python` or `pycuda` for the device buffer. The calibration cache is keyed on a hash of the ONNX bytes, so scales from one model are never reused for another that happens to write to the same output path. `half=True` and `int8=True` together warn and build INT8, which keeps an FP16 fallback for layers TensorRT cannot quantize. ## Run the artifact **Through LibreYOLO** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("weights/LibreYOLO9t_fp16.engine") result = model.predict(SAMPLE_IMAGE) print(result.boxes.xyxy[:3]) ``` **Bare TensorRT** ```python import json import tensorrt as trt path = "weights/LibreYOLO9t_fp16.engine" runtime = trt.Runtime(trt.Logger(trt.Logger.WARNING)) with open(path, "rb") as handle: engine = runtime.deserialize_cuda_engine(handle.read()) for i in range(engine.num_io_tensors): name = engine.get_tensor_name(i) print(engine.get_tensor_mode(name), name, engine.get_tensor_shape(name)) # Class names, task and input size live in the sidecar, not the engine. # Buffer allocation, preprocessing and postprocessing are yours here. print(json.load(open(path + ".json"))["names"]) ``` `LibreYOLO()` dispatches on the `.engine` suffix, reads the sidecar for class names, task and pose schema, and returns the same `Results` object as the checkpoint. It raises immediately when no CUDA device is present. The second snippet is the bare-runtime path. Host and device buffer allocation, preprocessing, decoding, NMS and coordinate rescaling all become yours, and the engine itself carries no class names, so the sidecar has to travel with it. ## Constraints A serialized engine is tied to the GPU architecture, the driver stack and the TensorRT version that built it. An engine built on a workstation will not load on a different architecture, which is why the build step runs on the deployment machine. `hardware_compatibility="ampere_plus"` trades some performance for portability across Ampere and newer. The `"same_compute_capability"` value maps to `NONE` and warns: the engine is optimized for the current GPU only, and the export says so rather than claiming a portability it did not apply. Only the batch axis is profiled. A build with dynamic spatial dimensions is not part of this contract, which is why FCOS is blocked: it needs dynamic padded height and width to preserve its 800 by 1333 aspect transform. Blocked before tracing: YOLO9 segmentation, RTMDet-Ins segmentation, SSD, Faster R-CNN and RetinaNet detection, and BiRefNet or FeyNobg matting, where TensorRT 10.16 reaches the shared ONNX `DeformConv` node and cannot parse it because `ModulatedDeformConv2d` is absent from the plugin registry. Where a combination is neither validated nor blocked, the converter path is available and the project has not recorded TensorRT runtime parity for it. That is a statement about evidence, not about whether the build succeeds. For the full family and task grid, see [the export matrix](/docs/reference/export-matrix). For one combination: **Check one family and task before building** ```bash libreyolo formats --family yolo9 --task detect ``` --- # TFLite TFLite is the FlatBuffer format LiteRT executes on mobile and embedded targets. LibreYOLO exports a static ONNX graph, converts it with onnx2tf in flatbuffer-direct mode, and writes the model metadata beside the artifact as a JSON sidecar. Verified against LibreYOLO v1.5.0. ## Install **Install** ```bash # LiteRT is Google's current name for TensorFlow Lite. Both extras # install the same toolchain and produce the same .tflite output. pip install "libreyolo[tflite]" ``` **Confirm the Python version first** ```bash python -c "import sys; print(sys.version_info >= (3, 12))" ``` The extra pulls `onnx2tf` for the conversion and `ai-edge-litert` for running the result, both behind a Python 3.12 marker. On an older interpreter the export raises an `ImportError` that names the version requirement rather than failing inside the converter. `libreyolo[litert]` installs exactly the same thing. The format string `litert` is an alias for `tflite`, and the output file is a `.tflite` either way. ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") # Writes weights/LibreYOLO9t.tflite and weights/LibreYOLO9t.tflite.json path = model.export(format="tflite", imgsz=640) print(path) ``` **CLI** ```bash libreyolo export --model LibreYOLO9t.pt --format tflite --imgsz 640 # "litert" is accepted as an alias and resolves to the same exporter. libreyolo export --model LibreYOLO9t.pt --format litert --imgsz 640 ``` **Arguments** ```python model.export( format="tflite", imgsz=640, # int, or (height, width) batch=1, simplify=True, # onnxsim over the ONNX intermediate output_path=None, # None writes weights/.tflite verbose=False, # True streams the onnx2tf log ) # dynamic=True raises ValueError: the converter needs static shapes. # half=True and int8=True are rejected before tracing. ``` The family and task are checked before anything else happens, so an unsupported combination fails immediately with the specific converter or runtime error that kept it out, not a generic message. The conversion itself is a subprocess call to `onnx2tf` in `flatbuffer_direct` mode over a static ONNX intermediate. Metadata is a sidecar. `weights/LibreYOLO9t.tflite.json` carries the family, task, class names, input size and pose schema; the FlatBuffer itself has no LibreYOLO metadata field, so the two files travel together. ## Run the artifact **Through LibreYOLO** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("weights/LibreYOLO9t.tflite") result = model.predict(SAMPLE_IMAGE) print(result.boxes.xyxy[:3]) ``` **Bare LiteRT** ```python import json import numpy as np from ai_edge_litert.interpreter import Interpreter interpreter = Interpreter(model_path="weights/LibreYOLO9t.tflite") interpreter.allocate_tensors() detail = interpreter.get_input_details()[0] print(detail["shape"], detail["dtype"]) # NHWC, not NCHW interpreter.set_tensor(detail["index"], np.zeros(detail["shape"], np.float32)) interpreter.invoke() for output in interpreter.get_output_details(): print(output["name"], interpreter.get_tensor(output["index"]).shape) # Class names, task and input size live in the sidecar. meta = json.load(open("weights/LibreYOLO9t.tflite.json")) print(meta["model_family"], meta["task"], meta["names"]) # Preprocessing, the NCHW-to-NHWC transpose and postprocessing are yours. ``` `LibreYOLO()` dispatches on the `.tflite` suffix and returns the same `Results` object as the checkpoint. The backend reads the sidecar, transposes the NCHW blob to NHWC when the interpreter asks for a channels-last input, applies the interpreter's quantization scale and zero point where present, and transposes outputs back into the layout LibreYOLO's postprocessing expects. The second snippet is the bare-runtime path. Preprocessing, the layout transpose, decoding, NMS and coordinate rescaling all become yours there, and the layout detail is the one most likely to be missed: onnx2tf emits channels-last inputs, so a blob shaped `(1, 3, 640, 640)` will not bind. ## Constraints Static shapes only. `dynamic=True` raises `ValueError` before tracing, and the export canvas is fixed at whatever `imgsz` resolved to. FP32 only. `half=True` and `int8=True` are both rejected during validation, so quantized deployment is not reachable from this exporter today. Coverage is narrower here than for the graph formats, and it is decided by measurement rather than by family. Validated combinations include YOLO9, YOLOX and YOLO-NAS detection, PIDNet semantic segmentation, the four CNN classification families, DINOv2 and SigLIP2 embedding, SigLIP2 classification, TEED and DexiNed edge, and Real-ESRGAN and SwinIR restoration. SwinIR carries an extra caveat: parity holds when the source dimensions match the export canvas exactly, and smaller sources are padded to the canvas before the transformer runs, which can diverge from native variable-size inference. The blocked entries name the exact failure, which is worth reading before attempting a workaround. A few examples: RF-DETR detection converts at its native 384 canvas but LiteRT cannot allocate it because `STRIDED_SLICE` receives an input above its supported 5-D rank; PicoDet is rejected because a `RESHAPE` maps 19,200 input elements to 9,600 output elements; D-FINE crashes the converter in `GatherElements` shape handling; RTMDet exports and reloads with raw parity intact but public boxes fall to 0.911 IoU with 29.9 px of coordinate drift. For the full family and task grid, see [the export matrix](/docs/reference/export-matrix). For one combination, including the reason string behind a block: **Check one family and task before exporting** ```bash libreyolo formats --family yolo9 --task detect ``` --- # TorchScript TorchScript is PyTorch's own serialized-graph format. LibreYOLO traces the model with torch.jit.trace and saves the result together with a libreyolo_metadata.json extra file, so the archive carries the family, task, class names and input size. Verified against LibreYOLO v1.5.0. ## Install **Install** ```bash pip install libreyolo ``` TorchScript needs nothing beyond the base install, because `torch.jit` ships with PyTorch. It is the one export target with no optional dependency and no external converter, which makes it a useful first check when a longer toolchain fails. ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") # Writes weights/LibreYOLO9t.torchscript path = model.export(format="torchscript") print(path) ``` **CLI** ```bash libreyolo export --model LibreYOLO9t.pt --format torchscript ``` **Arguments** ```python model.export( format="torchscript", imgsz=640, # int, or (height, width) batch=1, half=False, # FP16 weights and activations device=None, # None traces on CPU for this format output_path=None, # None writes weights/.torchscript ) # dynamic is accepted but the archive is always a fixed-shape trace, # and the embedded metadata records dynamic=False either way. ``` Tracing runs on CPU unless a device is named, and the archive is written to `weights/` under the checkpoint's stem when `output_path` is omitted. The retrace check that `torch.jit.trace` normally performs is turned off. Several export wrappers cache shape-dependent anchors during their first forward pass, so a second trace observes a different Python path even though the recorded fixed-shape graph is correct. Parity tests validate the saved module directly instead. Metadata does not live in a sidecar. `torch.jit.save` stores `libreyolo_metadata.json` inside the archive, and `torch.jit.load` hands it back through `_extra_files`. ## Run the artifact **Through LibreYOLO** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("weights/LibreYOLO9t.torchscript") result = model.predict(SAMPLE_IMAGE) print(result.boxes.xyxy[:3]) ``` **Bare PyTorch** ```python import json import torch extra_files = {"libreyolo_metadata.json": ""} module = torch.jit.load( "weights/LibreYOLO9t.torchscript", map_location="cpu", _extra_files=extra_files, ) module.eval() metadata = json.loads(extra_files["libreyolo_metadata.json"]) print(metadata["model_family"], metadata["task"], metadata["imgsz"]) # Preprocessing and postprocessing are yours on this path. with torch.no_grad(): out = module(torch.zeros(1, 3, 640, 640)) print(out.shape if torch.is_tensor(out) else [t.shape for t in out]) ``` `LibreYOLO()` routes on the `.torchscript` suffix and returns the same `Results` object as the checkpoint it came from. With `device="auto"` the module is mapped to CUDA when available, then MPS, then CPU. The second snippet is the path for a reader with no LibreYOLO installed, and for C++ deployment through libtorch, where the same archive loads with `torch::jit::load`. Preprocessing, decoding, NMS and coordinate rescaling become yours there. The metadata extra file is still readable, and it is the only place the class names exist. ## Constraints The graph is a trace at one input shape. `dynamic=True` is accepted for interface symmetry but changes nothing, and the embedded metadata reports `dynamic=False` so a backend never assumes an axis it cannot use. Export a second archive for a second resolution. `half=True` casts the model and the trace input to FP16. There is no INT8 path: `int8=True` raises `NotImplementedError` during validation. Rectangular `imgsz` works for the YOLO9 families, HRNet, NAFNet and Real-ESRGAN, and is rejected for families with a fixed square contract. Five combinations are refused before tracing. YOLO9 segmentation, because YOLO9 is detection only in LibreYOLO. RTMDet-Ins segmentation, whose dynamic-kernel mask decode has no exported-runtime contract. SSD, Faster R-CNN and RetinaNet detection, whose variable-length or dynamic-anchor graphs have parity evidence only through the ONNX Runtime contract. For the full family and task grid, see [the export matrix](/docs/reference/export-matrix). For one combination: **Check one family and task before exporting** ```bash libreyolo formats --family yolo9 --task detect ``` --- # Triton Inference Server Triton Inference Server hosts a model repository and answers inference requests over HTTP. LibreYOLO exports the ONNX graph, generates a config.pbtxt that carries the export metadata as one Triton parameter, and treats a model URL as a loadable model path. Verified against LibreYOLO v1.5.0. ## Install **Install** ```bash pip install "libreyolo[onnx,triton]" ``` The `triton` extra installs `tritonclient[http]`. gRPC and shared-memory extras are excluded on purpose: this integration is HTTP and HTTPS V2 inference only. `onnx` is needed because the served artifact and the config generator both work from an ONNX graph. ## Build the model repository Export with a dynamic batch axis, into the directory layout Triton expects. **Export into the repository layout** ```python from pathlib import Path from libreyolo import LibreYOLO model_dir = Path("triton_repo/yolo9/1") model_dir.mkdir(parents=True, exist_ok=True) LibreYOLO("LibreYOLO9t.pt").export( format="onnx", output_path=str(model_dir / "model.onnx"), dynamic=True, simplify=False, ) ``` **Generate config.pbtxt** ```python from libreyolo import create_triton_config create_triton_config( "triton_repo/yolo9/1/model.onnx", "triton_repo/yolo9/config.pbtxt", model_name="yolo9", max_batch_size=8, ) ``` **Resulting layout** ```text triton_repo/ yolo9/ config.pbtxt 1/ model.onnx ``` Triton does not preserve ONNX custom metadata in its model-config response, so the complete exported metadata has to travel some other way. `create_triton_config` encodes it as one JSON string parameter named `libreyolo_metadata` in `config.pbtxt`, emits the input and output declarations in graph order, handles the JSON escaping, and pins the model to `KIND_CPU`. The helper validates before writing. It requires exactly one ONNX graph input, at least one output, resolvable tensor shapes, and metadata whose `names` map defines every class index from 0 to `nc - 1`. A model that fails any of those checks is rejected at config time rather than at the first request. `max_batch_size: 8` matches a dynamic export and lets the server batch up to eight images per request. For a fixed batch-1 ONNX graph use `max_batch_size=0`; LibreYOLO then sends images sequentially. ## Start the server **Start the server** ```bash docker run --rm --name libreyolo-triton \ -p 8000:8000 -p 8002:8002 \ -v "$(pwd)/triton_repo:/models:ro" \ nvcr.io/nvidia/tritonserver:26.04-py3 \ tritonserver --model-repository=/models --exit-on-error=true ``` **Wait for readiness** ```bash until curl --fail --silent http://127.0.0.1:8000/v2/health/ready; do sleep 1; done ``` **Stop it** ```bash docker stop libreyolo-triton ``` The commands pin Triton Server 26.04 and deliberately omit Docker GPU flags, since `KIND_CPU` in the generated config prevents GPU placement anyway. ## Run the artifact A Triton model URL is a model path. `LibreYOLO()` checks for an `http` or `https` scheme before any local path handling and returns a backend that speaks to the server, so the call site is identical to a local checkpoint and so is the `Results` object that comes back. **Predict against the served model** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE remote = LibreYOLO("http://127.0.0.1:8000/yolo9") result = remote.predict(SAMPLE_IMAGE) print(result.boxes.xyxy[:3]) ``` **Compare with the local model** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE remote = LibreYOLO("http://127.0.0.1:8000/yolo9").predict(SAMPLE_IMAGE) native = LibreYOLO("LibreYOLO9t.pt").predict(SAMPLE_IMAGE) print(len(remote.boxes), len(native.boxes)) print(remote.boxes.xyxy[:3]) print(native.boxes.xyxy[:3]) ``` **Pin a version, or change the timeout** ```python from libreyolo import LibreYOLO from libreyolo.backends.triton import TritonBackend # A second path segment selects the model version. Without it, # Triton's configured version policy chooses. pinned = LibreYOLO("http://127.0.0.1:8000/yolo9/1") # Connection and network timeouts default to 30 seconds. patient = TritonBackend("http://127.0.0.1:8000/yolo9", timeout=120) ``` The URL form is `http(s)://host:port/model` with an optional version segment. The port must be explicit. Embedded credentials, a query string and a fragment are all rejected, as is a path with more than two segments. `device` is accepted and ignored with a log line, because placement is the server's decision. ## Constraints The backend fails with a direct error rather than a degraded result when the contract is not met: missing LibreYOLO metadata in the model config, more than one model input, a mismatch between the configured outputs and the model metadata, an input datatype it does not support, or a server or model that is not ready. Outside the contract in this version: gRPC, authentication, shared memory, and loading or unloading models through the API. Any format Triton itself supports can be served, but the metadata parameter and the generated config are ONNX-shaped here, so the LibreYOLO path is [ONNX](/docs/export/onnx) into the repository. For a full video pipeline rather than a request-response server, see [DeepStream](/docs/export/deepstream). --- # FAQ Answers to questions that are not specific to one model family. Anything family-specific lives on that family's page. Verified against LibreYOLO v1.5.0. ## Which model should I start with? YOLOv9 for a CNN detector and RF-DETR for a transformer one. Both sit in the flagship tier, which means features are designed and GPU-validated against them before anything else. See [YOLOv9](/docs/models/yolov9) and [RF-DETR](/docs/models/rf-detr), or [all models](/docs/models) for the rest. ## Do I need a GPU? No. Every model runs on CPU, and everything in the [quickstart](/docs/quickstart) is written to run there. A GPU changes how long training and video inference take, not whether they work. ## How does LibreYOLO choose a device? The default is `device="auto"`, which uses CUDA when PyTorch reports it available, then Metal Performance Shaders when those are available, and CPU otherwise. To pin it, pass `device` to the model or to `predict`, `train`, `val` and `export`. It accepts `"cpu"`, `"cuda"`, `"cuda:0"`, `"mps"`, a bare integer such as `0`, or a digit string; the last two expand to `cuda:`. `libreyolo checks` prints the Torch build, its CUDA and cuDNN versions, and every GPU it can see. If that command shows no CUDA, the PyTorch wheel is a CPU build; [install](/docs/install) covers replacing it. ## Where do downloaded weights go? Into `weights/` relative to the working directory. A model reference with no directory component resolves there and is downloaded on first use; a reference that includes a directory is used exactly as written and is never fetched. See [checkpoints and weights](/docs/weights). ## Can I run with no network access? Yes. Fetch the checkpoints once on a connected machine, copy the `weights/` directory across, and nothing will reach the network again. A shared read-only path also works, since a reference containing a directory is taken literally. Datasets resolve under `~/datasets`, or under `LIBREYOLO_DATASETS_DIR`. ## Can I use LibreYOLO commercially? The code is MIT licensed. Pretrained weights are a separate question: they can inherit terms from the project or dataset they came from, and those terms are not uniform even within one family. The license on the specific Hugging Face repository is authoritative, and every model page carries a licensing section that reproduces it. Where weights are restricted, LibreYOLO prints the restriction before the download starts. ## Can I load a checkpoint from another project? Usually, by passing its path to `LibreYOLO()`. Recognized upstream layouts are converted at load time, keeping their class count and names, and a LibreYOLO checkpoint is written next to the source. [Import existing weights](/docs/migrate) covers what is recognized and what needs a conversion script. ## Why does train raise NotImplementedError? Because that family ships inference only, and the exception names the reason. Predict, validate and, where supported, export all work; there is no training loop for that architecture in LibreYOLO. The support tier in a model page's header tells you before you try. See [core concepts](/docs/concepts). ## What does val return? A plain dictionary, not an object. Detection keys include `metrics/precision`, `metrics/recall`, `metrics/mAP50` and `metrics/mAP50-95`. Other tasks return the keys that make sense for them, such as `metrics/accuracy_top1` for classification or `metrics/PQ`, `metrics/SQ` and `metrics/RQ` for panoptic segmentation. ## How do I run on a folder, a video or a webcam? Pass it as the source. A file path is one image, a directory is every image in it, a video path is a video, an integer is a webcam index, and an RTSP, RTMP, TCP, UDP or HLS URL is a live stream. A `.streams` file lists several sources at once. Live sources require `stream=True`, which yields one `Results` per frame instead of building a list; the same flag is worth using for long videos and large directories. Only YouTube page URLs need an extra, `libreyolo[stream]`. ## How do I keep only some classes? Pass `classes` to `predict` with the class indices you want, for example `classes=[0, 2]`. `conf` sets the confidence threshold, default `0.25`, and `max_det` caps detections per image, default `300`. ## Does the CLI use flags or key=value pairs? Key and value joined by an equals sign, for every command: ```bash libreyolo predict model=yolo9-t source=my-image.jpg save=True libreyolo train model=yolo9-t data=coco8.yaml epochs=50 imgsz=640 ``` `model` accepts a path or a short name of the form `family-size`, optionally with a task suffix, and `libreyolo models` lists every valid one. Diagnostic and inventory commands also take `--json`, which prints the same data as a machine-readable object on stdout. ## Can every model export to every format? No. Coverage is per family and per task, not uniform, and each format has its own extra to install. Each model page carries its family's export matrix; the [export section](/docs/export) covers the formats themselves. ## What is the difference between segment, semantic and panoptic? Three separate tasks. `segment` produces one mask per detected object. `semantic` labels every pixel with a class and separates nothing into instances. `panoptic` gives every pixel exactly one label, merging countable things with amorphous stuff. They have different ground truth, different result fields and different metrics, and a family supports whichever of them appears in its task list. ## How do I train on my own classes? Write a dataset YAML with `train`, `val` and `names`. Labels sit beside the images in a parallel `labels/` tree, one `.txt` per image, with normalized coordinates. `nc` is optional and must agree with `names` when present. Run `libreyolo doctor ` first: it checks the dataset for problems and exits non-zero when it finds errors, which makes it usable as a CI gate. ## Why does loading print a metadata warning? Because the checkpoint does not carry complete v1.0 metadata. Loading continues through a compatibility path, and the warning names exactly which keys are missing. Run `libreyolo metadata path=` to see what is there, and see [checkpoints and weights](/docs/weights) for what the schema requires. ## An import stopped working after an upgrade. What changed? Two class names were renamed for consistency: `LibreYOLORTDETR` became `LibreRTDETR` and `LibreYOLORFDETR` became `LibreRFDETR`. The old names still resolve and emit a `DeprecationWarning` pointing at the new one, so existing code keeps running while you update it. --- # Install LibreYOLO is published on PyPI as libreyolo. The base package covers prediction, training, validation and the model families that need nothing beyond PyTorch; optional extras add the rest. Verified against LibreYOLO v1.5.0. ## Install **pip** ```bash pip install libreyolo ``` **With extras** ```bash # Comma-separate to combine several in one install. pip install "libreyolo[rfdetr,onnx]" ``` **Everything** ```bash pip install "libreyolo[all]" ``` **From source** ```bash git clone https://github.com/LibreYOLO/libreyolo.git cd libreyolo pip install -e . ``` Python 3.10 or newer is required. The base install pulls PyTorch, torchvision, NumPy, Pillow, OpenCV, PyYAML, requests, mss, tqdm, pycocotools, typer, click, safetensors and SciPy, so YOLOv9 and the other families that need nothing more work straight after `pip install libreyolo`. A clone checks out `release`, the stable branch whose code matches these docs. The integration branch, carrying unreleased work, is `dev`. ## Optional extras An extra is a bracketed name that adds the dependencies one model family or one export target needs. Nothing else changes: the API is the same whether or not an extra is present. ### Model families | Extra | Adds | |---|---| | `rfdetr` | `transformers`, which supplies the RF-DETR backbone | | `eomt` | `transformers` | | `midas` | `timm` 1.0.x, which supplies MiDaS's ViT-L/16 and EfficientNet-Lite3 encoders | | `vlm` | `transformers`, `num2words`, `decord`, `lmdb`, `peft` | | `sam` | `transformers`, `timm` | | `openvocab` | `transformers`, `timm`, `regex`, `ftfy` | | `sensenova` | `transformers`, `accelerate`, and `bitsandbytes` off macOS | | `modus` | `transformers`, `accelerate` | | `clip` | `regex` and `ftfy`, needed by the vendored CLIP text tokenizer | | `siglip2` | `sentencepiece`, needed by the multilingual SigLIP 2 tokenizer | | `gaze` | `gdown`, which turns on auto-download of the L2CS checkpoint | | `rtdetr` | Nothing. RT-DETR needs no extra dependency; the name is kept stable | ### Export and runtimes | Extra | Adds | |---|---| | `onnx` | `onnx`, `onnxsim`, `onnxruntime` | | `tensorrt` | `tensorrt-cu12` 10.16.1.11 and `pycuda`, off macOS | | `openvino` | `openvino` | | `coreml` | `coremltools` | | `coreai` | `coreai-torch`, macOS only | | `tflite`, alias `litert` | `libreyolo[onnx]` plus `onnx2tf`, `ai-edge-litert`, `onnx-graphsurgeon` and `onnx-simplifier` | | `mnn` | `libreyolo[onnx]` plus `MNN` | | `ncnn` | `pnnx` and `ncnn` | | `paddle` | `libreyolo[onnx]` plus `paddlepaddle` 2.6.2 and `x2paddle` 1.6.0 | | `executorch` | `executorch` | | `triton` | `tritonclient[http]` for HTTP and HTTPS V2 inference | ### Training, evaluation and logging | Extra | Adds | |---|---| | `lora` | `libreyolo[rfdetr]` plus `peft`, for `lora=True` fine-tuning | | `plots` | `matplotlib` | | `fast-eval` | `faster-coco-eval`, the C++ COCO evaluation backend | | `tensorboard` | `tensorboard` | | `mlflow` | `mlflow` | | `wandb` | `wandb` | | `comet` | `comet-ml` | | `clearml` | `clearml` | | `neptune` | `neptune-scale` | | `dvclive`, alias `dvc` | `dvclive` | `fast-eval` is opt-in rather than a hard dependency so that a platform without a prebuilt wheel cannot break a plain install. When the package is absent, COCO evaluation falls back to pycocotools and the run continues. ### Tooling | Extra | Adds | |---|---| | `stream` | `yt-dlp`, needed only to resolve YouTube page URLs | | `tracking` | Nothing. Every tracking dependency is already a core dependency | | `label` | `libreyolo[sam]`, which enables click-to-mask assist in `libreyolo label` | | `hub-kernels` | `kernels`, the optional loader for compiled Hub kernels. See [kernels](/docs/reference/kernels), which notes that installing it can shift RF-DETR predictions at float tolerance | | `clip-convert` | `libreyolo[clip]` plus `open_clip_torch`, for weight conversion and parity checks | | `siglip2-convert` | `libreyolo[siglip2]` plus `transformers`, for the same reason | Webcams, RTSP, RTMP, TCP, UDP, HLS and local multi-stream lists need no extra. Only YouTube page URLs do. ### The aggregate extra `libreyolo[all]` installs the model, export, tracking and logging extras in one command. Some are deliberately outside it. `neptune` is excluded because stable `neptune-scale` requires protobuf below 7 while the TFLite path requires protobuf 7. `executorch` is excluded because ExecuTorch constrains which PyTorch version it pairs with, and `coreai` because `coreai-torch` pins PyTorch to 2.11.x and would drag the whole environment onto that version. `fast-eval`, `hub-kernels`, `clip-convert` and `siglip2-convert` are also left out. Install any of them by name. ## Platform constraints Three extras are platform-scoped by their dependency markers, so the install succeeds everywhere and simply installs less where a wheel does not exist. | Extra | Constraint | |---|---| | `coreai` | macOS only. The Core AI toolchain neither converts nor runs elsewhere | | `tensorrt` | Skipped on macOS, which has no CUDA | | `tflite`, `litert` | `onnx2tf` and `ai-edge-litert` require Python 3.12 or newer | `sensenova` skips `bitsandbytes` on macOS, where no wheel is published; the rest of the extra installs normally. If disk is the constraint, most of it is PyTorch, and most of PyTorch is the CUDA payload its default wheel bundles. A CPU-only wheel removes that without giving anything up. For ONNX detection on a machine that should carry no torch at all, see the [lightweight install](/docs/lightweight-install). ## GPU and CUDA Device selection happens when a model is constructed. The default, `device="auto"`, uses CUDA when `torch.cuda.is_available()` is true, then Metal Performance Shaders when `torch.backends.mps.is_available()` is true, and CPU otherwise. Nothing else in the library inspects the hardware, so if PyTorch cannot see a GPU, neither can LibreYOLO. To pin the device instead, pass `device` to the model or to `predict`, `train`, `val` and `export`. It accepts `"cpu"`, `"cuda"`, `"cuda:0"`, `"mps"`, a bare integer such as `0`, or a digit string such as `"0"`; the last two are expanded to `cuda:`. Start with `libreyolo checks`, which prints the Torch version, the CUDA and cuDNN versions Torch was built against, and every visible GPU with its memory. When it reports no CUDA on a machine that has an NVIDIA card, the PyTorch wheel pip resolved is a CPU build. Install a CUDA build from the PyTorch index first, then install LibreYOLO: ```bash pip install torch torchvision --index-url https://download.pytorch.org/whl/cu128 pip install libreyolo ``` That is the same index the repository pins for its own uv-managed environment on Linux and Windows. It needs NVIDIA driver 555 or newer, which is the CUDA 12.8 runtime requirement. macOS keeps the PyPI wheel, since the PyTorch download host publishes no Darwin builds. ## Check the install **CLI** ```bash # Python, Torch, CUDA, cuDNN, every visible GPU, and which # optional packages are installed. libreyolo checks ``` **Python** ```python import libreyolo print(libreyolo.__version__) ``` **Model inventory** ```bash # Every registered family with its tasks, sizes and input # resolutions. Families whose extra is missing are listed with # the pip command that enables them. libreyolo models ``` `libreyolo models` is the fastest way to see whether an extra took effect: a family whose dependency is missing is printed with the exact pip command that enables it. Both commands also accept `--json`, which prints the same data as a machine-readable object on stdout. --- # Licensing LibreYOLO carries three separately licensed things: its own code, upstream code vendored into a model family, and pretrained checkpoints. They are often not the same license. Verified against LibreYOLO v1.5.0. ## LibreYOLO's own code The library is MIT. That covers the Python API, the CLI, the trainers, validators and exporters, the dataset loaders, and the conversion scripts under `weights/`. Use it in a commercial or closed-source product, keep the copyright line and the license text with any copy you redistribute, and the obligation ends there. The grant stops at the code. The [`LICENSE`](https://github.com/LibreYOLO/libreyolo/blob/release/LICENSE) file puts it plainly: > Those licenses vary and are not all permissive: some published weights are > non-commercial or otherwise restricted, and this MIT License does not extend > to them. Choosing a model means choosing its license. ## Upstream code, per family Most families are ports of published research, and several vendor upstream source directly. A vendored file keeps its original copyright header and its original license. MIT does not overwrite it, and LibreYOLO does not relicense anyone's work. Apache-2.0 and BSD-3-Clause are the two that come up most often. Apache-2.0 covers the DETR line and much of the transformer work: DETR from Meta AI (FAIR), Deformable DETR from SenseTime, LW-DETR from Baidu, OV-DEIM by Leilei Wang and coauthors, the SegFormer implementation LibreYOLO ports from Hugging Face Transformers, PP-OCRv5 from the PaddlePaddle Authors, SwinIR from the Computer Vision Lab at ETH Zurich, and Depth Anything 3 from ByteDance Seed. It also covers the classifiers derived from timm by Ross Wightman and the timm contributors, among them ResNet, DeiT, EfficientNetV2, MobileNetV4 and Swin, whose module names mirror timm so that its ImageNet tensors load unchanged. BSD-3-Clause covers everything derived from torchvision: Faster R-CNN, Mask R-CNN, FCOS, RetinaNet, SSD300, AlexNet, VGG, FCN and DeepLabv3. MIT covers a smaller group, including NAFNet from Megvii, CenterNet from Xingyi Zhou, and YOLOv7 as re-released by its own authors, Kin-Yiu Wong and Hao-Tang Tsui, at MultimediaTechLab. The YOLOv1 through YOLOv4 families reproduce architectures from the Darknet project, by Joseph Redmon and, for YOLOv4, by Alexey Bochkovskiy. Darknet is public domain, so those carry no obligation at all. One bundled subtree is not an open-source license. The DEIMv2 family ships DINOv3 backbone code from Meta Platforms under the DINOv3 License Agreement, a custom non-OSI license. Redistributing that code means shipping a copy of the agreement with it, and the agreement forbids use for activities subject to ITAR, military or warfare purposes, nuclear industries, espionage, and weapons development. Those terms bind that subtree only. Two files in the repository hold the full picture. [`NOTICE`](https://github.com/LibreYOLO/libreyolo/blob/release/NOTICE) lists every bundled third-party subtree with its path, its license file and its upstream source. [`THIRD_PARTY_NOTICES.txt`](https://github.com/LibreYOLO/libreyolo/blob/release/THIRD_PARTY_NOTICES.txt) lists the upstream projects LibreYOLO derives from and reproduces each license text in full. ## Weights, per checkpoint No pretrained weight file ships inside the package. Published checkpoints live on Hugging Face under the [LibreYOLO organization](https://huggingface.co/LibreYOLO), and each repository carries its own `LICENSE` and attribution reflecting the project the weights came from. That repository is the authoritative source for the terms. Not this page, not the model page, and not the summary in the source tree. See [checkpoints and weights](/docs/weights) for how files are named and where they are downloaded from. Licenses differ between families, and they differ between files inside one family. Two examples of the second: - The YOLO9 COCO checkpoints are MIT. `LibreYOLO9P2s-visdrone.pt`, trained on VisDrone2019-DET, is CC BY-NC-SA 3.0, which is non-commercial. - The RF-DETR detection checkpoints are Apache-2.0. The oriented-box checkpoints are CC BY 4.0, because they were fine-tuned on a Roboflow Universe dataset published under CC BY 4.0 and the weights carry that dataset's attribution requirement forward. Across families, the range runs wider, and several published checkpoints cannot be used in a commercial product: - SegFormer is the clearest split between the two layers. The implementation is an Apache-2.0 port of Hugging Face Transformers' code. The published ADE20K checkpoints are converted from NVIDIA's release under the NVIDIA Source Code License, which permits redistribution but limits use to non-commercial research or evaluation, and carries that limit forward into derivative works. Those checkpoints are not covered by LibreYOLO's permissive terms. - OV-DEIM checkpoints are CC BY-NC 4.0, confirmed by the upstream author. Every prediction also loads Apple's MobileCLIP-B(LT) text tower, whose license restricts use to research, a stricter term than the checkpoint's own. - SenseNova-Vision code is Apache-2.0 and its weights are CC BY-NC 4.0. The loader prints the non-commercial notice before every automatic download. Some families have no checkpoint hosted by LibreYOLO at all, and their pages say so in the Weights row. SAM 3 is gated on Hugging Face under Meta's custom SAM License and is downloaded from Meta directly. MiDaS release assets are fetched from the official URLs and hash-verified rather than rehosted. Dome-DETR is linked upstream because its model card states no license in its metadata while its prose claims Apache-2.0 and restricts use to academic research at the same time, and those do not agree. The TEED and DexiNed architectures are MIT, but the authors' released checkpoints were trained on BIPED, whose dataset terms are non-commercial, so LibreYOLO neither bundles nor auto-downloads them. Several torchvision checkpoints carry no license file of their own. LibreYOLO mirrors them on the license the releasing project uses, states on each model card that the basis is implied rather than granted per checkpoint, and repeats torchvision's own warning that pretrained model terms may derive from the training data. ## Finding the terms for one model The model page carries a **Licenses** row in its header, in the form `Code X, weights Y`, which links down to the page's Licensing section. That section lists the original work and its authors, the upstream license, the upstream source, the LibreYOLO code license, the weights, and an interpretation of what the terms allow. The Checkpoints table on the same page has a **Weights license** column, one row per published file, so a family with mixed terms shows them file by file. All of that renders from the same data the library is checked against, which is why this page does not repeat it as a table. A hand-typed license matrix is wrong within one release, and wrong here is expensive. In the source tree, the equivalents are `NOTICE` for bundled code, `THIRD_PARTY_NOTICES.txt` for upstream projects and their license texts, and [`weights/LICENSE_NOTICE.txt`](https://github.com/LibreYOLO/libreyolo/blob/release/weights/LICENSE_NOTICE.txt) for a per-family summary of the published checkpoints. Then check the Hugging Face repository of the exact file you are about to download. It is authoritative, and it can change without a docs page changing with it. ## Commercial use Code is rarely the problem. MIT, Apache-2.0 and BSD-3-Clause all permit commercial and closed-source use. Each asks you to keep its license text and attribution notices with copies you redistribute, Apache-2.0 also grants a patent license, and none of them places conditions on your own application code. Checkpoints are where products get stuck. A non-commercial checkpoint stays non-commercial however permissive the surrounding code is, and converting the file does not change its applicable terms, which is what `weights/LICENSE_NOTICE.txt` states directly. An ONNX or TensorRT artifact built from a restricted checkpoint inherits the restriction. Where a license carries its restriction into derivative works, as the NVIDIA Source Code License does, fine-tuning does not escape it either. Training the same architecture from scratch on data you have the right to use does: the code is permissive, so a model you train yourself is yours, and the pretrained checkpoint's terms never enter it. The SegFormer page spells that out for its own weights; read the Interpretation row on the page of whichever family you plan to ship. Decide the license question when you pick the model rather than when you ship, and read the terms on the file you actually downloaded, because a family with one permissive checkpoint can have a restricted one beside it. ## Not legal advice This page describes the licenses involved. It is a description, not legal advice, and it does not create any warranty. If the answer matters commercially, read the licenses yourself and take your own counsel. --- # Lightweight install LibreYOLO's ONNX inference path is numpy end to end, including decode and NMS. Nothing on it needs PyTorch at runtime, so an install that skips dependency resolution can run detection with torch absent from the machine. Verified against LibreYOLO v1.5.0. ## Why this works `pip install --no-deps libreyolo` installs the package and skips its dependency list entirely. Nothing is resolved on your behalf, and you become responsible for installing what you actually use. That is only useful if the code path you want genuinely does not need the dependencies you skipped, and for ONNX detection it does not. The decode, including non-maximum suppression, is numpy. The preprocessing recipes are numpy. PyTorch is a training and eager-inference dependency, and on this path it is never called. Before this release the import failed anyway: importing anything under `libreyolo.models` built every model class to populate the checkpoint auto-detection registry, and those classes are `torch.nn.Module` subclasses. The preprocessing recipes now live in their own package, `libreyolo.preprocess`, and the torch import is deferred until something touches a torch attribute, so the ONNX path imports with torch absent from the machine. That package holds a numpy-native preprocessor per family: `yolo9`, `yolonas`, `yolox`, `ec`, `rtdetr`, `rfdetr`, `dfine`, `deim` and `deimv2`, two more than the seven families verified end to end below. Each `libreyolo/models//utils.py` re-exports from it, so existing import paths keep working. ## Try the CPU-only wheel first Most people asking for this want to avoid a multi-gigabyte install, and the size is concentrated in one place: the default `torch` wheel bundles CUDA. A CPU-only build is a fraction of that and needs no special install path. **Lightweight** ```bash # Install the package without its dependency list, then supply the # four packages the ONNX detection path actually imports. pip install --no-deps libreyolo pip install numpy pillow opencv-python-headless onnxruntime ``` **CPU-only torch** ```bash # Try this first. It keeps every feature and avoids the CUDA wheel, # which is where most of the disk goes. pip install libreyolo --index-url https://download.pytorch.org/whl/cpu ``` The CPU-only option keeps every LibreYOLO feature: training, validation, every task, every family, the CLI. Take the lightweight path when you want zero torch on the machine, not merely less of it. ## What the lightweight install covers | | | |---|---| | Task | Detection | | Format | ONNX | | Entry point | `OnnxBackend` | | Interface | Python library | Seven families were verified on this path: [YOLOv9](/docs/models/yolov9), [YOLO-NAS](/docs/models/yolo-nas), [EdgeCrafter](/docs/models/edgecrafter), [RT-DETR](/docs/models/rt-detr), [RF-DETR](/docs/models/rf-detr), [D-FINE](/docs/models/d-fine) and [DEIM](/docs/models/deim), counting each family's variants with it. That is the verified scope, not a boundary the library enforces. Other tasks and other families are simply outside what was checked: some will pull torch when you call them, and a few may happen to work. Treat anything beyond this list as untested rather than as supported or as broken. Inside it, results are identical to the normal install, not merely close. Each family was exported to ONNX and run twice, once normally and once with torch blocked; boxes, scores and classes matched exactly. A parity test in the suite keeps that contract from drifting. ## The five things that catch people **Use `OnnxBackend`, not the model classes.** `LibreYOLO9("model.onnx")` still requires torch, because `LibreYOLO9` is itself an `nn.Module` subclass. This is the likeliest mistake, since every other page in these docs loads a model through its class or through `LibreYOLO()`. **Export somewhere else.** Producing the `.onnx` file requires torch, so the lightweight machine cannot make one. Export on a development or CI machine and ship the artifact to the slim target. **Results carry numpy arrays.** `result.boxes.xyxy` is an `ndarray` here. The containers accept either type so the attribute names are unchanged, but code that calls `.cpu()` or `.numpy()` on a result will fail. **A single image returns a single `Results`.** `predict()` returns one `Results` for one image and a list for several. Indexing a single result with `[0]` selects the first detection, not the first image, which silently gives you a one-box result instead of raising. **The CLI will not work.** `typer` and `click` are not in the four packages, so the `libreyolo` command is unavailable. This is a library install. ## Predict **Python** ```python from libreyolo.backends.onnx import OnnxBackend model = OnnxBackend("libreyolo9t.onnx") result = model.predict("https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg") # xyxy is a numpy ndarray here, not a torch tensor. print(result.boxes.xyxy) print(result.boxes.conf) print(result.boxes.cls) ``` Swap `onnxruntime` for `onnxruntime-gpu` to run on CUDA. The four packages are the ones a full torch-free `predict()` actually imports, recorded during the call rather than reasoned about. `opencv-python-headless` stands in for the declared `opencv-python`: same module, no GUI libraries, smaller on disk. Of the remaining declared dependencies, `requests` is needed only to load an image from a URL, `pycocotools` and `scipy` are validation and evaluation, and `typer` and `click` are the CLI. ## This list will drift, by design The package list above is correct for the release named at the top of this page. `--no-deps` opts you out of dependency resolution, so nothing checks it for you, and a later release may import something not listed here. If you hit a `ModuleNotFoundError`, you already understand the technique: install the missing package. That is the intended maintenance model rather than a bug report. This path is best effort and is not a separately supported distribution, which is also why there is no second lightweight package on PyPI and no plan for one. To confirm your environment is really torch-free rather than quietly falling back to an installed copy, assert it: ```python import importlib.util assert importlib.util.find_spec("torch") is None, "torch is installed" ``` That check is worth keeping in CI for the slim image. Without it, an environment that happens to have torch will pass every test and tell you nothing. --- # Import existing weights LibreYOLO ports its model families from upstream projects, so their released checkpoints are almost loadable already. What they lack is metadata. Autoconversion supplies it at load time. Verified against LibreYOLO v1.5.0. This page is about checkpoints from other projects. If you are moving your own code from an older LibreYOLO, see [upgrading to 1.5.0](/docs/upgrade). ## What happens when you load a foreign file `LibreYOLO()` loads any weight file through the restricted, weights-only path first. If the result carries complete LibreYOLO metadata, it is used directly. If it does not, the file goes to the autoconverter before anything else is attempted. If the restricted load fails outright, which happens when a checkpoint has a third-party object pickled into it, the autoconverter is tried with a loader that neutralizes those objects instead. Autoconversion does four things. It unwraps the tensor dictionary from whichever layout the upstream project used. It asks every registered family whether it recognizes the resulting keys, remapping names where the upstream naming differs from LibreYOLO's port. It wraps the winner in a checkpoint that satisfies metadata schema v1.0, reading size, task and class count from the tensors themselves. Then it writes the result next to the source file and loads that. **Python** ```python from libreyolo import LibreYOLO # Substitute the path to a checkpoint you already have. A recognized # upstream layout is converted on the fly, written next to the # source, and then loaded. model = LibreYOLO("path/to/upstream-checkpoint.pth") # Class count and names come from the tensors and the file's own # metadata, so a fine-tune keeps its label set instead of COCO's. print(model.family, model.size, model.task, model.nb_classes) print(model.names) ``` **CLI** ```bash libreyolo predict model=path/to/upstream-checkpoint.pth \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` **Check the result** ```bash # The converted file satisfies the same schema as a published one. libreyolo metadata path=path/to/upstream-checkpoint-LibreYOLO9t.pt ``` The conversion is not silent. A converted file is logged with the family, the source name, the output name and the resulting class count, so a run's log records exactly what was loaded. ## The layouts it unwraps Upstream checkpoints nest their weights in a handful of conventional places, and the converter tries them in order until one holds tensors: an EMA block under `ema.module` or a flat `ema`, an `ema_state_dict` with its `module.` prefix stripped, then `params_ema`, `params`, `ema_net`, `net`, `model`, `state_dict`, and finally the object itself. Trying several rather than the first means an `ema` block holding only counters does not mask the real weights below it. Wrapper prefixes come off too: `module.` from distributed training, `_orig_mod.` from a compiled model, and a `model.model.` nesting some redistributions add. ## What it reads, and from where Size, task and class count come from the tensors, not from the filename, which is why a fine-tuned checkpoint converts with its own class count instead of the architecture's default. Class names are taken from the checkpoint's own metadata when present, from an `args` or `hyper_parameters` block if the names sit there, and are trimmed to the detected class count so a fine-tune that kept its base label set does not carry indices its head no longer has. Dense tasks are handled explicitly rather than being given fabricated labels. A depth checkpoint gets one class named `depth`, a restore checkpoint one class named `image`. A pose checkpoint must yield a keypoint count, either from the tensors or from the family; if neither produces one, conversion is refused rather than writing an incomplete file. RF-DETR gets its own recognizer, because size detection needs the whole checkpoint and because its head has 91 outputs where LibreYOLO uses the 80-class COCO convention. A checkpoint is normalized to 80 classes when it carries exactly 80 names, or declares a class count of 80, or names COCO as its dataset, or carries no class or dataset metadata at all. A genuine 90-class model, identified by its names, an explicit non-80 count or a non-COCO dataset hint, is preserved as it is. ## Where the converted file goes The output is written beside the source, named after it: ```text -[-].pt ``` A tiny YOLOv9 detector saved as `upstream-checkpoint.pth` therefore becomes `upstream-checkpoint-LibreYOLO9t.pt`. Naming it after the source rather than after the family means two fine-tunes of the same family and size in one directory do not overwrite each other, and neither collides with an official checkpoint. The file is rewritten on every load, so it never goes stale against its source. If the directory is read-only, the converted file goes to a fresh private temporary directory instead and the log says where. From then on it is an ordinary LibreYOLO checkpoint: it loads through the metadata path, and `libreyolo metadata` reports it as valid. ## Cases that need a hand Two families sit outside the generic recognizer. The gaze family is excluded outright: it is inference only and its released weights carry redistribution restrictions. RF-DETR is excluded because it has the dedicated recognizer described above, which is what handles it instead. Raw upstream PIDNet checkpoints are refused, with an error pointing at `weights/convert_pidnet_weights.py`. That script writes the Cityscapes semantic metadata the checkpoint needs. D-FINE and DEIM share the same architecture keys, so tensors alone cannot separate them. When both claim a file and no sibling family with a distinguishing marker is in the running, the filename decides: a name in the shape of `dfine_hgnetv2_n_coco.pth` or `deim_hgnetv2_n_coco.pth` settles it, and a name that says nothing is refused with that explanation rather than guessed. Instantiating `LibreDFINE` or `LibreDEIM` directly also resolves it. When several families legitimately claim one file, a subclass beats the base class it refines, and registry order decides the rest, since that order encodes how specific each family's check is. The filename is consulted only for the D-FINE and DEIM tie, so a file's name can never promote a broad match over a precise one. ## The scripted converters The repository carries per-family conversion scripts under `weights/`, plus shared helpers for the repeated plumbing. They are the route for a file the runtime path declines, for producing a checkpoint ahead of time rather than at load time, and for the families whose metadata has to be supplied rather than inferred from tensors. Those scripts are part of the repository, not the installed package, so using one means cloning: ```bash git clone https://github.com/LibreYOLO/libreyolo.git cd libreyolo python weights/convert_pidnet_weights.py --help ``` Every script writes a checkpoint that satisfies schema v1.0, which is the same bar autoconversion meets and the same bar published weights meet. See [checkpoints and weights](/docs/weights) for what that schema contains. --- # AlexNet AlexNet is the convolutional network that won ILSVRC 2012 and helped start the deep learning era in computer vision. LibreYOLO ships the single-tower, later revision of the architecture for image classification. Tasks: classify. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install AlexNet needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreAlexNetb-cls.pt") result = model(SAMPLE_IMAGE, save=True) probs = result.probs print(probs.top1, probs.top1conf) print(probs.top5, probs.top5conf) ``` **CLI** ```bash libreyolo predict model=LibreAlexNetb-cls.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` A classifier returns `result.probs` instead of `result.boxes`: `top1` and `top5` give class indices, `top1conf` and `top5conf` give their confidences. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants One size. The shipped graph is the later single-tower revision released by torchvision, with 64 first-layer filters and no local response normalization, not the original two-GPU 2012 architecture. LibreYOLO ships this family inference-only: prediction, ImageNet-style top-1/top-5 validation and export are supported, and fine-tuning is not implemented. ## Validate `val()` runs against an ImageFolder-style split (a directory with `train/` and `val/` subfolders, one folder per class) and returns top-1 and top-5 accuracy. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreAlexNetb-cls.pt") # data is a directory root with train/ and val/ class-folder splits # (ImageFolder layout), not a dataset YAML. metrics = model.val(data="imagenet-1k/") print(metrics["metrics/accuracy_top1"]) print(metrics["metrics/accuracy_top5"]) ``` **CLI** ```bash libreyolo val model=LibreAlexNetb-cls.pt data=imagenet-1k/ ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | classify | yes | yes | yes | yes | yes | | | | yes | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts and the extras a few of them add. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreAlexNetb-cls.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreAlexNetb-cls.pt format=onnx libreyolo export model=LibreAlexNetb-cls.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreAlexNetb-cls.onnx") result = model(SAMPLE_IMAGE) print(result.probs.top1) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreAlexNetb-cls.pt` | 224 | classify | bsd-3-clause | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: AlexNet, University of Toronto - Upstream license: BSD-3-Clause - Upstream source: https://github.com/pytorch/vision - LibreYOLO code: BSD-3-Clause - Weights: BSD-3-Clause, republished at https://huggingface.co/LibreYOLO - Interpretation: BSD-3-Clause is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep the copyright notice, the list of conditions and the disclaimer with any copy you redistribute, and it forbids using the contributors' names to endorse a derived product without permission; it carries no explicit patent grant. LibreYOLO's code and the shipped checkpoint are both derived from torchvision's single-tower AlexNet, the later "one weird trick" graph, not the original two-GPU 2012 model, and no code from the 2012 paper's authors is republished here. Torchvision itself notes that BSD-3-Clause redistribution of a pretrained checkpoint is an implied basis rather than a grant written for that specific checkpoint, and that pretrained-model terms can depend on the data a model was trained on; LibreYOLO repeats that caveat on the hosted weights repository. --- # BiRefNet A bilateral-reference network that predicts a soft alpha matte separating a subject from its background. LibreYOLO ships inference and validation for BiRefNet's matte task. Tasks: matte. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install BiRefNet needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreBiRefNetl-matte.pt") result = model(SAMPLE_IMAGE, save=True) matte = result.matte print(matte.array.shape, matte.array.dtype) ``` **CLI** ```bash libreyolo predict model=LibreBiRefNetl-matte.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **Cutout** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreBiRefNetl-matte.pt") result = model(SAMPLE_IMAGE) # RGBA (H, W, 4) uint8: source RGB plus the matte as an alpha channel. rgba = result.cutout() result.save("subject.png") ``` A matte result carries no boxes; `result.matte` is a dense `(H, W)` float32 array in `[0, 1]`, 1 fully foreground and 0 fully background. Unlike a binary mask, the soft matte keeps anti-aliased edge detail such as hair and fur. `result.cutout()` composites the source image with that alpha channel into an RGBA array, and `result.save(path)` (or `save=True` on the predict call) writes it straight to a transparent-background PNG. The model runs at a fixed native 1024x1024 canvas; a different resolution is not supported, because the Swin backbone's relative-position tables are tied to it, and a mismatch interpolates them badly rather than raising an error. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants One published checkpoint, `l`, the Swin-L tier BiRefNet-general model and the quality default upstream. The family's code also supports a Swin-T lite tier, `t`, but no LibreYOLO conversion of it is published yet. ## Validate `val()` reports two metrics over a paired image/matte folder, both in `[0, 1]` and independent of resolution: MAE, the mean absolute error against the ground-truth alpha (lower is better), and S-measure (Fan et al., ICCV 2017), a structural similarity that credits preserving the subject's shape and holes, which pixel MAE alone misses (higher is better). Validation drives the model's own `predict`, so it uses the family's exact preprocessing. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreBiRefNetl-matte.pt") # A directory containing images/ and an auto-detected matte directory # (mattes/, matte/, gt/, masks/, mask/ or alpha/) also works in place # of a dataset YAML. metrics = model.val(data="my-matte-dataset/") print(metrics["metrics/MAE"]) print(metrics["metrics/Smeasure"]) ``` Validation is inference-only; fine-tuning is a documented follow-up rather than a shipped feature (see Predict for the exact resolution constraint that any future trainer would inherit). ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | matte | yes | yes | | | | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` file behaves like a checkpoint and returns the same `Results`. TorchScript is the validated path; ONNX conversion runs but has not cleared the same parity bar. [Export](/docs/export) lists the arguments every format accepts and the extras a few of them add. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreBiRefNetl-matte.pt") model.export(format="onnx") ``` **CLI** ```bash libreyolo export model=LibreBiRefNetl-matte.pt format=onnx ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreBiRefNetl-matte.onnx") result = model(SAMPLE_IMAGE) print(result.matte.array.shape) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreBiRefNetl-matte.pt` | | matte | mit | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: BiRefNet, Nankai University - Upstream license: MIT - Upstream source: https://github.com/ZhengPeng7/BiRefNet - LibreYOLO code: MIT - Weights: MIT, republished at https://huggingface.co/LibreYOLO - Interpretation: MIT is a permissive license, so these weights can be used in commercial and closed-source products. The one standing obligation is to keep the license text and copyright notice with any copy you redistribute. It places no condition on your own application code. LibreYOLO's checkpoint is a format conversion of the official pretrained BiRefNet-general weights (the Swin-L, quality-default tier), with the learned parameters unchanged; fine-tuning is not wired into this library in v1, so there is no LibreYOLO-trained variant to license separately. ## Citation ```bibtex @article{zheng2024birefnet, title={Bilateral Reference for High-Resolution Dichotomous Image Segmentation}, author={Zheng, Peng and Gao, Dehong and Fan, Deng-Ping and Liu, Li and Laaksonen, Jorma and Ouyang, Wanli and Sebe, Nicu}, journal={CAAI Artificial Intelligence Research}, volume = {3}, pages = {9150038}, year={2024} } ``` Copied from https://github.com/ZhengPeng7/BiRefNet#citation --- # CenterNet CenterNet models an object as the center point of its bounding box and regresses every other property from a heatmap peak, so it needs no anchors and no non-maximum-suppression step. LibreYOLO ships it as an inference-only detector. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install CenterNet needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreCenterNetresdcn18.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreCenterNetresdcn18.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **DLA-34** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreCenterNetdla34.pt") result = model(SAMPLE_IMAGE, save=True) ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. `conf` and `max_det` filter the ranked heatmap peaks; `iou` is accepted for API parity but has no effect, because CenterNet's top-k peak decode needs no box-IoU suppression step. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Two backbones. `resdcn18` pairs a ResNet-18 trunk with deformable-convolution upsampling; `dla34` pairs a DLA-34 trunk with iterative deep-aggregation upsampling. Both feed the same three dense heads (heatmap, width/height, offset) and the same input canvas. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreCenterNetresdcn18.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreCenterNetresdcn18.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. ONNX export requires opset 16 or newer: the deformable-convolution upsampling stage in both backbones lowers to the ONNX `GridSample` operator, which opset 16 introduced. Requesting an older opset raises before tracing starts. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreCenterNetresdcn18.pt") # ONNX export needs opset 16 or newer: the deformable-convolution # upsampling stage lowers to GridSample, which opset 16 introduced. model.export(format="onnx", opset=18) model.export(format="tensorrt") ``` **CLI** ```bash libreyolo export model=LibreCenterNetresdcn18.pt format=onnx opset=18 ``` **Use the exported file** ```python from libreyolo import LibreYOLO # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreCenterNetresdcn18.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreCenterNetresdcn18.pt` | 512 | Detection | mit | | `LibreCenterNetdla34.pt` | 512 | Detection | mit | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: CenterNet, UT Austin and UC Berkeley - Upstream license: MIT - Upstream source: https://github.com/xingyizhou/CenterNet - LibreYOLO code: MIT - Weights: MIT, republished at https://huggingface.co/LibreYOLO - Interpretation: MIT is a permissive license, so these weights can be used in commercial and closed-source products. It asks only that you keep the copyright notice and license text with any copy you redistribute, and it places no obligation on your own application code. The official ResDCN-18 and DLA-34 COCO checkpoints were published by the MIT-licensed CenterNet project but carry no separate per-checkpoint license file; LibreYOLO's mirror states MIT as implied by the releasing project rather than a publisher-confirmed, checkpoint-specific grant. The ResDCN-18 graph also credits Microsoft's MIT-licensed human-pose-estimation.pytorch, and the DLA-34 graph credits Fisher Yu's BSD-3-Clause DLA implementation. LibreYOLO does not vendor the original DCNv2 extension the upstream project used; native execution runs torchvision's BSD-3-Clause `deform_conv2d` instead, and the export-only portable implementation was authored separately for LibreYOLO. ## Citation ```bibtex @inproceedings{zhou2019objects, title={Objects as Points}, author={Zhou, Xingyi and Wang, Dequan and Kr{\"a}henb{\"u}hl, Philipp}, booktitle={arXiv preprint arXiv:1904.07850}, year={2019} } ``` Copied from https://github.com/xingyizhou/CenterNet#citation --- # CLIP CLIP is a dual-tower model that scores an image against text prompts instead of a fixed label set. LibreYOLO supports it for zero-shot classification and image/text embedding, with no training step. Tasks: classify, embed. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install CLIP needs its own extra, which pulls in the packages its vendored BPE tokenizer uses to reproduce exact token ids. ```bash pip install "libreyolo[clip]" ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreCLIPb32-cls.pt") model.set_classes(["a forklift", "an empty aisle", "a spill"]) result = model(SAMPLE_IMAGE, save=True) print(model.names[result.probs.top1], float(result.probs.top1conf)) ``` **CLI** ```bash # With no set_classes() call, CLI predict uses the 1,000 ImageNet # class names the model loads with by default. libreyolo predict model=LibreCLIPb32-cls.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **Image and text embedding** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreCLIPb32-cls.pt", task="embed") image_embed = model(SAMPLE_IMAGE).embeddings.data text_embed = model.embed_text("a photo of a forklift") # Both are L2-normalized, so a plain dot product is cosine similarity. similarity = (image_embed @ text_embed.T).item() ``` `set_classes()` is the one primitive that makes this an open-vocabulary classifier: it renders each label into every prompt template, encodes and averages the results, and caches the resulting `[K, D]` matrix as the classifier head, so it is not recomputed per image. Call it again to change classes at any time. With no call, LibreCLIP loads with the 1,000 ImageNet-1k class names already set. With `task="embed"`, prediction returns one L2-normalized image vector per input instead of class probabilities, and `embed_text()` returns normalized text rows in the same vector space, so a plain dot product between them is cosine similarity. `iou` has no effect on either task; there is no NMS step. See [prediction](/docs/predict) for sources, streaming and result handling. ## Validate `val()` reads the class-folder names under an ImageFolder `train/` split, calls `set_classes()` with them, then measures zero-shot top-1 and top-5 accuracy. Accuracy depends on how the class names read as prompts, not on any weight update, since there is nothing to train. Validation only covers `task="classify"`; `task="embed"` has no dataset validator. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreCLIPb32-cls.pt") # data is an ImageFolder root with a train/ split; its folder names # become the zero-shot class prompts for this run. metrics = model.val(data="imagenette160") print(metrics["metrics/accuracy_top1"]) print(metrics["metrics/accuracy_top5"]) ``` **CLI** ```bash libreyolo val model=LibreCLIPb32-cls.pt data=imagenette160 ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | classify | yes | yes | yes | yes | yes | | | | | | | yes | | embed | yes | yes | yes | yes | yes | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. Export bakes the model's current state into a fixed graph. For `task="classify"`, whatever labels `set_classes()` last set, and the resolution at export time, are baked into a final linear layer, so the exported ONNX or TensorRT graph is an ordinary `[B, K]` image classifier with no text tower and no tokenizer; export again after changing either the classes or the size. `task="embed"` export traces the image tower alone. Both need ONNX opset 14 or higher, which the exporter sets by default. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreCLIPb32-cls.pt") model.set_classes(["a forklift", "an empty aisle", "a spill"]) model.export(format="onnx") # The current set_classes() labels and the input resolution are baked # into the graph. Re-export after changing either one. ``` **CLI** ```bash # No set_classes() call here, so this bakes in the default 1,000 # ImageNet classes the model loads with. libreyolo export model=LibreCLIPb32-cls.pt format=onnx ``` **Embedding export** ```python from libreyolo import LibreYOLO # task="embed" traces the image tower alone; no classes needed. model = LibreYOLO("LibreCLIPb32-cls.pt", task="embed") model.export(format="onnx") ``` ## Checkpoints Every published weight file for this family. Both are converted from OpenCLIP's LAION-2B-trained checkpoints (`ViT-B-32` and `ViT-B-16`), not from any COCO training run. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreCLIPb32-cls.pt` | | classify | mit | | `LibreCLIPb16-cls.pt` | | classify | mit | The LAION-2B training data has a documented history of CSAM content (Stanford Internet Observatory, December 2023). LAION has since released Re-LAION, a cleaned re-release; prefer Re-LAION-derived checkpoints where available if you re-host these weights further. ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: CLIP / OpenCLIP, OpenAI; LAION / ML Foundations - Upstream license: MIT - Upstream source: https://github.com/mlfoundations/open_clip - LibreYOLO code: MIT - Weights: MIT, republished at https://huggingface.co/LibreYOLO - Interpretation: MIT is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep the license text and copyright notice with any copy you redistribute, and it places no obligation on your own application code. LibreCLIP's tokenizer is vendored from the OpenAI CLIP / OpenCLIP byte-pair-encoding tokenizer (also MIT); its image and text towers are a clean-room re-implementation of the standard CLIP architecture, built without open_clip as a runtime dependency. The shipped checkpoints (b32, b16) are converted from OpenCLIP's LAION-2B-trained weights, which OpenCLIP publishes as MIT-redistributable. Training is not offered for this family: LibreCLIP is zero-shot, and set_classes() replaces the fine-tuning step a trained classifier would otherwise need. ## Citation ```bibtex @software{ilharco_gabriel_2021_5143773, author = {Ilharco, Gabriel and Wortsman, Mitchell and Wightman, Ross and Gordon, Cade and Carlini, Nicholas and Taori, Rohan and Dave, Achal and Shankar, Vaishaal and Namkoong, Hongseok and Miller, John and Hajishirzi, Hannaneh and Farhadi, Ali and Schmidt, Ludwig}, title = {OpenCLIP}, month = jul, year = 2021, note = {If you use this software, please cite it as below.}, publisher = {Zenodo}, version = {0.1}, doi = {10.5281/zenodo.5143773}, url = {https://doi.org/10.5281/zenodo.5143773} } ``` Copied from https://github.com/mlfoundations/open_clip#citing --- # ConvNeXt ConvNeXt is an image classifier built entirely from standard convolutions, modernized block by block from a ResNet toward the design choices of a vision transformer. LibreYOLO supports it for one task: classification. Tasks: classify. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install ConvNeXt needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` Adapter fine-tuning with `lora=True` is the exception, and needs the `lora` extra. ```bash pip install "libreyolo[lora]" ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreConvNeXtt-cls.pt") result = model(SAMPLE_IMAGE, save=True) print(result.probs.top1, result.probs.top1conf) print(result.probs.top5) ``` **CLI** ```bash libreyolo predict model=LibreConvNeXtt-cls.pt source=cat.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different model is a one line change. A classifier carries no boxes or masks: `result.probs` holds the whole-image prediction, with `top1`, `top5`, `top1conf` and `top5conf`. `conf`, `iou` and `max_det` are accepted for API parity but have no effect, since there is nothing to threshold or suppress on a single probability vector. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Three sizes, tiny/small/base, all trained and evaluated the same way, so picking one is a straight parameter-count-for-accuracy trade. The task is fixed: every size covers classification only. The weights filename ends `-cls.pt` on every size, and that suffix is what the factory reads to route to this family; no `task=` argument is needed. ## Train Fine-tuning starts from the published ImageNet backbone and rebuilds the final classifier layer to the target dataset's class count automatically. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreConvNeXtt-cls.pt") model.train(data="imagenette160", epochs=5) ``` **CLI** ```bash libreyolo train model=LibreConvNeXtt-cls.pt data=imagenette160 epochs=5 ``` **LoRA** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreConvNeXtt-cls.pt") model.train(data="imagenette160", epochs=5, lora=True) ``` **Multi-GPU** ```bash libreyolo train model=LibreConvNeXtt-cls.pt data=imagenette160 \ epochs=50 device=0,1 batch=-1 ``` Left alone, the trainer runs 100 epochs at `lr0=1e-3` with AdamW, a batch of 64 and early stopping after 50 epochs without improvement. `data` accepts a dataset root (`train/` and `val/`, one folder per class), a known short name such as `imagenette160`, or a `.zip` URL. ConvNeXt's blocks carry the `nn.Linear` MLPs LoRA needs, so `lora=True` is supported here, and injects adapters into the block MLPs rather than fine-tuning the full backbone. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary of `metrics/` keys. For classification that is top-1 and top-5 accuracy over the validation split. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreConvNeXtt-cls.pt") metrics = model.val(data="imagenette160") print(metrics["metrics/accuracy_top1"]) print(metrics["metrics/accuracy_top5"]) ``` **CLI** ```bash libreyolo val model=LibreConvNeXtt-cls.pt data=imagenette160 ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | classify | yes | yes | yes | yes | yes | | | | yes | yes | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts and the extras a few of them add. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreConvNeXtt-cls.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreConvNeXtt-cls.pt format=onnx libreyolo export model=LibreConvNeXtt-cls.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreConvNeXtt-cls.onnx") result = model(SAMPLE_IMAGE) print(result.probs.top1) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreConvNeXtt-cls.pt` | 224 | classify | apache-2.0 | | `LibreConvNeXts-cls.pt` | 224 | classify | apache-2.0 | | `LibreConvNeXtb-cls.pt` | 224 | classify | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: ConvNeXt, Meta AI (FAIR) - Upstream license: Apache-2.0 - Upstream source: https://github.com/huggingface/pytorch-image-models - LibreYOLO code: MIT - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours. The architecture is Meta AI's original design, released under MIT at facebookresearch/ConvNeXt; the block definitions, layer-scale parameter and module naming that LibreYOLO's implementation follows come from timm, whose convnext_{tiny,small,base}.fb_in1k ImageNet-1k weights are licensed Apache-2.0 and are what LibreYOLO ships. Only ConvNeXt V1 is shipped here: ConvNeXt-V2's small pretrained checkpoints are CC-BY-NC 4.0 and are deliberately excluded as not redistributable in a commercial library. Only ConvNeXt V1 is shipped in this family. ConvNeXt-V2's small pretrained checkpoints are CC-BY-NC 4.0 and are deliberately excluded, since a non-commercial weight cannot be redistributed inside an MIT/commercial library. ## Citation ```bibtex @Article{liu2022convnet, author = {Zhuang Liu and Hanzi Mao and Chao-Yuan Wu and Christoph Feichtenhofer and Trevor Darrell and Saining Xie}, title = {A ConvNet for the 2020s}, journal = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, year = {2022}, } ``` Copied from https://github.com/facebookresearch/ConvNeXt#citation --- # D-FINE A detection transformer that reformulates box regression as a probability distribution over each box edge, refined across decoder layers. LibreYOLO supports it for detection and instance segmentation. Tasks: Detection, Instance segmentation. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install D-FINE needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` Adapter fine-tuning with `lora=True` is the exception, and needs the `lora` extra. ```bash pip install "libreyolo[lora]" ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreDFINEn.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreDFINEn.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **Instance segmentation** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The -seg suffix in the filename selects the mask head, so no task # argument is needed here. model = LibreYOLO("LibreDFINEn-seg.pt") result = model(SAMPLE_IMAGE, save=True) print(result.masks.data.shape) ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. A `-seg` filename resolves to the segmentation task on its own, and `result.masks` then carries the instance masks alongside the boxes. `conf` and `max_det` filter the query selection; `iou` is accepted for API parity but has no effect, because the decoder is a set predictor with no NMS step. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Five sizes. They all run at the same input resolution, so the table separates them by parameter count and accuracy. | Checkpoint | Input (px) | mAP 50-95 | Params (M) | | --- | --- | --- | --- | | `LibreDFINEl` | 640 | 60.0 | 31.24 | | `LibreDFINEm` | 640 | 57.8 | 19.59 | | `LibreDFINEn` | 640 | 45.8 | 3.78 | | `LibreDFINEs` | 640 | 53.4 | 10.32 | | `LibreDFINEx` | 640 | 61.4 | 62.62 | COCO val2017, 500 images. Published on https://www.visionanalysis.org/ Interactive chart: https://www.visionanalysis.org/embed/scatter?highlight=dfine-l%2Cdfine-m%2Cdfine-n%2Cdfine-s%2Cdfine-x&title=D-FINE%20on%20COCO&subtitle=Accuracy%20against%20latency%2C%20every%20benchmarked%20model Segmentation reuses the detection backbone, encoder and decoder and adds a mask head, so a `-seg` checkpoint takes the same arguments as its detect sibling. LibreYOLO's RT-DETRv4 family is written as a subclass of the D-FINE wrapper: it inherits this decoder line and then pins its task list back to detection, because it carries no mask head. ## Train Training starts from a published checkpoint, for both tasks. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDFINEn.pt") model.train(data="my-dataset.yaml", epochs=50, imgsz=640, batch=8, lr0=2e-4) ``` **CLI** ```bash libreyolo train model=LibreDFINEn.pt data=my-dataset.yaml \ epochs=50 imgsz=640 batch=8 lr0=2e-4 ``` **Instance segmentation** ```bash # Continues from published segmentation weights, mask head included. libreyolo train model=LibreDFINEn-seg.pt data=my-dataset.yaml \ task=segment epochs=50 imgsz=640 ``` **Segmentation from detect weights** ```bash # Detect weights carry no mask head, so this is an explicit transfer: # the head starts untrained and is only useful once trained. Asking for # task=segment here is what authorizes the transfer. libreyolo train model=LibreDFINEn.pt data=my-dataset.yaml \ task=segment epochs=50 imgsz=640 ``` **LoRA** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDFINEn.pt") model.train(data="my-dataset.yaml", epochs=50, lora=True) ``` **Multi-GPU** ```bash libreyolo train model=LibreDFINEn.pt data=my-dataset.yaml \ epochs=50 device=0,1 batch=16 ``` Left alone, the trainer runs 132 epochs at `lr0=2e-4` with `amp=False`, a batch of 16 and early stopping after 50 epochs without improvement. Detect weights are a legal starting point for segmentation training, but only as an explicit transfer, since the mask head begins untrained and would otherwise return meaningless masks. Passing `task=segment` to the CLI is what authorizes it. The Python route is narrower: `LibreDFINE` has to be constructed directly with `allow_detect_to_segment_transfer=True`, because the `LibreYOLO()` factory takes no such argument, and direct construction does not download, so the weights file must already be on disk. `lora=True` applies to detection. Segment training rejects it and points at `freeze='backbone'` instead, because the mask head has not been tested with adapters. On Apple silicon the trainer moves the whole run to CPU: the backward pass of the Integral's binned matmul hits a Metal compilation failure. Inference on MPS is unaffected. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary keyed by metric name, and prints per-class results when `verbose` is left on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDFINEn.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreDFINEn.pt data=my-dataset.yaml ``` **Instance segmentation** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDFINEn-seg.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95(M)"]) # masks print(metrics["metrics/mAP50-95(B)"]) # boxes ``` Against a `-seg` checkpoint the plain `metrics/mAP50-95` key holds the mask score, and the same run also reports boxes under `(B)` and masks under `(M)` so both are available from one pass. ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | | yes | yes | yes | yes | | | | | yes | | Instance segmentation | yes | yes | | yes | yes | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. The OpenVINO, Paddle, MNN and Core AI paths export at a fixed canvas rather than dynamic shapes. [Export](/docs/export) lists the arguments every format accepts and the extras a few of them add. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDFINEn.pt") model.export(format="onnx", imgsz=640) model.export(format="tensorrt", imgsz=640, half=True) ``` **CLI** ```bash libreyolo export model=LibreDFINEn.pt format=onnx imgsz=640 libreyolo export model=LibreDFINEn.pt format=tensorrt imgsz=640 half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreDFINEn.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreDFINEn.pt` | 640 | Detection | apache-2.0 | | `LibreDFINEs.pt` | 640 | Detection | apache-2.0 | | `LibreDFINEm.pt` | 640 | Detection | apache-2.0 | | `LibreDFINEl.pt` | 640 | Detection | apache-2.0 | | `LibreDFINEx.pt` | 640 | Detection | apache-2.0 | | `LibreDFINEn-seg.pt` | 640 | Instance segmentation | apache-2.0 | | `LibreDFINEs-seg.pt` | 640 | Instance segmentation | apache-2.0 | | `LibreDFINEm-seg.pt` | 640 | Instance segmentation | apache-2.0 | | `LibreDFINEl-seg.pt` | 640 | Instance segmentation | apache-2.0 | | `LibreDFINEx-seg.pt` | 640 | Instance segmentation | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: D-FINE, University of Science and Technology of China - Upstream license: Apache-2.0 - Upstream source: https://github.com/Peterande/D-FINE - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours. The segmentation weights carry a second Apache-2.0 upstream, ArgoHA/D-FINE-seg, under the same terms. The segmentation weights have a second upstream: their mask decoder, mask matching and mask loss come from ArgoHA/D-FINE-seg, also Apache-2.0, whose maintainer approved reuse with attribution. ## Citation ```bibtex @misc{peng2024dfine, title={D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement}, author={Yansong Peng and Hebei Li and Peixi Wu and Yueyi Zhang and Xiaoyan Sun and Feng Wu}, year={2024}, eprint={2410.13842}, archivePrefix={arXiv}, primaryClass={cs.CV} } ``` Copied from https://github.com/Peterande/D-FINE#citation --- # DeepLabv3 A semantic segmentation network that pools features at several dilation rates in parallel (atrous spatial pyramid pooling) before classifying each pixel. LibreYOLO ships it for semantic segmentation only. Tasks: semantic. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install DeepLabv3 needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. The `-sem` filename suffix is required for this family. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreDeepLabv3r50-sem.pt") result = model(SAMPLE_IMAGE, save=True) mask = result.semantic_mask print(mask.data.shape) # (H, W) class ids print(mask.classes) # sorted class ids present in the image ``` **CLI** ```bash libreyolo predict model=LibreDeepLabv3r50-sem.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` Semantic segmentation returns one class id per pixel, not boxes, so `result.semantic_mask` carries a `(H, W)` array on `.data` and the list of class ids present in the image on `.classes`. `conf`, `iou` and `max_det` are accepted for API parity but have no effect: the model assigns a class to every pixel by argmax, with no confidence threshold or NMS step. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Three backbones: dilated ResNet-50, dilated ResNet-101, and dilated MobileNetV3-Large. This is DeepLabv3, not DeepLabv3+, so there is no decoder stage or CRF refinement, matching torchvision's implementation rather than the paper's own reference code. LibreYOLO does not train DeepLabv3: `train()` raises `NotImplementedError` for this family, which the [support tier](/docs/models) above marks as inference only. The three published checkpoints are torchvision's own COCO-with-VOC-label weights, converted for LibreYOLO's loader. ## Validate `val()` returns `metrics/mIoU` and `metrics/pixel_accuracy`, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDeepLabv3r50-sem.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mIoU"]) print(metrics["metrics/pixel_accuracy"]) ``` **CLI** ```bash libreyolo val model=LibreDeepLabv3r50-sem.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | semantic | yes | yes | | yes | yes | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDeepLabv3r50-sem.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreDeepLabv3r50-sem.pt format=onnx libreyolo export model=LibreDeepLabv3r50-sem.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreDeepLabv3r50-sem.onnx") result = model(SAMPLE_IMAGE) print(result.semantic_mask.data.shape) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreDeepLabv3r50-sem.pt` | | semantic | bsd-3-clause | | `LibreDeepLabv3r101-sem.pt` | | semantic | bsd-3-clause | | `LibreDeepLabv3mv3-sem.pt` | | semantic | bsd-3-clause | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: DeepLabv3, PyTorch - Upstream license: BSD-3-Clause - Upstream source: https://github.com/pytorch/vision - LibreYOLO code: BSD-3-Clause - Weights: BSD-3-Clause, republished at https://huggingface.co/LibreYOLO - Interpretation: BSD-3-Clause is a permissive license, so this code and these weights can be used in commercial and closed-source products. It asks you to keep the copyright notice, license text and a non-endorsement clause with any copy you redistribute. LibreYOLO's inference graph is torchvision's ASPP head over its ResNet-50, ResNet-101 and MobileNetV3-Large backbones; it is DeepLabv3, not DeepLabv3+, so there is no decoder or CRF, and the paper's training-only auxiliary FCN classifier is excluded. The three published checkpoints are torchvision's official COCO-with-VOC-label weights; their separate LibreYOLO Hugging Face mirrors carry BSD-3-Clause on an implied basis disclosed by torchvision rather than an explicit checkpoint-specific grant, and torchvision's own documentation notes that pretrained-model terms can depend on the training data, leaving that determination to the user. --- # Deformable DETR Deformable DETR replaces DETR's dense cross-attention with sparse, multi-scale sampling around each reference point, which is what made transformer detectors practical to train. LibreYOLO ships five sizes for detection, inference only. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install Deformable DETR needs no optional extra. Everything it imports is in the base install, using a pure-PyTorch multi-scale deformable attention core. ```bash pip install libreyolo ``` Installing `libreyolo[hub-kernels]` is optional. Once the `kernels` package is present, LibreYOLO fetches a compiled multi-scale deformable attention kernel from the Hugging Face Hub at runtime and uses it in place of the pure-PyTorch core; `LIBREYOLO_HUB_KERNELS=0` turns it back off. ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreDeformableDETRr50.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreDeformableDETRr50.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. `conf` and `max_det` filter the query selection; `iou` is accepted for API parity but has no effect, because the decoder is a set predictor with no NMS step. See [prediction](/docs/predict) for sources, streaming and result handling. Deformable DETR is inference-only in LibreYOLO. Upstream trains with Hungarian matching and a focal classification loss; that recipe is not implemented here, so `train()` raises `NotImplementedError`. ## Variants Five checkpoints cover the released configurations, all at the same input resolution. `r50ss` restricts attention to a single feature scale; `r50ssdc5` adds a dilated C5 backbone stage on top of that. `r50` is the default multi-scale configuration, sampling across four feature-map levels. `r50refine` adds iterative bounding box refinement across decoder layers, and `r50twostage` generates its initial region proposals from the encoder output instead of learned queries. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDeformableDETRr50.pt") # val() returns a plain dict, not an object metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) print(metrics["metrics/precision"], metrics["metrics/recall"]) ``` **CLI** ```bash libreyolo val model=LibreDeformableDETRr50.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDeformableDETRr50.pt") model.export(format="onnx", imgsz=800) model.export(format="tensorrt", imgsz=800, half=True) ``` **CLI** ```bash libreyolo export model=LibreDeformableDETRr50.pt format=onnx imgsz=800 libreyolo export model=LibreDeformableDETRr50.pt format=tensorrt imgsz=800 half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreDeformableDETRr50.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreDeformableDETRr50ss.pt` | 800 | Detection | apache-2.0 | | `LibreDeformableDETRr50ssdc5.pt` | 800 | Detection | apache-2.0 | | `LibreDeformableDETRr50.pt` | 800 | Detection | apache-2.0 | | `LibreDeformableDETRr50twostage.pt` | 800 | Detection | apache-2.0 | | `LibreDeformableDETRr50refine.pt` | 800 | Detection | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: Deformable DETR, SenseTime - Upstream license: Apache-2.0 - Upstream source: https://github.com/fundamentalvision/Deformable-DETR - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours. The five checkpoints are converted from SenseTime's own Hugging Face mirrors, each of which declares apache-2.0 in its model card; that declaration, not the original repository's Google Drive release links, is the redistribution basis. ## Citation ```bibtex @article{zhu2020deformable, title={Deformable DETR: Deformable Transformers for End-to-End Object Detection}, author={Zhu, Xizhou and Su, Weijie and Lu, Lewei and Li, Bin and Wang, Xiaogang and Dai, Jifeng}, journal={arXiv preprint arXiv:2010.04159}, year={2020} } ``` Copied from https://github.com/fundamentalvision/Deformable-DETR#citing-deformable-detr --- # DEIM A detection transformer trained with dense one-to-one matching, which converges in far fewer epochs than the DETR recipes it builds on. LibreYOLO carries two versions of it, told apart by the checkpoint you load. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install Neither version needs an optional extra. Everything they import is in the base install. ```bash pip install libreyolo ``` Adapter fine-tuning with `lora=True` is the exception, and needs the `lora` extra. ```bash pip install "libreyolo[lora]" ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreDEIMn.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreDEIMn.pt save=True \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` **Video** ```python from libreyolo import LibreYOLO # The version is part of the file name, and the factory routes on the # checkpoint, so both load the same way. model = LibreYOLO("LibreDEIMv2pico.pt") # Any source the library accepts: file, folder, URL, webcam index, # RTSP stream, or a .streams list for result in model.predict("clip.mp4", stream=True, save=True): print(len(result.boxes)) ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. `conf` and `max_det` filter a top-k decode over queries and classes; there is no NMS step to tune, and `iou` is accepted but unused. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Version 1 ships five sizes, all at the same input size. Version 2 keeps those five names and adds three smaller ones, `atto`, `femto` and `pico`, the first two of which are native at a lower input size than the rest. Five size codes therefore exist in both versions and name different models; the version is written into the checkpoint file name. | Checkpoint | Input (px) | mAP 50-95 | Params (M) | | --- | --- | --- | --- | | `LibreDEIMl` | 640 | 57.8 | 31.24 | | `LibreDEIMm` | 640 | 55.4 | 19.59 | | `LibreDEIMn` | 640 | 46.8 | 3.78 | | `LibreDEIMs` | 640 | 52.1 | 10.32 | | `LibreDEIMx` | 640 | 59.6 | 62.62 | | `LibreDEIMv2atto` | 320 | 27.5 | 0.51 | | `LibreDEIMv2femto` | 416 | 34.5 | 0.98 | | `LibreDEIMv2l` | 640 | 58.6 | 32.55 | | `LibreDEIMv2m` | 640 | 56.0 | 18.36 | | `LibreDEIMv2n` | 640 | 46.7 | 3.6 | | `LibreDEIMv2pico` | 640 | 42.2 | 1.54 | | `LibreDEIMv2s` | 640 | 53.0 | 9.78 | | `LibreDEIMv2x` | 640 | 61.3 | 51.21 | COCO val2017, 500 images. Published on https://www.visionanalysis.org/ Interactive chart: https://www.visionanalysis.org/embed/scatter?highlight=deim-l%2Cdeim-m%2Cdeim-n%2Cdeim-s%2Cdeim-x%2Cdeimv2-atto%2Cdeimv2-femto%2Cdeimv2-l%2Cdeimv2-m%2Cdeimv2-n%2Cdeimv2-pico%2Cdeimv2-s%2Cdeimv2-x&title=DEIM%20on%20COCO&subtitle=Accuracy%20against%20latency%2C%20every%20benchmarked%20model Version 1 keeps D-FINE's architecture and swaps its classification objective for the matchability-aware loss from the dense one-to-one recipe, so the two families share almost every state dict key and are told apart by the metadata in the checkpoint. Version 2 keeps that training contract and mixes backbones: HGNetv2 below `s`, and a DINOv3 vision transformer with a spatial tuning adapter at `s` and above. That backbone is what puts a second license on those four checkpoints, so read [licensing](#licensing) before you ship one. ## Train Training starts from a published checkpoint. `pretrained` never reaches the trainer: version 1 warns that the key is unknown and ignores it, version 2 removes it. Neither gives you a randomly initialized model. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDEIMn.pt") # coco128.yaml downloads a 128-image sample on first use. Point `data` # at your own dataset YAML for a real run. model.train(data="coco128.yaml", epochs=50, batch=8, lr0=1e-4) ``` **CLI** ```bash libreyolo train model=LibreDEIMn.pt data=coco128.yaml \ epochs=50 batch=8 lr0=1e-4 ``` **DEIMv2** ```python from libreyolo import LibreYOLO # Left unset, epochs, batch, imgsz and lr0 come from the released # recipe for the size that was loaded. model = LibreYOLO("LibreDEIMv2pico.pt") model.train(data="coco128.yaml", epochs=50) ``` **LoRA** ```python # Needs the lora extra: pip install "libreyolo[lora]" from libreyolo import LibreYOLO model = LibreYOLO("LibreDEIMn.pt") model.train(data="coco128.yaml", epochs=50, lora=True) ``` **Multi-GPU** ```bash libreyolo train model=LibreDEIMn.pt data=coco128.yaml \ epochs=50 device=0,1 ``` Pass `lr0` yourself on version 1. Its Python `train()` signature defaults to `4e-4`, the rate from the published COCO recipe, while the family's training config carries `1e-4` as its fine-tune default, and that lower value is what the CLI resolves when the argument is absent. The config records the measurement behind it: at the batch sizes a fine-tune actually uses, on small datasets, the COCO rate measurably degraded transfer. Version 2 resolves those defaults itself. Leaving `epochs`, `batch`, `imgsz` and `lr0` unset makes it read each one from the released recipe for the size that was loaded, so the small sizes train at their own input resolution without being told, and a value you pass overrides the recipe. `imgsz` is the argument it constrains: it has to be a positive multiple of 32, and version 2 raises before the run starts otherwise. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDEIMn.pt") # val() returns a plain dict, not an object metrics = model.val(data="coco128.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) print(metrics["metrics/precision"], metrics["metrics/recall"]) ``` **CLI** ```bash libreyolo val model=LibreDEIMn.pt data=coco128.yaml ``` **Against COCO** ```bash # coco-val-only.yaml fetches the 5000 val2017 images and skips the # training set. It carries an embedded download script, so it needs # explicit permission unless the dataset is already local. libreyolo val model=LibreDEIMn.pt data=coco-val-only.yaml \ allow_download_scripts=True ``` The rows in the benchmark table above come from the LibreYOLO benchmark harness; the note under that table records which dataset produced them and links the run records. ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | | yes | yes | yes | yes | | | | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. The matrix covers both versions as one page: where they disagree about a format, the cell shows the weaker of the two, so nothing here is oversold for whichever version you load. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. **Python** ```python # Needs the onnx extra: pip install "libreyolo[onnx]" from libreyolo import LibreYOLO model = LibreYOLO("LibreDEIMn.pt") path = model.export(format="onnx") print(path) ``` **CLI** ```bash libreyolo export model=LibreDEIMn.pt format=onnx ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreDEIMn.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreDEIMn.pt` | 640 | Detection | apache-2.0 | | `LibreDEIMs.pt` | 640 | Detection | apache-2.0 | | `LibreDEIMm.pt` | 640 | Detection | apache-2.0 | | `LibreDEIMl.pt` | 640 | Detection | apache-2.0 | | `LibreDEIMx.pt` | 640 | Detection | apache-2.0 | | `LibreDEIMv2n.pt` | 640 | Detection | apache-2.0 | | `LibreDEIMv2s.pt` | 640 | Detection | other | | `LibreDEIMv2m.pt` | 640 | Detection | other | | `LibreDEIMv2l.pt` | 640 | Detection | other | | `LibreDEIMv2x.pt` | 640 | Detection | other | | `LibreDEIMv2atto.pt` | | Detection | apache-2.0 | | `LibreDEIMv2femto.pt` | | Detection | apache-2.0 | | `LibreDEIMv2pico.pt` | | Detection | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: DEIM and DEIMv2, Intellindust AI Lab - Upstream license: Apache-2.0; the DEIMv2 DINOv3 backbone adds Meta's DINOv3 License - Upstream source: https://github.com/Intellindust-AI-Lab/DEIM - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0; the DEIMv2 DINOv3 backbone adds Meta's DINOv3 License, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours. DEIM is Apache-2.0 throughout, and so are the DEIMv2 sizes below S, which use an HGNetv2 backbone. The DEIMv2 S, M, L and X sizes take their backbone from DINOv3, so both their weights and the backbone source vendored in LibreYOLO are additionally governed by Meta's DINOv3 License Agreement, which is not an OSI license: redistribution has to carry the agreement with it, and it forbids use for military or warfare purposes, weapons development, espionage, nuclear industries and anything subject to ITAR. Their Hugging Face repositories declare both licenses, and DEIMv2 is also cited separately (arXiv 2509.20787). The four DEIMv2 sizes from S upward take their backbone from DINOv3, so their weight repositories carry both Apache-2.0 and Meta's DINOv3 License, and LibreYOLO ships the DINOv3 backbone source under that same agreement. The rest of this family, including every DEIMv2 size below S, is Apache-2.0 alone. ## Citation ```bibtex @misc{huang2024deim, title={DEIM: DETR with Improved Matching for Fast Convergence}, author={Shihua, Huang and Zhichao, Lu and Xiaodong, Cun and Yongjun, Yu and Xiao, Zhou and Xi, Shen}, booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, year={2025}, } ``` Copied from https://github.com/Intellindust-AI-Lab/DEIM#5-citation DEIMv2 is a separate paper and has its own citation block at [github.com/Intellindust-AI-Lab/DEIMv2](https://github.com/Intellindust-AI-Lab/DEIMv2#5-citation); cite that one if you used a version 2 checkpoint. --- # DeiT DeiT (Data-efficient image Transformer) is a plain Vision Transformer classifier trained on ImageNet-1k alone, with no extra pretraining data. LibreYOLO carries the tiny, small and base patch-16 sizes as a frozen, inference-only exhibit. Tasks: classify. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install DeiT needs no extra beyond the base package. ```bash pip install libreyolo ``` ## Predict This family is inference-only: `train()` raises `NotImplementedError`, so this page has no Train section. Predict, validate and export are all supported. Weights download from Hugging Face on first use and are cached locally. The `-cls` suffix in the filename is required and selects the classification task. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreDeiTb-cls.pt") result = model(SAMPLE_IMAGE) print(result.probs.top1, result.probs.top1conf) print(result.probs.top5) ``` **CLI** ```bash libreyolo predict model=LibreDeiTb-cls.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` The returned `Results` object carries a `probs` tensor instead of `boxes`; `top1` and `top5` index the 1,000 ImageNet-1k classes and `top1conf` is the softmax score for the top prediction. Each size has a fixed input resolution from its positional embedding: preprocessing resizes and center-crops to it, and passing a different `imgsz` raises rather than silently resampling. See [prediction](/docs/predict) for sources, streaming and result handling. ## Validate `val()` returns a dictionary with top-1 and top-5 accuracy, measured against a dataset laid out in the conventional `train//` and `val//` folder structure. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDeiTb-cls.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/accuracy_top1"]) print(metrics["metrics/accuracy_top5"]) ``` **CLI** ```bash libreyolo val model=LibreDeiTb-cls.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | classify | yes | yes | yes | yes | yes | | | | yes | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Running the graph in a bare runtime, with no LibreYOLO installed, is also supported, but then preprocessing and postprocessing are yours to write. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDeiTb-cls.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreDeiTb-cls.pt format=onnx libreyolo export model=LibreDeiTb-cls.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreDeiTb-cls.onnx") result = model(SAMPLE_IMAGE) print(result.probs.top1) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreDeiTt-cls.pt` | 224 | classify | apache-2.0 | | `LibreDeiTs-cls.pt` | 224 | classify | apache-2.0 | | `LibreDeiTb-cls.pt` | 224 | classify | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: DeiT, Meta Research - Upstream license: Apache-2.0 - Upstream source: https://github.com/facebookresearch/deit - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours. LibreYOLO ships the plain tiny, small and base patch-16 classifiers at fixed 224px only; the distillation-token, CaiT, DeiT III and 384px variants from the same repository are out of scope. ## Citation ```bibtex @InProceedings{pmlr-v139-touvron21a, title = {Training data-efficient image transformers & distillation through attention}, author = {Touvron, Hugo and Cord, Matthieu and Douze, Matthijs and Massa, Francisco and Sablayrolles, Alexandre and Jegou, Herve}, booktitle = {International Conference on Machine Learning}, pages = {10347--10357}, year = {2021}, volume = {139}, month = {July} } ``` Copied from https://github.com/facebookresearch/deit#-model-zoo --- # Depth Anything 3 Depth Anything 3 is a plain DINOv2 transformer trained to predict depth and camera geometry from one or more views with no architectural specialization. LibreYOLO ports its DA3MONO-LARGE checkpoint for the depth task: predict and zero-shot validation, with no training path. Tasks: depth. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install Depth Anything 3 needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreDepthAnything3l-depth.pt") result = model(SAMPLE_IMAGE, save=True) depth = result.depth_map print(depth.min, depth.max, depth.mean) ``` **CLI** ```bash libreyolo predict model=LibreDepthAnything3l-depth.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **Read the depth map** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreDepthAnything3l-depth.pt") result = model(SAMPLE_IMAGE) depth = result.depth_map # DepthMap: dense (H, W), higher = closer raw = depth.data # tensor, no metric unit or cross-image scale normalized = depth.normalized() # rescaled to [0, 1] for visualization ``` `result.depth_map` carries a dense relative inverse-depth map: higher values mean closer to the camera, and the values have no metric unit or cross-image scale. The upstream checkpoint emits positive relative depth; LibreYOLO's network wrapper inverts it and reproduces the official sky handling so the output follows LibreYOLO's shared depth contract. `save=True` writes a colormapped visualization of that map to disk; `Results.plot()` does not cover this family, since it is defined for surface normals and edges only. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants One size, `l`, at a fixed input resolution. Upstream DA3 also publishes Small and Base any-view checkpoints, a metric-depth checkpoint, and Nested and Giant checkpoints; LibreYOLO exposes none of them. Metric depth needs a different public contract than LibreYOLO's relative-inverse-depth task, and the any-view and Nested checkpoints need a multi-image camera API LibreYOLO does not offer. The Large and Giant any-view checkpoints are also CC-BY-NC-4.0 and are not referenced by any LibreYOLO download path. Training is not offered for this family. `LibreDepthAnything3.train()` raises `NotImplementedError` unconditionally; train upstream and convert a compatible DA3MONO-LARGE checkpoint with `weights/convert_depth_anything3_weights.py`. ## Validate `val()` runs the shared depth validator: it aligns each prediction to its ground truth with a per-image least-squares scale and shift, then reports the standard zero-shot relative-depth metrics, AbsRel, RMSE and the three delta thresholds. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDepthAnything3l-depth.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/abs_rel"]) print(metrics["metrics/rmse"]) print(metrics["metrics/delta1"]) ``` **CLI** ```bash libreyolo val model=LibreDepthAnything3l-depth.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | depth | yes | yes | yes | yes | yes | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. Export is restricted to five formats for this family: ONNX, TorchScript, ExecuTorch, TensorRT and OpenVINO. Requesting any other format raises `NotImplementedError` rather than attempting an unvalidated conversion. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`, with `depth_map` in place of boxes. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDepthAnything3l-depth.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreDepthAnything3l-depth.pt format=onnx libreyolo export model=LibreDepthAnything3l-depth.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreDepthAnything3l-depth.onnx") result = model(SAMPLE_IMAGE) print(result.depth_map.data.shape) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreDepthAnything3l-depth.pt` | | depth | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: Depth Anything 3, ByteDance Seed - Upstream license: Apache-2.0 - Upstream source: https://github.com/ByteDance-Seed/Depth-Anything-3 - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so the DA3MONO-LARGE checkpoint LibreYOLO ports can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy you redistribute, and it grants a patent license. It places no obligation on your own application code. The upstream project also publishes CC-BY-NC-4.0 Large, Giant and Nested checkpoints for its any-view and multi-view modes; LibreYOLO does not port those, so that non-commercial license never reaches anything this family downloads. ## Citation ```bibtex @article{depthanything3, title={Depth Anything 3: Recovering the visual space from any views}, author={Haotong Lin and Sili Chen and Jun Hao Liew and Donny Y. Chen and Zhenyu Li and Guang Shi and Jiashi Feng and Bingyi Kang}, journal={arXiv preprint arXiv:2511.10647}, year={2025} } ``` Copied from https://github.com/ByteDance-Seed/Depth-Anything-3#-citations --- # Depth Anything V2 Depth Anything V2 is a DINOv2 encoder paired with a DPT decoder that predicts a dense relative inverse-depth map from a single image. LibreYOLO supports it for the depth task: predict and zero-shot validation, with no training path. Tasks: depth. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install Depth Anything V2 needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreDepthAnythingV2s-depth.pt") result = model(SAMPLE_IMAGE, save=True) depth = result.depth_map print(depth.min, depth.max, depth.mean) ``` **CLI** ```bash libreyolo predict model=LibreDepthAnythingV2s-depth.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **Read the depth map** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreDepthAnythingV2s-depth.pt") result = model(SAMPLE_IMAGE) depth = result.depth_map # DepthMap: dense (H, W), higher = closer raw = depth.data # tensor, no metric unit or cross-image scale normalized = depth.normalized() # rescaled to [0, 1] for visualization ``` `result.depth_map` carries a dense relative inverse-depth map: higher values mean closer to the camera, and the values have no metric unit or cross-image scale. `save=True` writes a colormapped visualization of that map to disk; `Results.plot()` does not cover this family, since it is defined for surface normals and edges only. Input resolution must divide evenly by 14, the DINOv2 patch grid the DPT head builds on; LibreYOLO checks this before running and raises if it does not. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Four encoder sizes, s/b/l/g, corresponding to ViT-S/B/L/G. The checkpoint table below lists only s, b and l; no Giant checkpoint is published. All four share the same input resolution, so choosing a size trades encoder capacity, not image size. Licensing is also a factor: the Small checkpoint is Apache-2.0, while Base and Large are CC-BY-NC-4.0, see Licensing below. Training and fine-tuning are not offered for this family. `LibreDepthAnythingV2.train()` raises `NotImplementedError` unconditionally; convert a compatible upstream checkpoint instead, with `weights/convert_depth_anything_v2_weights.py`. ## Validate `val()` runs the shared depth validator: it aligns each prediction to its ground truth with a per-image least-squares scale and shift, then reports the standard zero-shot relative-depth metrics, AbsRel, RMSE and the three delta thresholds. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDepthAnythingV2s-depth.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/abs_rel"]) print(metrics["metrics/rmse"]) print(metrics["metrics/delta1"]) ``` **CLI** ```bash libreyolo val model=LibreDepthAnythingV2s-depth.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | depth | yes | yes | yes | yes | yes | | | | | | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`, with `depth_map` in place of boxes. [Export](/docs/export) lists the arguments every format accepts. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDepthAnythingV2s-depth.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreDepthAnythingV2s-depth.pt format=onnx libreyolo export model=LibreDepthAnythingV2s-depth.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreDepthAnythingV2s-depth.onnx") result = model(SAMPLE_IMAGE) print(result.depth_map.data.shape) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreDepthAnythingV2s-depth.pt` | | depth | apache-2.0 | | `LibreDepthAnythingV2l-depth.pt` | | depth | cc-by-nc-4.0 | | `LibreDepthAnythingV2b-depth.pt` | | depth | cc-by-nc-4.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: Depth Anything V2, The University of Hong Kong and TikTok - Upstream license: Apache-2.0 (Small checkpoint); CC-BY-NC-4.0 (Base and Large checkpoints) - Upstream source: https://github.com/DepthAnything/Depth-Anything-V2 - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0 (Small checkpoint); CC-BY-NC-4.0 (Base and Large checkpoints), republished at https://huggingface.co/LibreYOLO - Interpretation: The two licenses are not interchangeable. The Small checkpoint is Apache-2.0, a permissive license: it can be used in commercial and closed-source products, it asks you to keep its license text and attribution notices with any redistributed copy, and it grants a patent license. The Base and Large checkpoints are CC-BY-NC-4.0, which forbids commercial use outright and requires attribution on any redistribution, so treat them as research and evaluation weights unless you obtain separate terms from the authors. LibreYOLO's own code for this family is MIT throughout, and training is not offered for this family so there is no self-trained-weights exception to reach for. ## Citation ```bibtex @article{depth_anything_v2, title={Depth Anything V2}, author={Yang, Lihe and Kang, Bingyi and Huang, Zilong and Zhao, Zhen and Xu, Xiaogang and Feng, Jiashi and Zhao, Hengshuang}, journal={arXiv:2406.09414}, year={2024} } ``` Copied from https://github.com/DepthAnything/Depth-Anything-V2#citation --- # DETR DETR is the original detection transformer, predicting a fixed set of objects with a Hungarian-matched transformer decoder instead of anchors or a dense grid. LibreYOLO ships four sizes for detection, inference only. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install DETR needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreDETRr50.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreDETRr50.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. `conf` and `max_det` filter the query selection; `iou` is accepted for API parity but has no effect, because the decoder is a set predictor with no NMS step. See [prediction](/docs/predict) for sources, streaming and result handling. DETR is inference-only in LibreYOLO. Upstream trains for 500 epochs with Hungarian matching; that recipe is not implemented here, so `train()` raises `NotImplementedError`. ## Variants Four checkpoints combine two backbone depths, ResNet-50 or ResNet-101, with an optional dilated C5 stage: the DC5 variants keep the last backbone stage at full resolution instead of downsampling further, so the decoder reads a finer feature map from the same input size. All four share 100 learned object queries and a six-layer transformer encoder-decoder, and all run at the same input resolution. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDETRr50.pt") # val() returns a plain dict, not an object metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) print(metrics["metrics/precision"], metrics["metrics/recall"]) ``` **CLI** ```bash libreyolo val model=LibreDETRr50.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDETRr50.pt") model.export(format="onnx", imgsz=800) model.export(format="tensorrt", imgsz=800, half=True) ``` **CLI** ```bash libreyolo export model=LibreDETRr50.pt format=onnx imgsz=800 libreyolo export model=LibreDETRr50.pt format=tensorrt imgsz=800 half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreDETRr50.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreDETRr50dc5.pt` | 800 | Detection | apache-2.0 | | `LibreDETRr50.pt` | 800 | Detection | apache-2.0 | | `LibreDETRr101dc5.pt` | 800 | Detection | apache-2.0 | | `LibreDETRr101.pt` | 800 | Detection | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: DETR, Meta AI (FAIR) - Upstream license: Apache-2.0 - Upstream source: https://github.com/facebookresearch/detr - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours. The four checkpoints are the official releases linked from the Apache-2.0 repository's model zoo, and the official Hugging Face DETR card independently declares the same license. --- # DexiNed DexiNed (Dense Extreme Inception Network) is a convolutional network that predicts a dense edge-probability map from one RGB image. LibreYOLO wraps its architecture for edge detection only; no checkpoint ships with the library. Tasks: edge. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install DexiNed needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict LibreYOLO ships no DexiNed checkpoint. The officially released weights are trained on BIPED, whose published dataset terms restrict use to non-commercial purposes, so LibreYOLO does not mirror them. Convert a checkpoint you are licensed to use with `weights/convert_dexined_weights.py`, which checks the tensor keys against the runtime architecture before writing a file LibreYOLO can load directly: ```bash python weights/convert_dexined_weights.py upstream.pth weights/LibreDexiNedb-edge.pt --verify ``` **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("weights/LibreDexiNedb-edge.pt") result = model(SAMPLE_IMAGE, save=True) edges = result.edges print(edges.array.shape) # (H, W) float32 in [0, 1] print(edges.binary(0.5).sum()) # thresholded edge-pixel count ``` **CLI** ```bash libreyolo predict model=weights/LibreDexiNedb-edge.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` `result.edges` holds the result: an `(H, W)` float32 array in `[0, 1]`, with `.binary(threshold)` returning a boolean edge mask. There are no boxes, so `conf`, `iou` and `max_det` have no effect. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants DexiNed ships one size in LibreYOLO. LibreYOLO's benchmark harness has not measured this family, so there are no published numbers to compare it against. ## Validate `val()` reports BSDS-style ODS and OIS F-measures against a paired edge dataset: images beside same-stem edge maps, with an optional validity mask so padded pixels never count. `imgsz` must be divisible by the network's downsample stride, and LibreYOLO raises a clear error if it is not. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("weights/LibreDexiNedb-edge.pt") metrics = model.val(data="my-dataset.yaml", imgsz=352) print(metrics["metrics/ODS"]) # optimal-dataset-scale F-measure print(metrics["metrics/OIS"]) # optimal-image-scale F-measure ``` **CLI** ```bash libreyolo val model=weights/LibreDexiNedb-edge.pt data=my-dataset.yaml imgsz=352 ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | edge | yes | yes | yes | yes | yes | | | | | yes | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. Edge export uses a fixed-resolution, batch-1 runtime contract: `dynamic` and a `batch` other than 1 are rejected, and the exported graph outputs a single fused probability map. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` file behaves like a checkpoint and returns the same `Results`. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("weights/LibreDexiNedb-edge.pt") model.export(format="onnx", imgsz=352) model.export(format="tensorrt", imgsz=352, half=True) ``` **CLI** ```bash libreyolo export model=weights/LibreDexiNedb-edge.pt format=onnx imgsz=352 libreyolo export model=weights/LibreDexiNedb-edge.pt format=tensorrt imgsz=352 half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("weights/LibreDexiNedb-edge.onnx") result = model(SAMPLE_IMAGE) print(result.edges.array.shape) ``` ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: DexiNed, Xavier Soria - Upstream license: MIT - Upstream source: https://github.com/xavysp/DexiNed - LibreYOLO code: MIT - Weights: MIT, distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: MIT is a permissive license, so the DexiNed architecture LibreYOLO ports can be used in commercial and closed-source products, with the license text and copyright notice kept alongside any copy you redistribute. LibreYOLO ships no checkpoint for this family: the officially released weights are trained on BIPED, whose published dataset terms restrict use to non-commercial purposes, and mirroring them would carry that restriction into a nominally MIT-licensed download. Convert a checkpoint you hold a license for with `weights/convert_dexined_weights.py`; the MIT code license does not change the terms attached to whatever checkpoint you convert. LibreYOLO publishes no DexiNed checkpoint. Nothing is mirrored under the LibreYOLO organization; convert a checkpoint you hold a license for with `weights/convert_dexined_weights.py` instead. ## Citation ```bibtex @INPROCEEDINGS {xsoria2020dexined, author = {X. Soria and E. Riba and A. Sappa}, booktitle = {2020 IEEE Winter Conference on Applications of Computer Vision (WACV)}, title = {Dense Extreme Inception Network: Towards a Robust CNN Model for Edge Detection}, year = {2020}, volume = {}, issn = {}, pages = {1912-1921}, keywords = {image edge detection;convolution;training;feeds;machine learning;task analysis;kernel}, doi = {10.1109/WACV45572.2020.9093290}, url = {https://doi.ieeecomputersociety.org/10.1109/WACV45572.2020.9093290}, publisher = {IEEE Computer Society}, address = {Los Alamitos, CA, USA}, month = {mar} } @article{soria2023dexined_ext, title = {Dense extreme inception network for edge detection}, journal = {Pattern Recognition}, volume = {139}, pages = {109461}, year = {2023}, issn = {0031-3203}, doi = {https://doi.org/10.1016/j.patcog.2023.109461}, url = {https://www.sciencedirect.com/science/article/pii/S0031320323001619}, author = {Xavier Soria and Angel Sappa and Patricio Humanante and Arash Akbarinia}, keywords = {Edge detection, Deep learning, CNN, Contour detection, Boundary detection, Segmentation} } ``` Copied from https://github.com/xavysp/DexiNed#citation --- # DINO-DETR DINO-DETR, published by IDEA Research as DINO, combines contrastive denoising training with mixed query selection on top of Deformable DETR's sparse attention. LibreYOLO ships three sizes for detection, inference only. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install DINO-DETR needs no optional extra. Everything it imports is in the base install, using the same pure-PyTorch multi-scale deformable attention core as LibreYOLO's Deformable DETR family. ```bash pip install libreyolo ``` Installing `libreyolo[hub-kernels]` is optional. Once the `kernels` package is present, LibreYOLO fetches a compiled multi-scale deformable attention kernel from the Hugging Face Hub at runtime and uses it in place of the pure-PyTorch core; `LIBREYOLO_HUB_KERNELS=0` turns it back off. ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreDINODETRr50.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreDINODETRr50.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. `conf` and `max_det` filter the query selection; `iou` is accepted for API parity but has no effect, because the decoder is a set predictor with no NMS step. See [prediction](/docs/predict) for sources, streaming and result handling. DINO-DETR is inference-only in LibreYOLO. Upstream trains with contrastive denoising and Hungarian matching; that recipe is not implemented here, so `train()` raises `NotImplementedError`. ## Variants Three checkpoints, all at the same input resolution. `r50` and `r50s5` share a ResNet-50 backbone and differ in how many feature-map scales feed the decoder, four against five. `swinl` swaps the backbone for Swin-L and also samples five scales. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDINODETRr50.pt") # val() returns a plain dict, not an object metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) print(metrics["metrics/precision"], metrics["metrics/recall"]) ``` **CLI** ```bash libreyolo val model=LibreDINODETRr50.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDINODETRr50.pt") model.export(format="onnx", imgsz=800) model.export(format="tensorrt", imgsz=800, half=True) ``` **CLI** ```bash libreyolo export model=LibreDINODETRr50.pt format=onnx imgsz=800 libreyolo export model=LibreDINODETRr50.pt format=tensorrt imgsz=800 half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreDINODETRr50.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreDINODETRr50.pt` | 800 | Detection | apache-2.0 | | `LibreDINODETRr50s5.pt` | 800 | Detection | apache-2.0 | | `LibreDINODETRswinl.pt` | 800 | Detection | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: DINO-DETR, IDEA Research - Upstream license: Apache-2.0 - Upstream source: https://github.com/IDEA-Research/DINO - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours. The three checkpoints come from the authors' Google Drive release rather than a Hugging Face model card, and the upstream repository does not attach a license to the checkpoint files individually, so the redistribution basis is the repository-level Apache-2.0 declaration rather than a checkpoint-specific grant. The three official checkpoints come from the authors' Google Drive release folder, not a Hugging Face model card. The upstream repository declares Apache-2.0 at the repository level but does not attach a license file or license metadata to the checkpoints themselves, so the redistribution basis is that repository-level declaration rather than a checkpoint-specific grant. Every LibreYOLO mirror ships the verbatim upstream Apache-2.0 license text alongside a notice explaining this. ## Citation ```bibtex @misc{zhang2022dino, title={DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection}, author={Hao Zhang and Feng Li and Shilong Liu and Lei Zhang and Hang Su and Jun Zhu and Lionel M. Ni and Heung-Yeung Shum}, year={2022}, eprint={2203.03605}, archivePrefix={arXiv}, primaryClass={cs.CV} } @inproceedings{li2022dn, title={Dn-detr: Accelerate detr training by introducing query denoising}, author={Li, Feng and Zhang, Hao and Liu, Shilong and Guo, Jian and Ni, Lionel M and Zhang, Lei}, booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pages={13619--13627}, year={2022} } @inproceedings{ liu2022dabdetr, title={{DAB}-{DETR}: Dynamic Anchor Boxes are Better Queries for {DETR}}, author={Shilong Liu and Feng Li and Hao Zhang and Xiao Yang and Xianbiao Qi and Hang Su and Jun Zhu and Lei Zhang}, booktitle={International Conference on Learning Representations}, year={2022}, url={https://openreview.net/forum?id=oMI9PjOb9Jl} } ``` Copied from https://github.com/IDEA-Research/DINO#bibtex --- # DINOv2 DINOv2 is a self-supervised vision transformer trained by Meta AI to produce general-purpose image features without labels. LibreYOLO wraps its DINOv2-with-Registers backbone for three tasks: semantic segmentation, classification and whole-image embedding. Tasks: semantic, classify, embed. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install LibreDINOv2 registers only when `transformers` is installed, the same optional dependency RF-DETR needs for its DINOv2 backbone, so it needs the same extra. ```bash pip install "libreyolo[rfdetr]" ``` ## Predict LibreYOLO does not publish a LibreDINOv2 checkpoint. Construct the wrapper directly instead of loading a file: `model_path=None` (the default) downloads Meta's Apache-2.0 `facebook/dinov2-with-registers-small` backbone from Hugging Face on first use. `task=` selects what runs on top of it. **Semantic** ```python from libreyolo import SAMPLE_IMAGE from libreyolo.models.dinov2.model import LibreDINOv2 # No LibreYOLO-hosted checkpoint exists for this family: this # downloads the Apache-2.0 DINOv2-with-Registers-small backbone from # Meta's Hugging Face org. The dense head starts at random # initialization until you train it (see Train below). model = LibreDINOv2(size="s", task="semantic", nb_classes=19) result = model(SAMPLE_IMAGE) mask = result.semantic_mask print(mask.data.shape, mask.classes) ``` **Classify** ```python from libreyolo import SAMPLE_IMAGE from libreyolo.models.dinov2.model import LibreDINOv2 # nb_classes= is your dataset's class count; the linear head starts # at random initialization until you train it. model = LibreDINOv2(size="s", task="classify", nb_classes=10) result = model(SAMPLE_IMAGE) print(result.probs.top1, result.probs.top1conf) ``` **Embed** ```python from libreyolo import SAMPLE_IMAGE from libreyolo.models.dinov2.model import LibreDINOv2 # Bypasses every task head: the backbone alone is enough, so this # needs no fine-tuning to be useful. model = LibreDINOv2(size="s", task="embed") result = model(SAMPLE_IMAGE) print(result.embeddings.data.shape) # (1, D), L2-normalized ``` **Embed a batch** ```python from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="embed") # Convenience wrapper: runs predict() and stacks every row into one # (N, D) tensor. features = model.embed(["a.jpg", "b.jpg", "c.jpg"]) print(features.shape) ``` `task="semantic"` and `task="classify"` add a dense or linear head on top of the backbone; that head is randomly initialized and only useful after you train it (see [Train](#train)). `task="embed"` skips every head and returns the backbone's final normalized CLS token as one whole-image row in `result.embeddings`, so it needs no training at all. `result.boxes` is always `None`: none of the three tasks produce per-instance detections. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants `size` selects the RF-DETR-style projector width layered on top of the backbone, not the backbone itself: every size shares the same DINOv2-S (small) encoder. Semantic segmentation runs at DINOv2's native square patch grid; classification and embedding run at the smaller classification resolution used to train the linear probe. ## Train `task="semantic"` and `task="classify"` both train; `task="embed"` has no class-dependent head to fit and raises `NotImplementedError` if you call `train()` on it. **Semantic** ```python from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="semantic", nb_classes=19) model.train(data="my-dataset.yaml", epochs=100, batch_size=4, lr=1e-4) ``` **Classify** ```python from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="classify", nb_classes=10) model.train(data="my-dataset.yaml", epochs=100, batch_size=4, lr=1e-4) ``` **Multi-GPU** ```python from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="semantic", nb_classes=19) model.train( data="my-dataset.yaml", epochs=100, batch_size=4, lr=1e-4, device="0,1", ) ``` The primary keyword arguments here are `batch_size` and `lr`, not `batch` and `lr0` used by most other families; `batch` and `lr0` are still accepted and mapped onto them, but passing both raises a conflict error. `output_dir=` (default `"runs/train"`) replaces `project=`/`name=` as the primary way to place a run, though passing `project=`/`name=` directly still works. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary of `metrics/` keys: mIoU and pixel accuracy for `task="semantic"`, top-1 and top-5 accuracy for `task="classify"`. `task="embed"` has no ground truth to score against and raises `NotImplementedError` if you call `val()` on it. **Semantic** ```python from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="semantic", nb_classes=19) metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mIoU"]) print(metrics["metrics/pixel_accuracy"]) ``` **Classify** ```python from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="classify", nb_classes=10) metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/accuracy_top1"]) print(metrics["metrics/accuracy_top5"]) ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | semantic | yes | yes | yes | yes | yes | | | | | | | | | classify | yes | yes | yes | yes | yes | | | | | | | yes | | embed | yes | yes | yes | yes | yes | | | | | yes | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. Each task supports a different subset of formats, shown above. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts. **Semantic** ```python from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="semantic", nb_classes=19) model.export(format="onnx") ``` **Classify** ```python from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="classify", nb_classes=10) model.export(format="onnx") ``` **Embed** ```python from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="embed") model.export(format="tflite") ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. Export # names the file from the task, here LibreDINOv2s-sem.onnx. model = LibreYOLO("LibreDINOv2s-sem.onnx") result = model(SAMPLE_IMAGE) ``` ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: DINOv2, Meta AI (FAIR) - Upstream license: Apache-2.0 - Upstream source: https://github.com/facebookresearch/dinov2 - LibreYOLO code: MIT - Weights: Apache-2.0, distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. LibreYOLO does not host or republish a DINOv2 checkpoint of its own: LibreDINOv2 downloads the pretrained backbone directly from Meta's facebook/dinov2-with-registers-small repository on Hugging Face the first time it runs, unmodified. The semantic and classification heads start at random initialization until you train them, since LibreYOLO does not publish a trained head for this family. The "Weights" row above names the license that applies, Apache-2.0, but nothing is actually republished under the LibreYOLO Hugging Face org for this family: LibreYOLO hosts no LibreDINOv2 checkpoint of its own. What `LibreDINOv2(model_path=None)` downloads is Meta's own `facebook/dinov2-with-registers-small` repository, untouched. ## Citation ```bibtex @misc{oquab2023dinov2, title={DINOv2: Learning Robust Visual Features without Supervision}, author={Oquab, Maxime and Darcet, Timothée and Moutakanni, Theo and Vo, Huy V. and Szafraniec, Marc and Khalidov, Vasil and Fernandez, Pierre and Haziza, Daniel and Massa, Francisco and El-Nouby, Alaaeldin and Howes, Russell and Huang, Po-Yao and Xu, Hu and Sharma, Vasu and Li, Shang-Wen and Galuba, Wojciech and Rabbat, Mike and Assran, Mido and Ballas, Nicolas and Synnaeve, Gabriel and Misra, Ishan and Jegou, Herve and Mairal, Julien and Labatut, Patrick and Joulin, Armand and Bojanowski, Piotr}, journal={arXiv:2304.07193}, year={2023} } ``` Copied from https://github.com/facebookresearch/dinov2#citing-dinov2 --- # Dome-DETR A tiny-object specialist built on D-FINE: a density head decides where objects are, encoder attention is restricted to the windows that hold them, and the query count is sized from that density instead of being fixed. LibreYOLO supports it for detection. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install Dome-DETR needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict There is nothing to auto-download. LibreYOLO does not host these weights, so the flow is: fetch the upstream checkpoint, convert it once, then load the converted file by path. [Licensing](#licensing) explains why. **Convert, then predict** ```bash # No Dome-DETR weights are hosted by LibreYOLO, so the checkpoint is # fetched from the upstream repository and converted once. hf download RicePasteM/Dome-DETR --include 'best_ckpts_dome_2026/*' \ --local-dir dome-ckpts python weights/convert_domedetr_weights.py \ dome-ckpts/best_ckpts_dome_2026/dome-s-visdrone_converted.pth \ LibreDOMEDETRs-visdrone.pt --size s --variant visdrone ``` **Python** ```python from libreyolo import LibreYOLO # A local path, not a bare name: nothing downloads for this family. model = LibreYOLO("LibreDOMEDETRs-visdrone.pt") result = model("drone-frame.jpg", save=True) for box in result.boxes: print(result.names[int(box.cls)], box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreDOMEDETRs-visdrone.pt source=drone-frame.jpg save=True ``` **Class names** ```python from libreyolo import LibreYOLO # There is no COCO checkpoint, so the classes come from the dataset the # weights were trained on and are read from the checkpoint metadata. aitod = LibreYOLO("LibreDOMEDETRs-aitod.pt") print(aitod.model.names) # 9 AI-TOD-V2 classes visdrone = LibreYOLO("LibreDOMEDETRs-visdrone.pt") print(visdrone.model.names) # 12 VisDrone classes ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. `conf` and `max_det` filter the query selection; `iou` is accepted for API parity but has no effect, because the decoder is a set predictor with no NMS step. See [prediction](/docs/predict) for sources, streaming and result handling. Two capabilities are off for this family. CUDA graph capture is disabled, because PAQI's query count is data dependent and the forward pass therefore changes shape from image to image, which is exactly what graph capture cannot absorb. Test-time augmentation runs at a single fixed square size, so a multi-scale TTA request is a no-op. ## Variants Three sizes, s, m and l, all at 800 by 800. The size selects the backbone, and the dataset the weights came from selects the decoder depth and the query budget, so a size code on its own does not identify a graph. AI-TOD-V2 weights select between 300 and 1500 queries per image, VisDrone weights between 250 and 500, and the large model runs four decoder layers on AI-TOD-V2 against six on VisDrone. Dome-DETR is D-FINE with three additions. DeFE predicts a density map. MWAS uses that map to restrict encoder attention to the windows that actually hold objects, rather than attending everywhere. PAQI sizes the query set from the same density instead of decoding a fixed 300. The gain concentrates where objects are smallest, and narrows as they grow: upstream's own ablation moves AP on very tiny objects from 14.0 to 17.8 while AP on medium objects moves only from 45.4 to 46.4. Treat it as a companion to [D-FINE](/docs/models/d-fine) for aerial, drone and remote-sensing imagery, not a replacement for it. LibreYOLO publishes no benchmark rows for this family, because it publishes no checkpoints to benchmark. ## Train Dome-DETR is trainable. Training runs upstream's full objective: the D-FINE losses plus DeFE density and count supervision, with padded queries masked out of the classification terms and per-image denoising attention masks so one image's padding cannot leak into another's. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDOMEDETRs-visdrone.pt") model.train(data="my-dataset.yaml", epochs=160, imgsz=800, batch=4, lr0=2e-4) ``` **CLI** ```bash libreyolo train model=LibreDOMEDETRs-visdrone.pt data=my-dataset.yaml \ epochs=160 imgsz=800 batch=4 lr0=2e-4 ``` **Multi-GPU** ```bash libreyolo train model=LibreDOMEDETRs-visdrone.pt data=my-dataset.yaml \ epochs=160 device=0,1 batch=4 ``` The configuration inherits D-FINE's recipe and changes what MWAS requires. `imgsz` is 800, `lr0` is `2e-4`, the backbone parameter group is scaled by `backbone_lr_mult=0.1`, and `multi_scale` is forced off, because MWAS windows need the input to stay divisible by stride 8. `batch` defaults to 4 rather than D-FINE's 16: PAQI pads every batch to its widest member, so memory tracks the busiest image in the batch rather than the average one. One honest caveat about accuracy. Upstream trains for 160 epochs on `MultiStepLR(milestones=[80, 120], gamma=0.8)`, while these defaults run D-FINE's flat-cosine schedule for the same 160 epochs. That schedule has not been reproduced here and the paper's AP numbers have not been reproduced either, so read them as the upstream authors' results rather than as a promise that this recipe reaches them. Supply the upstream schedule if matching the paper is the goal. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary keyed by metric name, and prints per-class results when `verbose` is left on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDOMEDETRs-visdrone.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreDOMEDETRs-visdrone.pt data=my-dataset.yaml ``` Validation runs against your own dataset in the format you trained on. The library's COCO validation gate does not apply here, since no COCO checkpoint exists for this family to be measured against. ## Export Export is not supported, for every format, and asking for one raises rather than producing a file. The reason is PAQI. It decides the query count per image, from density-filtered proposals and a greedy density-adaptive suppression loop, so the decoder's output length is a property of the input rather than of the graph. Tracing bakes in whichever count the tracing image happened to produce, which yields an artifact that silently returns wrong results for every other image. A static formulation would have to unroll that suppression over all 250 to 1500 candidates, and collapsing to a fixed top-k would remove exactly the tiny-object recall the family exists for. If you need an exportable detection transformer, [D-FINE](/docs/models/d-fine) is the one to reach for. ## Checkpoints There are none to list. LibreYOLO publishes no Dome-DETR weights, and no name of the form `LibreDOMEDETR-.pt` resolves to a download. Upstream publishes six checkpoints, s, m and l for each of two datasets: AI-TOD-V2 with 9 classes and VisDrone with 12. There is no COCO checkpoint, so a canonical filename always carries the dataset suffix, and the class names travel in the checkpoint metadata rather than coming from a family constant. Asking for a bare `LibreDOMEDETRs.pt` raises immediately with a message naming the two real filenames and the conversion command, rather than attempting a download that would 404. `weights/convert_domedetr_weights.py` does the conversion. It rebuilds the LibreYOLO graph, loads the upstream tensors into it, and refuses to write anything if a single key is missing, unexpected or the wrong shape, so a converted file is either an exact match or it does not exist. Point it at an upstream `.pth` and pass the size and the variant: ```bash python weights/convert_domedetr_weights.py \ dome-ckpts/best_ckpts_dome_2026/aitod-s-best.pth \ LibreDOMEDETRs-aitod.pt --size s --variant aitod ``` On numerical fidelity, `weights/parity_domedetr.py` compares this port against the upstream implementation across all six checkpoints and reports `max_abs_diff == 0.0` on both `pred_logits` and `pred_boxes`, after first checking the MWAS window mask bit for bit, and separately diffs every loss term against upstream's criterion. Be clear about what that is: a manual script that needs the upstream checkout and the published checkpoints on disk, run by hand. It is not part of continuous integration, and no CI job reproduces it. ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: Dome-DETR, The Dome-DETR Authors - Upstream license: unclear, not redistributed - Upstream source: https://github.com/RicePasteM/Dome-DETR - LibreYOLO code: Apache-2.0 - Weights: unclear, not redistributed, distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: The code and the weights part company here. The upstream repository is Apache-2.0, permissive and safe for commercial and closed-source use, and LibreYOLO's own port is MIT, so nothing restricts the architecture or the training code. The weights are the unresolved part: the upstream model card carries no license field in its metadata, and its prose states that the project is Apache-2.0 while also restricting the material to academic research purposes only. Those two readings do not agree, and the stricter one is not a grant to redistribute, so LibreYOLO mirrors nothing for this family and hosts no checkpoint. Download the six upstream checkpoints yourself and convert them with weights/convert_domedetr_weights.py, and read the upstream terms before using them for anything commercial. Weights you train yourself on your own data derive from no upstream checkpoint and are yours. The weights are the reason this family is not mirrored. The upstream model card carries no license field in its metadata, and its prose states that the project is Apache-2.0 while also restricting the material to academic research purposes only. Those two readings do not agree, and the stricter one is not a redistribution grant, so LibreYOLO links the upstream repository instead of copying the files, pending clarification. The same reasoning is what governs [YOLO-NAS](/docs/models/yolo-nas) here. The code is a separate question and a clearer one. The upstream repository is Apache-2.0, LibreYOLO's port is MIT, and weights you train yourself on your own data are yours. ## Citation Dome-DETR was published at ACM Multimedia 2025 as "Dome-DETR: DETR with Density-Oriented Feature-Query Manipulation for Efficient Tiny Object Detection". The preprint is at [arxiv.org/abs/2505.05741](https://arxiv.org/abs/2505.05741). The authors publish no BibTeX block in their repository, so none is reproduced here rather than assembled by hand. --- # EdgeCrafter A compact vision transformer for dense prediction on edge hardware, published upstream as three sibling models: ECDet, ECPose and ECSeg. LibreYOLO loads all three as one family, with the task carried by the checkpoint. Tasks: Detection, Pose, Instance segmentation. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install EdgeCrafter needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` Adapter fine-tuning with `lora=True` is the exception, and needs the `lora` extra. ```bash pip install "libreyolo[lora]" ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreECs.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreECs.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **Pose** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The -pose suffix in the filename selects the keypoint head, so no # task argument is needed here. model = LibreYOLO("LibreECs-pose.pt") result = model(SAMPLE_IMAGE, save=True) print(result.keypoints.xy) print(result.boxes.conf) ``` **Instance segmentation** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreECs-seg.pt") result = model(SAMPLE_IMAGE, save=True) print(result.masks.data.shape) ``` The task comes from the filename, so a `-pose` or `-seg` checkpoint selects its own head and takes no task argument. All three return the `Results` object every family returns, with `result.keypoints` added for pose and `result.masks` for segmentation. Pose covers one class, person, with the 17 COCO keypoints, and the count is fixed when the model is built. It has no box head, so each pose box is the bounding extent of its own keypoints, and the third keypoint channel is a constant rather than a per-point score. `conf` and `max_det` filter the query selection; `iou` is accepted for API parity but has no effect, because all three heads decode a set of queries with no NMS step. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Four sizes. They all run at the same input resolution, so the table separates them by parameter count and accuracy. | Checkpoint | Input (px) | mAP 50-95 | Params (M) | | --- | --- | --- | --- | | `LibreECl` | 640 | 60.1 | 32.97 | | `LibreECm` | 640 | 58.4 | 19.43 | | `LibreECs` | 640 | 54.3 | 9.88 | | `LibreECx` | 640 | 61.1 | 49.94 | COCO val2017, 500 images. Published on https://www.visionanalysis.org/ Interactive chart: https://www.visionanalysis.org/embed/scatter?highlight=ec-l%2Cec-m%2Cec-s%2Cec-x&title=EdgeCrafter%20on%20COCO&subtitle=Accuracy%20against%20latency%2C%20every%20benchmarked%20model Upstream publishes ECDet, ECPose and ECSeg as three separate models rather than one model with three heads. They share the ECViT backbone and the hybrid encoder and differ only in the head, so LibreYOLO folds them into a single family and lets the checkpoint filename carry the task. A size letter therefore means the same backbone and encoder across all three, and predict, validate and export take the same arguments whichever one you load. ## Train All three tasks train through `train()`, which reads the task from the loaded checkpoint and picks the matching trainer. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreECs.pt") model.train( data="my-dataset.yaml", epochs=50, imgsz=640, batch=8, lr0=5e-4, ) ``` **CLI** ```bash libreyolo train model=LibreECs.pt data=my-dataset.yaml epochs=50 imgsz=640 batch=8 lr0=5e-4 ``` **Pose** ```python from libreyolo import LibreYOLO # Needs a single-class keypoint dataset whose data.yaml declares # kpt_shape, and imgsz at the checkpoint's native size. model = LibreYOLO("LibreECs-pose.pt") model.train( data="my-pose-dataset.yaml", epochs=50, imgsz=640, ) ``` **Instance segmentation** ```python from libreyolo import LibreYOLO # Needs polygon labels, and imgsz at the checkpoint's native size. model = LibreYOLO("LibreECs-seg.pt") model.train( data="my-dataset.yaml", epochs=50, imgsz=640, ) ``` **LoRA** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreECs.pt") model.train( data="my-dataset.yaml", epochs=50, lora=True, ) ``` What has been checked for detection and segmentation: inference parity against upstream at 1e-5, layer by layer and per size, and that the loss and a single training step run on synthetic input. What has not, per `train()`'s own docstring: convergence of a full fine-tune, multi-GPU training, the stop-augmentation best-reload step, and the Objects365 to COCO class remap. The pose path follows DETRPose's published recipe, a Hungarian matcher over class, keypoint L1 and OKS costs with contrastive keypoint denoising, and its convergence has not been checked end to end either. Left alone, the trainer runs 74 epochs at `lr0=5e-4` with mixed precision on, following upstream's recipe: AdamW, a flat cosine schedule, EMA at 0.9999 and ImageNet-normalized inputs. Pose and segmentation both require `imgsz` at the checkpoint's native size, because their evaluation anchor grid is built when the model is constructed; a different value raises before the run starts. Pose also requires a single-class dataset whose `data.yaml` declares `kpt_shape`, with a keypoint count matching the head. `lora=True` applies to detection only; pose and segmentation raise a `ValueError` on it. On Apple silicon the trainer keeps the run on the GPU and sends one operation to CPU, the grid-sample backward inside deformable attention, which PyTorch does not implement in Metal. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary keyed by metric name, and prints per-class results when `verbose` is left on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreECs.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreECs.pt data=my-dataset.yaml ``` **Pose** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreECs-pose.pt") metrics = model.val(data="my-pose-dataset.yaml") print(metrics["metrics/keypoints_mAP50-95"]) print(metrics["metrics/keypoints_mAP50"]) ``` **Instance segmentation** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreECs-seg.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95(M)"]) # masks print(metrics["metrics/mAP50-95(B)"]) # boxes ``` Pose reports keypoint OKS metrics under `metrics/keypoints_*`. Segmentation reports masks under the plain `metrics/mAP50-95` key and repeats both views in one pass, boxes under `(B)` and masks under `(M)`. ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | yes | yes | | | | | yes | | Pose | yes | yes | yes | yes | yes | yes | | | | | | | | Instance segmentation | yes | yes | yes | yes | yes | yes | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Pose and segmentation export at a fixed 640 by 640 input rather than dynamic shapes, and several detection targets are fixed-canvas too, including OpenVINO, Paddle, MNN, ExecuTorch and Core AI. [Export](/docs/export) lists the arguments every format accepts and the extras a few of them add. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreECs.pt") model.export(format="onnx", imgsz=640) model.export(format="tensorrt", imgsz=640, half=True) ``` **CLI** ```bash libreyolo export model=LibreECs.pt format=onnx imgsz=640 libreyolo export model=LibreECs-pose.pt format=onnx imgsz=640 libreyolo export model=LibreECs-seg.pt format=onnx imgsz=640 ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreECs.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreECs-pose.pt` | 640 | Pose | apache-2.0 | | `LibreECm-pose.pt` | 640 | Pose | apache-2.0 | | `LibreECl-pose.pt` | 640 | Pose | apache-2.0 | | `LibreECx-pose.pt` | 640 | Pose | apache-2.0 | | `LibreECs-seg.pt` | 640 | Instance segmentation | apache-2.0 | | `LibreECm-seg.pt` | 640 | Instance segmentation | apache-2.0 | | `LibreECl-seg.pt` | 640 | Instance segmentation | apache-2.0 | | `LibreECx-seg.pt` | 640 | Instance segmentation | apache-2.0 | | `LibreECs.pt` | 640 | Detection | apache-2.0 | | `LibreECm.pt` | 640 | Detection | apache-2.0 | | `LibreECl.pt` | 640 | Detection | apache-2.0 | | `LibreECx.pt` | 640 | Detection | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: EdgeCrafter, Intellindust AI Lab - Upstream license: Apache-2.0 - Upstream source: https://github.com/Intellindust-AI-Lab/EdgeCrafter - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. One license covers all three upstream models, so the detection, pose and segmentation weights carry identical terms, and weights you train yourself on your own data are yours. ## Citation ```bibtex @article{liu2026edgecrafter, title={EdgeCrafter: Compact ViTs for Edge Dense Prediction via Task-Specialized Distillation}, author={Liu, Longfei and Hou, Yongjie and Li, Yang and Wang, Qirui and Sha, Youyang and Yu, Yongjun and Wang, Yinzhi and Ru, Peizhe and Yu, Xuanlong and Shen, Xi}, journal={arXiv}, year={2026} } ``` Copied from https://github.com/Intellindust-AI-Lab/EdgeCrafter#-citation --- # EdgeTAM EdgeTAM is an on-device variant of SAM 2, built for mobile inference speed while keeping the same point-and-box promptable workflow. LibreYOLO supports its image segmentation path through a dedicated LibreSAM factory, separate from the LibreYOLO() detector factory. Tasks: Instance segmentation. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install EdgeTAM needs the `sam` extra, which pulls in `transformers` and `timm`. ```bash pip install "libreyolo[sam]" ``` ## Predict `LibreSAM(...)` (or the family-specific `LibreEdgeTAM(...)`) is a separate entry point from `LibreYOLO(...)`: it returns a promptable segmenter rather than a detector, because a forward pass here is meaningless without a spatial prompt. There is no `libreyolo predict` CLI command for this family; use the Python API. Only image segmentation is supported; EdgeTAM's video tracking is out of scope here. **Point and box prompts** ```python from libreyolo import LibreSAM, SAMPLE_IMAGE # EdgeTAM has a single size, "edge". Aliases: "edgetam", "edge-tam", # "edgetam-edge". model = LibreSAM("edgetam") # A point prompt: [x, y] in pixel coordinates, label 1 = foreground. result = model.predict(SAMPLE_IMAGE, points=[640, 420], labels=[1]) print(result.masks.xy) # polygon per mask print(result.boxes.xyxy) # tight box derived from the mask # A box prompt instead of a point. result = model.predict(SAMPLE_IMAGE, bboxes=[300, 200, 900, 700]) # No prompt at all segments the whole image (a simplified automatic # mask generator, not the exhaustive reference one). result = model.predict(SAMPLE_IMAGE) ``` **Encode once, prompt many** ```python from libreyolo import LibreEdgeTAM, SAMPLE_IMAGE model = LibreEdgeTAM() # The image encoder is the expensive part. set_image() runs it once; # every predict() call after that reuses the cached embedding. model.set_image(SAMPLE_IMAGE) a = model.predict(points=[640, 420], labels=[1]) b = model.predict(bboxes=[300, 200, 900, 700]) model.reset_image() ``` A point prompt accepts `[x, y]` for one object, `[[x, y], ...]` for several, or numpy arrays; `labels` marks each point `1` (foreground) or `0` (background) and defaults to all foreground. A box prompt takes `[x1, y1, x2, y2]` or a list of boxes, one mask per box. Omitting both prompts segments the whole image by prompting a dense grid and keeping the confident, non-overlapping masks; this "segment everything" mode is simplified against the reference automatic mask generator and can under-segment crowded scenes, so a real point or box prompt is the precise path. `conf` filters by predicted mask quality (IoU), not a detection confidence: pass `0.0` to keep every candidate. `multimask=True` returns all three of SAM's whole-versus-part ambiguity masks per prompt instead of the single best one. `device=` moves the model and, if a `set_image()` session is active, its cached embedding. Every mask carries class id `0`, named `"object"`, since a promptable mask has no fixed class set. `train()`, `val()`, `export()` and `track()` all raise `NotImplementedError` for this family: image inference is what LibreYOLO supports here. See [prediction](/docs/predict) for source types. ## Variants One size, edge, at a fixed input resolution, so choosing this family over the rest of the SAM tier is a hardware decision rather than a sizing one: EdgeTAM exists specifically for constrained, on-device inference. ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreEdgeTAM.pt` | | Instance segmentation | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: EdgeTAM, Meta Reality Labs - Upstream license: Apache-2.0 - Upstream source: https://github.com/facebookresearch/EdgeTAM - LibreYOLO code: MIT - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so this code and these weights can be used in commercial and closed-source products. It asks you to keep the license text and attribution notices with any copy you redistribute, and it grants a patent license. LibreYOLO does not vendor EdgeTAM's model source: it calls the Apache-2.0 Transformers adapter and reproduces the pinned upstream image and prompt-coordinate transforms from facebookresearch/EdgeTAM commit 7711e012a30a2402c4eaab637bdb00a521302c91. The republished LibreYOLO/LibreEdgeTAM snapshot is converted from facebook/EdgeTAM revision 14d7ecc48c656b94e5184519f698cd5386c5a2bf, checked tensor-by-tensor against a Transformers-format reference before publication, and tagged Apache-2.0 on the LibreYOLO Hugging Face org. LibreYOLO ships image inference only; EdgeTAM's video tracking is out of scope for this family. ## Citation ```bibtex @article{zhou2025edgetam, title={EdgeTAM: On-Device Track Anything Model}, author={Zhou, Chong and Zhu, Chenchen and Xiong, Yunyang and Suri, Saksham and Xiao, Fanyi and Wu, Lemeng and Krishnamoorthi, Raghuraman and Dai, Bo and Loy, Chen Change and Chandra, Vikas and Soran, Bilge}, journal={arXiv preprint arXiv:2501.07256}, year={2025} } ``` Copied from https://github.com/facebookresearch/EdgeTAM#citing-edgetam --- # EfficientDet EfficientDet pairs an EfficientNet backbone with a repeated bi-directional feature pyramid network (BiFPN) and scales depth, width and resolution together across five sizes. LibreYOLO ships it as an inference-only detector. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install EfficientDet needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreEfficientDetd0.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreEfficientDetd0.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. EfficientDet decodes anchor-based candidates and then runs class-wise non-maximum suppression, so `conf`, `iou` and `max_det` all have a real effect here. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Five sizes, D0 through D4. Each step up pairs a larger EfficientNet backbone with a deeper, wider BiFPN and a deeper prediction head, so parameter count and compute grow together, following the paper's compound-scaling rule. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreEfficientDetd0.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreEfficientDetd0.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | | | | yes | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreEfficientDetd0.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreEfficientDetd0.pt format=onnx libreyolo export model=LibreEfficientDetd0.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreEfficientDetd0.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreEfficientDetd0.pt` | | Detection | apache-2.0 | | `LibreEfficientDetd1.pt` | | Detection | apache-2.0 | | `LibreEfficientDetd2.pt` | | Detection | apache-2.0 | | `LibreEfficientDetd3.pt` | | Detection | apache-2.0 | | `LibreEfficientDetd4.pt` | | Detection | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: EfficientDet, Google Research - Upstream license: Apache-2.0 - Upstream source: https://github.com/rwightman/efficientdet-pytorch - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code. LibreYOLO's D0-D4 checkpoints are converted from Google's official TensorFlow-trained release assets through the Apache-2.0 rwightman/efficientdet-pytorch project; those release assets carry no separate per-checkpoint license, so redistribution here rests on Apache-2.0 implied by the releasing project, the same basis rwightman/efficientdet-pytorch itself uses for its own converted weights. LibreYOLO's D0-D4 checkpoints are converted through the Apache-2.0 rwightman/efficientdet-pytorch project, which itself mirrors the official TensorFlow-trained weights from google/automl without changing learned tensors. No source from the LGPL-licensed zylo117/Yet-Another-EfficientDet-Pytorch project was consulted or used. --- # EfficientNetV2 EfficientNetV2 is an image classifier whose depth, width and per-stage block choices were found by neural architecture search, jointly optimizing for accuracy and training speed rather than accuracy alone. LibreYOLO supports it for one task: classification. Tasks: classify. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install EfficientNetV2 needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreEfficientNetV2b0-cls.pt") result = model(SAMPLE_IMAGE, save=True) print(result.probs.top1, result.probs.top1conf) print(result.probs.top5) ``` **CLI** ```bash libreyolo predict model=LibreEfficientNetV2b0-cls.pt source=cat.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different model is a one line change. A classifier carries no boxes or masks: `result.probs` holds the whole-image prediction, with `top1`, `top5`, `top1conf` and `top5conf`. `conf`, `iou` and `max_det` are accepted for API parity but have no effect, since there is nothing to threshold or suppress on a single probability vector. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Four sizes, b0 through b3, each evaluated at its own resolution and crop ratio rather than sharing one input size across the family. Picking a size is a straight parameter-count-for-accuracy trade. The task is fixed: every size covers classification only. The weights filename ends `-cls.pt` on every size, and that suffix is what the factory reads to route to this family; no `task=` argument is needed. ## Train Fine-tuning starts from the published ImageNet backbone and rebuilds the final classifier layer to the target dataset's class count automatically. `imgsz` defaults to the size's own evaluation resolution unless set explicitly. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreEfficientNetV2b0-cls.pt") model.train(data="imagenette160", epochs=5) ``` **CLI** ```bash libreyolo train model=LibreEfficientNetV2b0-cls.pt data=imagenette160 epochs=5 ``` **Multi-GPU** ```bash libreyolo train model=LibreEfficientNetV2b0-cls.pt data=imagenette160 \ epochs=50 device=0,1 batch=-1 ``` Left alone, the trainer runs 100 epochs at `lr0=1e-3` with AdamW, a batch of 64 and early stopping after 50 epochs without improvement. `data` accepts a dataset root (`train/` and `val/`, one folder per class), a known short name such as `imagenette160`, or a `.zip` URL. `lora=True` is not supported here; passing it raises, since LoRA in LibreYOLO targets transformer components with `nn.Linear` layers and this family's MBConv blocks have none. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary of `metrics/` keys. For classification that is top-1 and top-5 accuracy over the validation split. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreEfficientNetV2b0-cls.pt") metrics = model.val(data="imagenette160") print(metrics["metrics/accuracy_top1"]) print(metrics["metrics/accuracy_top5"]) ``` **CLI** ```bash libreyolo val model=LibreEfficientNetV2b0-cls.pt data=imagenette160 ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | classify | yes | yes | yes | yes | yes | | | | yes | yes | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts and the extras a few of them add. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreEfficientNetV2b0-cls.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreEfficientNetV2b0-cls.pt format=onnx libreyolo export model=LibreEfficientNetV2b0-cls.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreEfficientNetV2b0-cls.onnx") result = model(SAMPLE_IMAGE) print(result.probs.top1) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreEfficientNetV2b0-cls.pt` | 224 | classify | apache-2.0 | | `LibreEfficientNetV2b1-cls.pt` | 240 | classify | apache-2.0 | | `LibreEfficientNetV2b2-cls.pt` | 260 | classify | apache-2.0 | | `LibreEfficientNetV2b3-cls.pt` | 300 | classify | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: EfficientNetV2, Google - Upstream license: Apache-2.0 - Upstream source: https://github.com/huggingface/pytorch-image-models - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours. The architecture is Google's design, whose reference implementation at google/automl is also Apache-2.0; LibreYOLO's implementation follows the block definitions, TensorFlow "SAME" padding and naming in timm, whose tf_efficientnetv2_b{0,1,2,3} ImageNet-1k weights (ported by Ross Wightman, no ImageNet-21k or extra data) are licensed Apache-2.0 and are what LibreYOLO ships. --- # EoMT A segmentation network built on a plain vision transformer with no dedicated pixel decoder: extra learned queries added to the encoder itself predict the masks. LibreYOLO supports it for semantic, instance and panoptic segmentation. Tasks: semantic, Instance segmentation, panoptic. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install EoMT needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. The task suffix in the filename (`-sem`, `-seg`, `-panoptic`) selects the task, and `LibreYOLO()` infers it from that filename so no `task=` argument is needed. **Semantic** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreEoMTl-sem.pt") result = model(SAMPLE_IMAGE, save=True) mask = result.semantic_mask print(mask.data.shape) # (H, W) class ids print(mask.classes) # sorted class ids present in the image ``` **Instance segmentation** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The -seg suffix in the filename selects the instance task, so no # task argument is needed here. model = LibreYOLO("LibreEoMTl-seg.pt") result = model(SAMPLE_IMAGE, save=True) print(result.boxes.xyxy) print(result.masks.data.shape) ``` **Panoptic** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreEoMTl-panoptic.pt") result = model(SAMPLE_IMAGE, save=True) pan = result.panoptic print(pan.data.shape) # (H, W) segment ids print(pan.segments_info) # [{"id": ..., "category_id": ...}, ...] ``` **CLI** ```bash libreyolo predict model=LibreEoMTl-sem.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` Semantic segmentation fills `result.semantic_mask`, a `(H, W)` array of class ids on `.data`. Instance segmentation fills `result.boxes` and `result.masks`, the same shape every other segmentation family returns. Panoptic segmentation fills `result.panoptic`: a `(H, W)` segment-id map on `.data`, plus `.segments_info`, a list of `{"id", "category_id"}` dicts, one per segment. `conf` filters query selection; `iou` has no effect on the semantic task, since it argmaxes per pixel with no NMS step. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Three encoder sizes, s/b/l, all DINOv2-backed. The semantic checkpoint is trained on ADE20K at 512 px; the instance and panoptic checkpoints are trained on COCO at 640 px, with a second instance checkpoint trained at 1280 px. Upstream ships DINOv2 instance-segmentation weights only at size l; s and b are published for semantic and panoptic only. DINOv3-backed EoMT variants exist upstream but are not shipped here, because they depend on gated non-commercial DINOv3 weights. LibreYOLO does not train EoMT: `train()` raises `NotImplementedError` for this family, which the [support tier](/docs/models) above marks as inference only. ## Validate `val()` dispatches by task. Semantic returns `metrics/mIoU` and `metrics/pixel_accuracy`. Instance segmentation returns the same mask and box mAP keys as other segmentation families. Panoptic returns Panoptic Quality as `metrics/PQ`, split into `metrics/SQ` (segmentation quality) and `metrics/RQ` (recognition quality), plus `metrics/PQ_things` and `metrics/PQ_stuff`. **Semantic** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreEoMTl-sem.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mIoU"]) print(metrics["metrics/pixel_accuracy"]) ``` **Instance segmentation** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreEoMTl-seg.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95(M)"]) # masks print(metrics["metrics/mAP50-95(B)"]) # boxes ``` **Panoptic** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreEoMTl-panoptic.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/PQ"]) print(metrics["metrics/SQ"], metrics["metrics/RQ"]) ``` **CLI** ```bash libreyolo val model=LibreEoMTl-sem.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | semantic | yes | yes | | yes | yes | | | | | | | | | Instance segmentation | | | | | | | | | | | | | | panoptic | | | | | | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. Only the semantic task exports today: instance and panoptic segmentation call `export()` and get `NotImplementedError`, because their query-mask output has no runtime export contract yet. An exported semantic artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreEoMTl-sem.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreEoMTl-sem.pt format=onnx libreyolo export model=LibreEoMTl-sem.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreEoMTl-sem.onnx") result = model(SAMPLE_IMAGE) print(result.semantic_mask.data.shape) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreEoMTl-sem.pt` | 512 | semantic | mit | | `LibreEoMTl-seg.pt` | 640 | Instance segmentation | mit | | `LibreEoMTl-seg-1280.pt` | | Instance segmentation | mit | | `LibreEoMTs-panoptic.pt` | | panoptic | mit | | `LibreEoMTb-panoptic.pt` | | panoptic | mit | | `LibreEoMTl-panoptic.pt` | | panoptic | mit | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: EoMT, TU Eindhoven, Mobile Perception Systems Lab - Upstream license: MIT - Upstream source: https://github.com/tue-mps/eomt - LibreYOLO code: MIT - Weights: MIT, republished at https://huggingface.co/LibreYOLO - Interpretation: MIT is a permissive license, so this code and these weights can be used in commercial and closed-source products. It asks only that you keep the copyright and license notice with any copy you redistribute. LibreYOLO ships only the DINOv2-backed EoMT checkpoints, sizes s/b/l; the DINOv3 EoMT variants are excluded because they depend on gated non-commercial DINOv3 weights. DINOv2 itself is Apache-2.0. The semantic checkpoint is trained on ADE20K and the instance and panoptic checkpoints on COCO; both are research datasets, and users remain responsible for dataset-license compliance when validating or fine-tuning against them. ## Citation ```bibtex @inproceedings{kerssies2025eomt, author = {Kerssies, Tommie and Cavagnero, Niccol\`{o} and Hermans, Alexander and Norouzi, Narges and Averta, Giuseppe and Leibe, Bastian and Dubbelman, Gijs and {de Geus}, Daan}, title = {{Your ViT is Secretly an Image Segmentation Model}}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, year = {2025}, } ``` Copied from https://github.com/tue-mps/eomt#bibtex-citation --- # Faster R-CNN Faster R-CNN detects objects with a region proposal network feeding a two-stage classifier, the architecture that made region proposals part of the same trained network instead of a separate step. LibreYOLO ports the torchvision implementation for detection. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install Faster R-CNN needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreFasterRCNNl.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreFasterRCNNl.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. `conf` and `iou` set the confidence and NMS thresholds; Faster R-CNN keeps its upstream NMS step, unlike a query-based detector. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Four sizes, each a different torchvision configuration rather than a scaled version of the same one: `n` is MobileNetV3-Large at a 320 px input, `s` is the same backbone at 800 px, `m` is ResNet-50 with a feature pyramid, and `l` is the v2 revision, with a deeper region proposal head and a four-convolution box head in place of `m`'s. `n` and `s` trade accuracy for a lighter backbone. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreFasterRCNNl.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreFasterRCNNl.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | | | | | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. Faster R-CNN exports to ONNX only, at batch size 1. The exported graph keeps the upstream resize step inside it, so LibreYOLO forces `dynamic=True` regardless of what is passed, to keep the graph valid for sources that are not square. An exported `.onnx` file loads back through `LibreYOLO()` on its file suffix and returns the same `Results`. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreFasterRCNNl.pt") model.export(format="onnx", imgsz=800) ``` **CLI** ```bash libreyolo export model=LibreFasterRCNNl.pt format=onnx imgsz=800 ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreFasterRCNNl.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreFasterRCNNn.pt` | 320 | Detection | bsd-3-clause | | `LibreFasterRCNNs.pt` | 800 | Detection | bsd-3-clause | | `LibreFasterRCNNm.pt` | 800 | Detection | bsd-3-clause | | `LibreFasterRCNNl.pt` | 800 | Detection | bsd-3-clause | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: Faster R-CNN, PyTorch - Upstream license: BSD-3-Clause - Upstream source: https://github.com/pytorch/vision - LibreYOLO code: BSD-3-Clause - Weights: BSD-3-Clause, republished at https://huggingface.co/LibreYOLO - Interpretation: BSD-3-Clause is a permissive license, so this code can be used in commercial and closed-source products with no obligation on your own application code. It asks only that you keep the copyright notice and disclaimer with any copy you redistribute, and it carries no patent grant. The four published checkpoints used for parity testing are not distributed in the LibreYOLO source tree: torchvision's own documentation notes that a pretrained model's terms may depend on its training data, so each Hugging Face mirror ships the BSD text on that implied basis and repeats the caveat rather than issuing an explicit checkpoint-specific grant. ## Citation ```bibtex @inproceedings{renNIPS15fasterrcnn, Author = {Shaoqing Ren and Kaiming He and Ross Girshick and Jian Sun}, Title = {Faster {R-CNN}: Towards Real-Time Object Detection with Region Proposal Networks}, Booktitle = {Advances in Neural Information Processing Systems ({NIPS})}, Year = {2015} } ``` Copied from https://github.com/rbgirshick/py-faster-rcnn#citing-faster-r-cnn --- # FCN A dense per-pixel classifier that replaces a detector's fully connected layers with convolutions, so it outputs a full-resolution class map instead of boxes. LibreYOLO ships it for semantic segmentation only. Tasks: semantic. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install FCN needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreFCNr50.pt") result = model(SAMPLE_IMAGE, save=True) mask = result.semantic_mask print(mask.data.shape) # (H, W) class ids print(mask.classes) # sorted class ids present in the image ``` **CLI** ```bash libreyolo predict model=LibreFCNr50.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` Semantic segmentation returns one class id per pixel, not boxes, so `result.semantic_mask` carries a `(H, W)` array on `.data` and the list of class ids present in the image on `.classes`. `conf`, `iou` and `max_det` are accepted for API parity but have no effect: the model assigns a class to every pixel by argmax, with no confidence threshold or NMS step. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Two ResNet depths, both at a fixed 520 px input. The library's inference graph is torchvision's dilated-ResNet FCN, not the original paper's VGG-based FCN-8s network with skip connections. LibreYOLO does not train FCN: `train()` raises `NotImplementedError` for this family, which the [support tier](/docs/models) above marks as inference only. The two published checkpoints are torchvision's own COCO-trained weights, converted for LibreYOLO's loader. ## Validate `val()` returns `metrics/mIoU` and `metrics/pixel_accuracy`, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreFCNr50.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mIoU"]) print(metrics["metrics/pixel_accuracy"]) ``` **CLI** ```bash libreyolo val model=LibreFCNr50.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | semantic | yes | yes | | yes | yes | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreFCNr50.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreFCNr50.pt format=onnx libreyolo export model=LibreFCNr50.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreFCNr50.onnx") result = model(SAMPLE_IMAGE) print(result.semantic_mask.data.shape) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreFCNr50.pt` | 520 | semantic | bsd-3-clause | | `LibreFCNr101.pt` | 520 | semantic | bsd-3-clause | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: FCN, PyTorch - Upstream license: BSD-3-Clause - Upstream source: https://github.com/pytorch/vision - LibreYOLO code: BSD-3-Clause - Weights: BSD-3-Clause, republished at https://huggingface.co/LibreYOLO - Interpretation: BSD-3-Clause is a permissive license, so this code and these weights can be used in commercial and closed-source products. It asks you to keep the copyright notice, license text and a non-endorsement clause with any copy you redistribute. LibreYOLO's inference graph is torchvision's dilated-ResNet FCN, not the original VGG-based FCN-8s skip-fusion network from the paper. The two published checkpoints are torchvision's official COCO-trained weights; their separate LibreYOLO Hugging Face mirrors carry BSD-3-Clause on an implied basis disclosed by torchvision rather than an explicit checkpoint-specific grant, and torchvision's own documentation notes that pretrained-model terms can depend on the training data, leaving that determination to the user. --- # FCOS FCOS detects objects per pixel instead of relying on a set of predefined anchor boxes, predicting a box and a centerness score at every location on the feature map. LibreYOLO ports the torchvision implementation for detection. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install FCOS needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreFCOSr50.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreFCOSr50.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. Calling the model with no threshold arguments applies FCOS's own published defaults, `conf=0.2`, `iou=0.6` and `max_det=100`; pass any of the three to override them. FCOS keeps a final NMS step over its per-pixel predictions. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants One size: ResNet-50 with a feature pyramid, the only variant this family recognizes. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreFCOSr50.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreFCOSr50.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | | | yes | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. FCOS exports to ONNX, TorchScript and OpenVINO. FCOS preserves the source aspect ratio before the graph runs, so LibreYOLO forces `dynamic=True` for the ONNX and OpenVINO paths regardless of what is passed, to keep the graph valid for padded input shapes. An exported `.onnx` file loads back through `LibreYOLO()` on its file suffix and returns the same `Results`. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreFCOSr50.pt") model.export(format="onnx", imgsz=800) model.export(format="torchscript", imgsz=800) ``` **CLI** ```bash libreyolo export model=LibreFCOSr50.pt format=onnx imgsz=800 ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreFCOSr50.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreFCOSr50.pt` | 800 | Detection | bsd-3-clause | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: FCOS, PyTorch - Upstream license: BSD-3-Clause - Upstream source: https://github.com/pytorch/vision - LibreYOLO code: BSD-3-Clause - Weights: BSD-3-Clause, republished at https://huggingface.co/LibreYOLO - Interpretation: BSD-3-Clause is a permissive license, so this code can be used in commercial and closed-source products with no obligation on your own application code. It asks only that you keep the copyright notice and disclaimer with any copy you redistribute, and it carries no patent grant. The published checkpoint used for parity testing is not distributed in the LibreYOLO source tree: torchvision's own documentation notes that a pretrained model's terms may depend on its training data, so the Hugging Face mirror ships the BSD text on that implied basis and repeats the caveat rather than issuing an explicit checkpoint-specific grant. ## Citation ```bibtex @inproceedings{tian2019fcos, title = {{FCOS}: Fully Convolutional One-Stage Object Detection}, author = {Tian, Zhi and Shen, Chunhua and Chen, Hao and He, Tong}, booktitle = {Proc. Int. Conf. Computer Vision (ICCV)}, year = {2019} } @article{tian2021fcos, title = {{FCOS}: A Simple and Strong Anchor-free Object Detector}, author = {Tian, Zhi and Shen, Chunhua and Chen, Hao and He, Tong}, booktitle = {IEEE T. Pattern Analysis and Machine Intelligence (TPAMI)}, year = {2021} } ``` Copied from https://github.com/tianzhi0549/FCOS#citations --- # FeyNobg A background-removal model from Feyn Inc. that deepens BiRefNet's architecture and retrains it. LibreYOLO ships inference and validation for FeyNobg's matte task. Tasks: matte. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install FeyNobg needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict The checkpoint downloads from the LibreYOLO organization on Hugging Face on first use and is cached locally, the same as any other family, though it is not yet listed in the Checkpoints table on this page. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreFeyNobgl-matte.pt") result = model(SAMPLE_IMAGE, save=True) matte = result.matte print(matte.array.shape, matte.array.dtype) ``` **CLI** ```bash libreyolo predict model=LibreFeyNobgl-matte.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **Cutout** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreFeyNobgl-matte.pt") result = model(SAMPLE_IMAGE) # RGBA (H, W, 4) uint8: source RGB plus the matte as an alpha channel. rgba = result.cutout() result.save("subject.png") ``` A matte result carries no boxes; `result.matte` is a dense `(H, W)` float32 array in `[0, 1]`, 1 fully foreground and 0 fully background. Unlike a binary mask, the soft matte keeps anti-aliased edge detail such as hair and fur. `result.cutout()` composites the source image with that alpha channel into an RGBA array, and `result.save(path)` (or `save=True` on the predict call) writes it straight to a transparent-background PNG. The model runs at a fixed native 1024x1024 canvas; a different resolution is not supported, because the Swin backbone's relative-position tables are tied to it, and a mismatch interpolates them badly rather than raising an error. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants One published size, `l`, a Swin-L tier backbone. FeyNobg takes BiRefNet's architecture and deepens its third Swin stage from 18 to 24 blocks before retraining, so the LibreYOLO port reuses BiRefNet's forward path, preprocessing and single-logit output contract; predict, validate and checkpoint handling behave the same as the `birefnet` family. ## Validate `val()` reports two metrics over a paired image/matte folder, both in `[0, 1]` and independent of resolution: MAE, the mean absolute error against the ground-truth alpha (lower is better), and S-measure (Fan et al., ICCV 2017), a structural similarity that credits preserving the subject's shape and holes, which pixel MAE alone misses (higher is better). Validation drives the model's own `predict`, so it uses the family's exact preprocessing. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreFeyNobgl-matte.pt") # A directory containing images/ and an auto-detected matte directory # (mattes/, matte/, gt/, masks/, mask/ or alpha/) also works in place # of a dataset YAML. metrics = model.val(data="my-matte-dataset/") print(metrics["metrics/MAE"]) print(metrics["metrics/Smeasure"]) ``` Validation is inference-only. The upstream `nobg` library ships Apache-2.0 training code; fine-tuning today means training there and converting the result with LibreYOLO's own conversion script, not calling `train()` on this family, which raises rather than running a partial trainer. ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: FeyNobg, Feyn Inc. - Upstream license: Apache-2.0 - Upstream source: https://github.com/feyninc/nobg - LibreYOLO code: MIT - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code. FeyNobg builds on BiRefNet's architecture (MIT) with a deepened backbone stage, and Feyn Inc. releases both the nobg library and the FeyNobg weights under Apache-2.0. LibreYOLO's checkpoint is a format conversion of Feyn's own published weights, with the learned parameters unchanged; fine-tuning is not wired into this library, so there is no LibreYOLO-trained variant to license separately. ## Citation ```bibtex @software{nobg, title={nobg: Open Source Background Removal Models for Image and Video Matting}, author={Hichri, Hafedh}, year={2026}, url={https://github.com/feyninc/nobg}, license={Apache-2.0}, } ``` Copied from https://github.com/feyninc/nobg#citation --- # Florence-2 Florence-2 is Microsoft's vision foundation model, prompted with a task token instead of run through a fixed detection head. LibreYOLO wraps it as an open-vocabulary object detector: supply the class list at predict time. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install Florence-2 belongs to LibreYOLO's VLM-as-detector tier, a separate product surface from the checkpoint-based families with its own factory. It needs the `vlm` extra. ```bash pip install "libreyolo[vlm]" ``` ## Predict Weights download from Hugging Face on first use and are cached locally. LibreYOLO downloads the florence-community re-upload of the checkpoint rather than the original `microsoft/Florence-2-*` repository; see Licensing for why. **Python** ```python from libreyolo import LibreVLM, SAMPLE_IMAGE model = LibreVLM("florence-2-base") model.set_classes(["car", "person", "traffic light"]) result = model.predict(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **Video** ```python from libreyolo import LibreVLM model = LibreVLM("florence-2-base") model.set_classes(["car", "person", "traffic light"]) # Any source the library accepts: file, folder, URL, webcam index, # RTSP stream, or a .streams list for result in model.predict("clip.mp4", stream=True, save=True): print(len(result.boxes)) ``` This family loads through the `LibreVLM()` factory, not `LibreYOLO()`: VLM families declare no checkpoint loader, so the file-suffix routing described on other model pages does not apply here. `set_classes()` sets the vocabulary Florence-2 is asked to find in the image; it is sticky, so it stays in effect across every later `predict()`/`track()` call until you set it again. The returned `Results` carries `boxes` in the same shape as any other family, but every detection carries the same placeholder confidence, so `conf` filtering is all-or-nothing rather than a ranking, and `iou` has no effect: Florence-2's wrapper builds the detection list directly from the parsed task-token output, with no deduplication step. `chat()` raises `NotImplementedError` here, because Florence-2 is driven by the `` task token rather than a chat template. LibreYOLO's CLI does not cover this tier: there is no `libreyolo predict model=...` form for it. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Two sizes: Florence-2-base and Florence-2-large, both at 768 px, loaded as `LibreVLM("florence-2-base")` or `LibreVLM("florence-2-large")`. LibreYOLO has not published a benchmark comparing accuracy between them. LibreYOLO does not train, validate or export Florence-2: `train()`, `val()` and `export()` all raise `NotImplementedError` for every family in this tier (see the support tier above). Fine-tune Florence-2 upstream and load the resulting weights if you need a custom vocabulary baked in; check `predict()` output by eye instead of a COCO-style validation pass, since every detection carries the same placeholder confidence. ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: Florence-2, Microsoft - Upstream license: MIT - Upstream source: https://huggingface.co/microsoft/Florence-2-large - LibreYOLO code: MIT - Weights: MIT, distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: MIT is a permissive license, so these weights can be used in commercial and closed-source products, provided the copyright notice and license text travel with any copy you redistribute. LibreYOLO downloads the florence-community re-upload of the checkpoint (florence-community/Florence-2-base and florence-community/Florence-2-large) rather than the original microsoft/Florence-2-* repositories, because those ship with custom remote code that no longer loads on current transformers releases. florence-community republishes the same weights through the native Florence2ForConditionalGeneration class, also under MIT, so nothing about the license changes. ## Citation ```bibtex @article{xiao2023florence, title={Florence-2: Advancing a unified representation for a variety of vision tasks}, author={Xiao, Bin and Wu, Haiping and Xu, Weijian and Dai, Xiyang and Hu, Houdong and Lu, Yumao and Zeng, Michael and Liu, Ce and Yuan, Lu}, journal={arXiv preprint arXiv:2311.06242}, year={2023} } ``` Copied from https://huggingface.co/microsoft/Florence-2-large#bibtex --- # FOMO FOMO is a grid-based point localizer: each cell of a low-resolution grid is classified as background or an object center, with no bounding-box regression. LibreYOLO supports it for the point task. Tasks: point. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install FOMO needs no extra beyond the base package. ```bash pip install libreyolo ``` ## Predict Unlike every other family on this site, LibreFOMO weights are not auto-downloaded: `LibreYOLO("LibreFOMOs-point.pt")` looks for that file on disk and raises a `ValueError` naming it rather than fetching it from Hugging Face. Download a checkpoint from the [LibreYOLO org](https://huggingface.co/LibreYOLO) first and load it by local path, or train your own (see Train below). **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # LibreFOMO weights are not auto-downloaded (see Checkpoints below). # Point this at a checkpoint you already downloaded locally. model = LibreYOLO("./LibreFOMOs-point.pt") result = model(SAMPLE_IMAGE, save=True) for point in result.points: print(point.cls, point.conf, point.xy) ``` **CLI** ```bash libreyolo predict model=./LibreFOMOs-point.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` The result carries a `points` payload instead of `boxes`: each row is `x, y, class, confidence`, available as `result.points.data`, or through the `.xy`, `.xyn`, `.cls` and `.conf` accessors. There is no `iou` threshold to set, because there are no boxes to suppress; `predict(..., nms_radius=1)` controls how many grid cells apart two detections must be to both survive, and the filename must carry FOMO's `-point` task suffix for the loader to recognize it. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Three sizes, `s`, `m` and `l`, use progressively wider MobileNetV2-style backbones at correspondingly larger, fixed input resolutions, each behind a single 1x1 classification head. This family carries no benchmark table here; checkpoint file size in the table below is the clearest per-size signal currently published. ## Train **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("./LibreFOMOs-point.pt") model.train( data="my-dataset.yaml", epochs=40, batch=32, lr0=3e-4, ) ``` **CLI** ```bash # imgsz must be passed: the CLI defaults it to 640, and the s # checkpoint accepts only its native 96. libreyolo train model=./LibreFOMOs-point.pt data=my-dataset.yaml imgsz=96 epochs=40 batch=32 lr0=3e-4 ``` `imgsz` is not a free choice: it defaults to the loaded checkpoint's native resolution, and passing a different value raises `ValueError` naming the size it expects. Those sizes are 96 for `s`, 192 for `m` and 224 for `l`. The CLI defaults `imgsz` to 640, so a `libreyolo train` command has to set it explicitly to match the checkpoint. Left alone otherwise, the trainer runs 40 epochs at batch 32 with Adam at `lr0=3e-4`, no weight decay, and a foreground class weighted 100x over background in the per-cell cross-entropy loss, since almost every grid cell is background in a typical scene. EMA and mixed precision are both off by default, and none of the geometric or color augmentations used elsewhere in LibreYOLO are applied: mosaic, mixup, HSV jitter, flip, rotation, translation and shear are all zero. This is the path the published LibreFOMO checkpoints were trained with, from scratch on COCO. See [training](/docs/train) for datasets and loggers. ## Validate `val()` dispatches to a grid-level validator built for this family. Alongside the point-matching `metrics/precision`, `metrics/recall` and `metrics/mAP@` keys shared with other point tasks, it sweeps confidence thresholds and `nms_radius` values and publishes the best-F1 combination under `metrics/grid_F1`, `metrics/grid_precision`, `metrics/grid_recall` and `metrics/grid_mean_distance`, plus the threshold and radius that produced it under `decode/threshold` and `decode/nms_radius`. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("./LibreFOMOs-point.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/grid_F1"]) print(metrics["metrics/grid_precision"], metrics["metrics/grid_recall"]) ``` **CLI** ```bash libreyolo val model=./LibreFOMOs-point.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | point | yes | yes | yes | yes | yes | | | | yes | | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Running the graph in a bare runtime, with no LibreYOLO installed, is also supported, but then preprocessing and postprocessing are yours to write. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("./LibreFOMOs-point.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=./LibreFOMOs-point.pt format=onnx ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("./LibreFOMOs-point.onnx") result = model(SAMPLE_IMAGE) print(result.points.xy) ``` ## Checkpoints Every published weight file for this family. None of them download automatically: fetch the file you want from the linked Hugging Face page and pass its local path to `LibreYOLO()`. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreFOMOs-point.pt` | | point | mit | | `LibreFOMOm-point.pt` | | point | mit | | `LibreFOMOl-point.pt` | | point | mit | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: FOMO (Faster Objects, More Objects), Edge Impulse - Upstream license: MIT - Upstream source: https://docs.edgeimpulse.com/docs/edge-impulse-studio/learning-blocks/object-detection/fomo-object-detection-for-constrained-devices - LibreYOLO code: MIT - Weights: MIT, republished at https://huggingface.co/LibreYOLO - Interpretation: FOMO is a technique Edge Impulse introduced through a blog post and its product documentation, not a code release, so there is no upstream repository or license to inherit. LibreYOLO's architecture is an original MobileNetV2-style reimplementation of the published description, and the LibreFOMO checkpoints are trained from scratch on COCO, so both the code and these weights are MIT, LibreYOLO's own. They are also not auto-downloaded: LibreYOLO("LibreFOMOs-point.pt") looks for that file locally and raises rather than fetching it, so get the checkpoint from the Hugging Face repository first. The name FOMO and the technique it describes remain Edge Impulse's. There is no upstream code repository for FOMO to link: Edge Impulse describes the technique through a blog post and its product documentation, but has not released FOMO training or inference code. The architecture and training here are LibreYOLO's own implementation of that published description, and the published LibreFOMO checkpoints are trained from scratch on COCO, so both the code and these weights are MIT, LibreYOLO's own. The name FOMO and the technique it describes remain Edge Impulse's. --- # Grounding DINO Grounding DINO is an open-set object detector, developed by IDEA Research, that scores an image against a free-text prompt instead of a fixed class list. LibreYOLO wraps it as a predict-only family in its open-vocabulary detector tier. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install Grounding DINO loads through LibreYOLO's open-vocabulary detector tier, which needs the `openvocab` extra: ```bash pip install "libreyolo[openvocab]" ``` That extra pulls in `transformers` and `timm`, the Hugging Face libraries this tier calls into. ## Predict Grounding DINO is not a checkpoint LibreYOLO loads through `LibreYOLO()`. It loads through the sibling `LibreOpenVocab` factory, which downloads a Hugging Face snapshot on first use and caches it under `weights/`. **Python** ```python from libreyolo import LibreOpenVocab, SAMPLE_IMAGE model = LibreOpenVocab("grounding-dino-t") model.set_classes(["person", "dog", "skateboard"]) result = model.predict(SAMPLE_IMAGE, conf=0.25) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **Text threshold** ```python from libreyolo import LibreOpenVocab, SAMPLE_IMAGE model = LibreOpenVocab("grounding-dino-b") model.set_classes(["remote control", "school bus"]) # conf filters by box score, text_threshold by the decoded phrase's # token score. Both default to 0.25 when left unset. result = model.predict(SAMPLE_IMAGE, conf=0.25, text_threshold=0.3) print(result.names) ``` `set_classes()` sets a sticky text vocabulary: call it again to replace the list, or skip it to keep the default COCO-80 labels. Grounding DINO decodes free-form phrases from its own text output and maps them back to that vocabulary itself, an exact normalized match wins, a whole-token match is accepted, and an ambiguous or unmatched phrase is dropped rather than guessed at, so `school bus` never gets mapped to `bus` or `school` alone. A vocabulary long enough to exceed the text encoder's token limit is split into several prompts, run as separate forward passes, and merged back into one set of detections capped by `max_det`. `iou` is accepted for API compatibility but warns and does nothing, since nothing here runs non-maximum suppression. `imgsz` and `augment=True` are rejected outright: the `transformers` processor owns resizing, and test-time augmentation is out of scope for this tier. `predict()` on a single image returns one `Results`, not a list; pass a directory, a list of images, or `stream=True` for a video source to get several. There is no CLI path for this family, `libreyolo predict` only loads `.pt` checkpoints through `LibreYOLO()`, so `LibreOpenVocab` families run from Python. See [prediction](/docs/predict) for source types and streaming. ## Variants Two checkpoints, `t` and `b`. `t` is this tier's default size when none is given. Both mirror the official IDEA Research release through `transformers`' `GroundingDinoForObjectDetection`, downloaded once into a LibreYOLO-hosted Hugging Face snapshot that preserves the upstream files. No accuracy or latency numbers are published for this family yet. Training, dataset validation and export are all out of scope for this tier: `train()`, `val()` and `export()` all raise `NotImplementedError` unconditionally. This is a predict-only wrapper around a published checkpoint. ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreGroundingDINOt.pt` | 800 | Detection | apache-2.0 | | `LibreGroundingDINOb.pt` | 800 | Detection | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: Grounding DINO, IDEA Research - Upstream license: Apache-2.0 - Upstream source: https://github.com/IDEA-Research/GroundingDINO - LibreYOLO code: MIT - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so this checkpoint can be used in commercial and closed-source products. It asks you to keep the license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. LibreYOLO vendors no Grounding DINO model source of its own: LibreGroundingDINO calls the Apache-2.0 `transformers` implementation, `GroundingDinoForObjectDetection`, directly, and downloads the official checkpoint into a LibreYOLO-hosted mirror repository that preserves the upstream snapshot files. ## Citation ```bibtex @article{liu2023grounding, title={Grounding dino: Marrying dino with grounded pre-training for open-set object detection}, author={Liu, Shilong and Zeng, Zhaoyang and Ren, Tianhe and Li, Feng and Zhang, Hao and Yang, Jie and Li, Chunyuan and Yang, Jianwei and Su, Hang and Zhu, Jun and others}, journal={arXiv preprint arXiv:2303.05499}, year={2023} } ``` Copied from https://github.com/IDEA-Research/GroundingDINO#black_nib-citation --- # HRNet HRNet is a convolutional network that keeps a high-resolution feature stream through repeated multi-scale fusion, instead of recovering resolution after downsampling. LibreYOLO wraps the official top-down pose variant for inference and validation. Tasks: Pose. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install HRNet needs no extra beyond the base package. ```bash pip install libreyolo ``` Its default person detector, a lightweight LibreYOLO9t checkpoint, downloads automatically the first time HRNet pairs with it. ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # No person source given: HRNet pairs itself with a lightweight # LibreYOLO9t detector automatically and logs that choice once. model = LibreYOLO("LibreHRNetw32-pose.pt") result = model(SAMPLE_IMAGE, save=True) print(result.keypoints.xy) print(result.boxes.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreHRNetw32-pose.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **Person source** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreHRNetw32-pose.pt") # Skip detection entirely: treat the whole image as one person. result = model(SAMPLE_IMAGE, cropped=True) # Or hand HRNet boxes from a detector you already ran. result = model(SAMPLE_IMAGE, person_boxes=[[34, 12, 220, 400]]) # Or pair it with a specific LibreYOLO detector instead of the # LibreYOLO9t default. result = model(SAMPLE_IMAGE, person_detector="rfdetr") ``` HRNet is a top-down pose estimator: it needs a person box before the pose head can run, so every call resolves one. Left alone, it pairs itself with a LibreYOLO9t detector the first time and logs that choice. `cropped=True` skips detection and treats the whole image as one person; `person_boxes` accepts boxes from a detector you already ran; `person_detector` accepts `"auto"`, `"rfdetr"`, any LibreYOLO detection model, or a plain callable. `flip_test=True` runs the model on the horizontally flipped crop as well and averages the two heatmaps, HRNet's own test-time augmentation; the generic `augment=True` is not defined here. Multi-image sources run sequentially: HRNet's detector and variable per-image person count do not support stacked prediction. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Two sizes, `w32` and `w48`, both predicting the standard COCO-17 keypoint set from a fixed-resolution person crop; `w48` is the wider of the two backbones. The upstream model zoo reports pose accuracy for each size with its own person detector, its own flip-testing setup, and the official COCO evaluation protocol. LibreYOLO's default pairing uses a different detector, so a validation run here measures that combination, not the upstream one; matching the upstream figures needs the same person boxes, detector scores, and flip setting the original evaluation used. ## Validate `val()` runs COCO-style keypoint OKS-AP and accepts a YOLO-pose `data.yaml` or a COCO keypoints JSON plus an images directory. The metrics backend is faster-coco-eval by default, with `pycocotools` used automatically when faster-coco-eval is not installed; `faster_coco_eval=False` forces the `pycocotools` path. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreHRNetw32-pose.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/keypoints_mAP50-95"]) print(metrics["metrics/keypoints_mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreHRNetw32-pose.pt data=my-dataset.yaml ``` Validation drives HRNet's own `predict()` internally, so it uses whatever person detector the model was built or called with. Construct the model with an explicit `person_detector=` to keep that source fixed across runs, rather than letting each call re-resolve the default. ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Pose | yes | yes | | yes | yes | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. HRNet's export contract covers ONNX, TorchScript, OpenVINO and TensorRT only; any other format raises before the trace starts. Every export is the fixed-canvas heatmap head alone, batch-one FP32, taking a person crop and returning raw heatmaps: the affine crop geometry ahead of it and the heatmap decoding, flip restoration and OKS suppression behind it stay in Python, so a full image-in, keypoints-out pipeline still needs LibreYOLO on the other end. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreHRNetw32-pose.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreHRNetw32-pose.pt format=onnx ``` **Use the exported file** ```python import numpy as np import onnxruntime as ort # The exported graph is the fixed-canvas heatmap head alone: it takes # a batch of already-cropped, already-normalized person crops and # returns raw heatmaps. Person detection, crop geometry, heatmap # decoding and OKS suppression are not part of this graph; running it # outside LibreYOLO means reimplementing that decode step yourself. session = ort.InferenceSession("LibreHRNetw32-pose.onnx") name = session.get_inputs()[0].name heatmaps = session.run( None, {name: np.zeros((1, 3, 256, 192), dtype=np.float32)} )[0] ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreHRNetw32-pose.pt` | | Pose | mit | | `LibreHRNetw48-pose.pt` | | Pose | mit | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: HRNet, Microsoft - Upstream license: MIT - Upstream source: https://github.com/leoxiaobin/deep-high-resolution-net.pytorch - LibreYOLO code: MIT - Weights: MIT, republished at https://huggingface.co/LibreYOLO - Interpretation: MIT permits commercial and non-commercial use, modification and redistribution of both the code and the two published checkpoints, with the copyright notice retained. The official repository does not attach a separate license to its model-zoo checkpoints; LibreYOLO's redistribution basis is the MIT license the releasing project implies, the same basis the upstream repository's own files state. ## Citation ```bibtex @inproceedings{sun2019deep, title={Deep High-Resolution Representation Learning for Human Pose Estimation}, author={Sun, Ke and Xiao, Bin and Liu, Dong and Wang, Jingdong}, booktitle={CVPR}, year={2019} } ``` Copied from https://github.com/leoxiaobin/deep-high-resolution-net.pytorch#citation --- # InternVL3 InternVL3 is a native multimodal large language model released by OpenGVLab that jointly learns vision and language in a single pre-training stage. LibreYOLO wraps it as an open-vocabulary object detector: any list of text labels becomes the class set, with no fixed head and no fine-tuning required. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install InternVL3 needs the `vlm` extra, which pulls in `transformers` for the chat-template backbone. ```bash pip install "libreyolo[vlm]" ``` ## Predict `LibreInternVL3` is a Python class, not a `.pt` checkpoint: it is not loaded through the `LibreYOLO()` factory, and the `libreyolo` CLI does not resolve it. The `LibreVLM(...)` factory (`from libreyolo import LibreVLM`) also reaches this family by alias, e.g. `LibreVLM("internvl3-2b")`; the class used below is what it constructs. Weights come from OpenGVLab's own `-hf` Hugging Face repositories, not a LibreYOLO mirror; the first call downloads and caches them locally, and logs a one-time license notice for the gated Qwen weights before it does. **Python** ```python from libreyolo import LibreInternVL3, SAMPLE_IMAGE model = LibreInternVL3(size="2b") # Open vocabulary: any words work, not a fixed class head. Sticky # across every later predict()/track() call until set again. model.set_classes(["person", "bicycle", "dog"]) result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **Raw chat** ```python from libreyolo import LibreInternVL3, SAMPLE_IMAGE model = LibreInternVL3(size="2b") # The escape hatch beneath the detection convenience: free-form # questions, counting, or any prompt the boxes wrapper doesn't cover. text = model.chat(SAMPLE_IMAGE, "Describe the scene in one sentence.") print(text) ``` `result.boxes` carries the parsed detections like any other family. Confidence is a placeholder: InternVL3 emits no per-box score, so every detection gets the same constant confidence, and `conf=` only drops rows below that constant, it does not rank them. `iou` discards near-duplicate boxes of the same class above the given overlap, a side effect of greedy decoding repeating an object; it is not a class-wise NMS pass. Skip `set_classes()` and the vocabulary defaults to the COCO-80 names. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Three sizes: 1b, 2b and 8b, all OpenGVLab's native `-hf` checkpoints (a Qwen LLM backbone, not the two-tower architecture the original InternVL paper describes). LibreYOLO's benchmark harness has not measured this family, so there are no published accuracy numbers to compare them by; pick a size against your own compute budget. LibreYOLO exposes this family for prediction only. `train()`, `val()` and `export()` all raise `NotImplementedError`: fine-tune upstream and load the result instead, dataset validation is skipped because a placeholder confidence would make COCO mAP misleading, and export is out of scope for a generative model with no state dict to trace. ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: InternVL3, Shanghai AI Laboratory (OpenGVLab) - Upstream license: MIT (code); Qwen License (weights) - Upstream source: https://github.com/OpenGVLab/InternVL - LibreYOLO code: MIT - Weights: MIT (code); Qwen License (weights), distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: InternVL3's own code is MIT, permissive and usable in commercial and closed-source products. The `-hf` checkpoints this family loads carry a Qwen LLM backbone and are licensed separately, under Alibaba Cloud's Qwen License: free to use, modify and redistribute with a "Built with Qwen" or "Improved using Qwen" attribution requirement, and a 100 million monthly-active-user ceiling on commercial use above which Alibaba's own authorization is required. LibreYOLO does not host or redistribute these weights: LibreInternVL3 downloads the matching size directly from OpenGVLab/InternVL3--hf on Hugging Face the first time it runs, and logs a one-time notice for the Qwen License before that download. InternVL3's own code is MIT, permissive and usable in commercial and closed-source products. The `-hf` checkpoints this family loads carry a Qwen LLM backbone and are licensed separately, under Alibaba Cloud's Qwen License: free to use, modify and redistribute with a "Built with Qwen" or "Improved using Qwen" attribution requirement, and a 100 million monthly-active-user ceiling on commercial use above which Alibaba's own authorization is required. LibreYOLO does not host or redistribute these weights: `LibreInternVL3` downloads the matching size directly from `OpenGVLab/InternVL3--hf` on Hugging Face the first time it runs, and logs a one-time notice for the Qwen License before that download. ## Citation ```bibtex @article{zhu2025internvl3, title={Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models}, author={Zhu, Jinguo and Wang, Weiyun and Chen, Zhe and Liu, Zhaoyang and Ye, Shenglong and Gu, Lixin and Tian, Hao and Duan, Yuchen and Su, Weijie and Shao, Jie and others}, journal={arXiv preprint arXiv:2504.10479}, year={2025} } ``` Copied from https://github.com/OpenGVLab/InternVL#citation --- # Kosmos-2 Kosmos-2 is Microsoft's grounding model: it captions an image, then locates each noun phrase in that caption with a box. LibreYOLO wraps it as an open-vocabulary object detector: supply the class list at predict time. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install Kosmos-2 belongs to LibreYOLO's VLM-as-detector tier, a separate product surface from the checkpoint-based families with its own factory. It needs the `vlm` extra. ```bash pip install "libreyolo[vlm]" ``` ## Predict Weights download from Hugging Face on first use and are cached locally. LibreYOLO loads Microsoft's own `microsoft/kosmos-2-patch14-224` repository directly; unlike Florence-2, no community re-upload is needed here. **Python** ```python from libreyolo import LibreVLM, SAMPLE_IMAGE model = LibreVLM("kosmos-2") model.set_classes(["boat", "person"]) result = model.predict(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **Video** ```python from libreyolo import LibreVLM model = LibreVLM("kosmos-2") model.set_classes(["boat", "person"]) # Any source the library accepts: file, folder, URL, webcam index, # RTSP stream, or a .streams list for result in model.predict("clip.mp4", stream=True, save=True): print(len(result.boxes)) ``` This family loads through the `LibreVLM()` factory, not `LibreYOLO()`: VLM families declare no checkpoint loader, so the file-suffix routing described on other model pages does not apply here. `set_classes()` sets the vocabulary Kosmos-2 is asked to find; it is sticky, so it stays in effect across every later `predict()`/`track()` call until you set it again. Kosmos-2 grounds noun phrases rather than matching a label exactly, so LibreYOLO's wrapper accepts a partial match: a class named `"boat"` also matches a generated phrase like "the boats". Every detection carries the same placeholder confidence, so `conf` filtering is all-or-nothing rather than a ranking, and `iou` has no effect here, since the wrapper builds the detection list directly from the grounded entities with no deduplication step. `chat()` raises `NotImplementedError`, because Kosmos-2 is driven by a `` prompt rather than a chat template. LibreYOLO's CLI does not cover this tier: there is no `libreyolo predict model=...` form for it. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants One size: `kosmos-2-patch14-224`, at 224 px, loaded as `LibreVLM("kosmos-2")`. It is a 2023-era model, and LibreYOLO's own wrapper notes its grounding is coarser than the newer detectors in this tier. LibreYOLO does not train, validate or export Kosmos-2: `train()`, `val()` and `export()` all raise `NotImplementedError` for every family in this tier (see the support tier above). Fine-tune Kosmos-2 upstream and load the resulting weights if you need a custom vocabulary baked in; check `predict()` output by eye instead of a COCO-style validation pass, since every detection carries the same placeholder confidence. ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: Kosmos-2, Microsoft - Upstream license: MIT - Upstream source: https://github.com/microsoft/unilm/tree/master/kosmos-2 - LibreYOLO code: MIT - Weights: MIT, distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: MIT is a permissive license, so these weights can be used in commercial and closed-source products, provided the copyright notice and license text travel with any copy you redistribute. LibreYOLO downloads Microsoft's own microsoft/kosmos-2-patch14-224 repository directly; unlike Florence-2, no community re-upload is needed for it to load on current transformers. ## Citation ```bibtex @article{kosmos-2, title={Kosmos-2: Grounding Multimodal Large Language Models to the World}, author={Zhiliang Peng and Wenhui Wang and Li Dong and Yaru Hao and Shaohan Huang and Shuming Ma and Furu Wei}, journal={ArXiv}, year={2023}, volume={abs/2306} } ``` Copied from https://github.com/microsoft/unilm/blob/master/kosmos-2/README.md#citation --- # L2CS-Net L2CS-Net is a two-stage gaze estimator: a face detector locates faces, and a ResNet trunk with two angle-bin classification heads predicts pitch and yaw per face. LibreYOLO wraps it for inference only. Tasks: gaze. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install L2CS-Net needs no extra to construct, predict on, or export a model you already have a checkpoint for. ```bash pip install libreyolo ``` The one checkpoint LibreYOLO can fetch automatically, a Gaze360-trained ResNet-50, downloads over `gdown` rather than a plain HTTP mirror, because it lives on the author's Google Drive rather than the LibreYOLO org. That path needs the `gaze` extra: ```bash pip install "libreyolo[gaze]" ``` Without it, LibreYOLO prints manual download instructions instead of failing silently. ## Predict **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # No face_detector given: falls back to OpenCV's bundled face # detector (Haar on OpenCV 4, YuNet on OpenCV 5), so this runs with # no extra download beyond the L2CS checkpoint itself. model = LibreYOLO("LibreL2CSr50.pt") result = model(SAMPLE_IMAGE) print(result.gaze.pitch, result.gaze.yaw) ``` **CLI** ```bash libreyolo predict model=LibreL2CSr50.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **Face source** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreL2CSr50.pt") # Hand L2CS boxes from a detector you already ran. result = model(SAMPLE_IMAGE, face_boxes=[[34, 12, 90, 80]]) # Or name a specific bundled face detector. result = model(SAMPLE_IMAGE, face_detector="yunet") ``` L2CS-Net is a two-stage estimator: a face detector runs first, and the gaze head reads pitch and yaw from each face crop it returns. Left alone, prediction falls back to OpenCV's bundled detector, so a bare call works with no additional download once the L2CS checkpoint itself is in hand. `face_boxes` accepts boxes from a detector you already ran; `face_detector` accepts `"auto"`, `"haar"`, `"yunet"`, a LibreYOLO detection model, or a plain callable. `result.gaze` carries pitch and yaw in radians, aligned row by row with `result.boxes`, the detected face boxes. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Five backbone depths share one input resolution and take the same arguments. Gaze360, the dataset behind the only published checkpoint, trained a ResNet-50; the other four depths are supported architecturally but have no published weights to load. ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | gaze | yes | yes | yes | yes | yes | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreL2CSr50.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreL2CSr50.pt format=onnx ``` **Use the exported file** ```python import numpy as np import onnxruntime as ort # The exported graph is the ResNet trunk and the two angle-bin heads # alone: it takes a preprocessed 448x448 face crop and returns raw # (yaw_logits, pitch_logits), not decoded angles. The softmax, # bin-expectation and degree conversion stay in Python; see # libreyolo.models.l2cs.utils.bin_logits_to_angles. session = ort.InferenceSession("LibreL2CSr50.onnx") name = session.get_inputs()[0].name yaw_logits, pitch_logits = session.run( None, {name: np.zeros((1, 3, 448, 448), dtype=np.float32)} ) ``` ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: L2CS-Net, Ahmed A. Abdelrahman et al. - Upstream license: Gaze360 dataset terms: research and non-commercial use only, no redistribution - Upstream source: https://github.com/Ahmednull/L2CS-Net - LibreYOLO code: MIT - Weights: Gaze360 dataset terms: research and non-commercial use only, no redistribution, distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: The L2CS-Net source is MIT and may be used, modified and redistributed, including commercially. The one checkpoint LibreYOLO can fetch automatically, a ResNet-50 trained on Gaze360, is not covered by that grant: the Gaze360 dataset license restricts use of models trained on it to research and non-commercial purposes and forbids redistribution. LibreYOLO does not mirror or host that checkpoint. It downloads it, on request, directly from the author's own distribution and prints the Gaze360 terms once before the transfer starts. A model trained on data you hold the rights to, using this MIT architecture, carries none of those restrictions. LibreYOLO does not host or mirror any L2CS checkpoint: nothing for this family exists in the LibreYOLO Hugging Face org, unlike most other families on this site. The one checkpoint the library can fetch automatically comes straight from the author's own Google Drive distribution, gated behind the Gaze360 license notice printed before the transfer starts, and is not the "republished at huggingface.co/LibreYOLO" copy the summary above implies. --- # LFM2-VL LFM2-VL is a compact, on-device vision-language model released by Liquid AI. LibreYOLO wraps it as an open-vocabulary object detector: any list of text labels becomes the class set, with no fixed head and no fine-tuning required. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install LFM2-VL needs the `vlm` extra, which pulls in `transformers` for the chat-template backbone. ```bash pip install "libreyolo[vlm]" ``` ## Predict `LibreLFM2VL` is a Python class, not a `.pt` checkpoint: it is not loaded through the `LibreYOLO()` factory, and the `libreyolo` CLI does not resolve it. The `LibreVLM(...)` factory (`from libreyolo import LibreVLM`) also reaches this family by alias, e.g. `LibreVLM("lfm2-vl-450m")`; the class used below is what it constructs. Weights come from Liquid AI's own Hugging Face repository, not a LibreYOLO mirror; the first call downloads and caches them locally, and logs a one-time license notice before it does. **Python** ```python from libreyolo import LibreLFM2VL, SAMPLE_IMAGE model = LibreLFM2VL(size="450m") # Open vocabulary: any words work, not a fixed class head. Sticky # across every later predict()/track() call until set again. model.set_classes(["person", "bicycle", "dog"]) result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **Raw chat** ```python from libreyolo import LibreLFM2VL, SAMPLE_IMAGE model = LibreLFM2VL(size="450m") # The escape hatch beneath the detection convenience: free-form # questions, counting, or any prompt the boxes wrapper doesn't cover. text = model.chat(SAMPLE_IMAGE, "Describe the scene in one sentence.") print(text) ``` `result.boxes` carries the parsed detections like any other family. Confidence is a placeholder: LFM2-VL emits no per-box score, so every detection gets the same constant confidence, and `conf=` only drops rows below that constant, it does not rank them. `iou` discards near-duplicate boxes of the same class above the given overlap, a side effect of greedy decoding repeating an object; it is not a class-wise NMS pass. Skip `set_classes()` and the vocabulary defaults to the COCO-80 names. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Two sizes: 450m and 1.6b, both from Liquid AI's LFM2.5-VL release, built for on-device deployment. LibreYOLO's benchmark harness has not measured this family, so there are no published accuracy numbers to compare them by; pick a size against your own compute budget. LibreYOLO exposes this family for prediction only. `train()`, `val()` and `export()` all raise `NotImplementedError`: fine-tune upstream and load the result instead, dataset validation is skipped because a placeholder confidence would make COCO mAP misleading, and export is out of scope for a generative model with no state dict to trace. ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: LFM2-VL, Liquid AI - Upstream license: LFM Open License v1.0 - Upstream source: https://huggingface.co/LiquidAI/LFM2.5-VL-450M - LibreYOLO code: MIT - Weights: LFM Open License v1.0, distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: The LFM Open License v1.0 permits Commercial Use, reproduction and modification, but only below a $10 million annual revenue threshold; a Legal Entity at or above that threshold is not licensed under this agreement at all for Commercial Use, and must contact Liquid AI directly. Qualified non-profit organizations are exempt from the threshold for non-commercial or research use. Redistribution must keep the license text, and any modified file must carry a notice saying so. LibreYOLO ships no LiquidAI source code, since the model loads through the Apache-2.0 transformers library, and does not host or redistribute the weights: LibreLFM2VL downloads the matching size directly from Liquid AI's own Hugging Face repository the first time it runs, and logs a one-time notice before that download. The LFM Open License v1.0 permits commercial use, reproduction and modification, but only below a $10 million annual revenue threshold; a legal entity at or above that threshold is not licensed under this agreement at all for commercial use, and must contact Liquid AI directly. Qualified non-profit organizations are exempt from the threshold for non-commercial or research use. LibreYOLO ships no LiquidAI source code, since the model loads through the Apache-2.0 `transformers` library, and does not host or redistribute the weights: `LibreLFM2VL` downloads the matching size directly from Liquid AI's own Hugging Face repository the first time it runs, and logs a one-time notice before that download. ## Citation ```bibtex @article{liquidai2025lfm2, title={LFM2 Technical Report}, author={Liquid AI}, journal={arXiv preprint arXiv:2511.23404}, year={2025} } ``` Copied from https://huggingface.co/LiquidAI/LFM2.5-VL-450M#citation --- # LibreFaceRec LibreFaceRec is LibreYOLO's face-embedding task: a face detector locates and aligns faces, and a recognition head produces an L2-normalized identity embedding for verification or search. Tasks: embed. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install LibreFaceRec's recognition head runs through `onnxruntime`, which is not part of the base install. ```bash pip install "libreyolo[onnx]" ``` ## Predict **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # librefacerec-* names route to this family regardless of file # suffix and download from the LibreYOLO Hugging Face org on first # use, along with the default face detector. model = LibreYOLO("librefacerec-l.onnx") result = model(SAMPLE_IMAGE) print(result.embeddings.data.shape) # (N, D), L2-normalized ``` **CLI** ```bash libreyolo predict model=librefacerec-l.onnx source=face.jpg ``` **Verify** ```python from libreyolo import LibreYOLO model = LibreYOLO("librefacerec-l.onnx") # Compares the most prominent face in each image via cosine # similarity of their L2-normalized embeddings. result = model.verify("person_a.jpg", "person_b.jpg", threshold=0.4) print(result["similarity"], result["same_person"]) ``` **Gallery search** ```python from libreyolo import LibreYOLO model = LibreYOLO("librefacerec-l.onnx") query = model("query.jpg").embeddings # this image's faces gallery = model.embed(["a.jpg", "b.jpg", "c.jpg"]) # (N_total, D) # (query_faces, N_total) cosine similarities. scores = query.similarity(gallery) ``` Detection and recognition are two separate ONNX graphs behind one call: a face detector locates and aligns each face to a canonical crop, and the recognition head returns an L2-normalized embedding per face. Left alone, `predict()` downloads and pairs the bundled default detector automatically. `face_detector` accepts a callable, a LibreYOLO detection model, or a `FaceDetector` instance; `face_boxes` bypasses detection entirely with boxes you already have. `result.embeddings` holds one row per detected face, aligned with `result.boxes`; its `.similarity()` method computes cosine similarity against another embedding or a whole gallery in one call. For comparing two images directly rather than two already-computed embeddings, `model.verify(image_a, image_b)` runs detection and embedding on both and compares their most confident face. Any other ArcFace-convention ONNX recognition model (aligned crop in, `(N, D)` embeddings out) can be substituted by passing its file path instead of a `librefacerec-*` name. See [prediction](/docs/predict) for sources, streaming and result handling. ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | embed | | | | | | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. LibreFaceRec already wraps a pre-exported ONNX graph; re-exporting it to another format is not implemented. ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: AuraFace-v1, fal.ai - Upstream license: Apache-2.0 - Upstream source: https://huggingface.co/fal/AuraFace-v1 - LibreYOLO code: MIT - Weights: Apache-2.0, distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: Apache-2.0 covers the embedding weights (AuraFace-v1's glintr100 recognition head), so they may be used, modified and redistributed, including commercially, provided the license text and attribution notices travel with any copy. LibreYOLO's default face detector is a separate artifact under a separate license, MIT (OpenCV Zoo's YuNet, copyright Shiqi Yu). No code is ported from either project: both graphs are consumed opaquely through onnxruntime, so LibreYOLO's own wrapper code, which carries no third-party architecture, is MIT throughout. Any other ArcFace-convention ONNX recognition model can be substituted by passing its file path, and that file's own license then applies instead of AuraFace-v1's. The bundled default face detector is a second artifact under a second license: OpenCV Zoo's YuNet, MIT, copyright Shiqi Yu. No architecture code is ported from either project; both graphs are consumed opaquely through `onnxruntime`, so LibreYOLO's own wrapper carries no third-party code and is MIT throughout. ## Citation ```bibtex @inproceedings{deng2019arcface, title={Arcface: Additive angular margin loss for deep face recognition}, author={Deng, Jiankang and Guo, Jia and Xue, Niannan and Zafeiriou, Stefanos}, booktitle={CVPR}, year={2019} } ``` Copied from https://github.com/deepinsight/insightface/tree/master/recognition/arcface_torch#citations --- # LibreMODUS LibreMODUS is an inference-only integration of the MODUS 14B-A7B checkpoint, an any-to-any model that turns one image-derived input into another: RGB in, depth out; depth in, normals out; any of those plus a phrase, boxes out. LibreYOLO supports four tasks through the standard predict API and a wider set through any2any(). Tasks: Detection, depth, normal, edge. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install LibreMODUS needs its own extra, which pulls in `accelerate` for the big-model dispatch this checkpoint needs. ```bash pip install "libreyolo[modus]" ``` LibreYOLO does not redistribute or mirror MODUS weights. By default, loading a `LibreMODUS` model downloads the required files directly from `EPFL-VILAB/MODUS` at a pinned Hugging Face revision, and a fresh download always needs the user's own authenticated Hugging Face account, even if the upstream hosting gate is temporarily open. Review and accept the upstream terms, then authenticate: ```bash hf auth login ``` ```python from libreyolo import LibreMODUS model = LibreMODUS(token="hf_...") ``` To avoid any network request, point at a snapshot you already have: ```python model = LibreMODUS(checkpoint_path="/models/MODUS") ``` That directory must contain `model.safetensors`, `ae.safetensors`, `llm_config.json`, `vit_config.json`, `tokenizer_config.json`, `vocab.json` and `merges.txt`. See Licensing below for what the checkpoint's terms permit. ## Predict **Python** ```python from libreyolo import LibreMODUS model = LibreMODUS(size="14b-a7b", task="normal") result = model.predict("room.jpg") normals = result.normal_map.data model.set_task("edge") result = model.predict("room.jpg") edges = result.edges.data # With no custom vocabulary, detect decodes the checkpoint's COCO # label tokens into contiguous COCO-80 class ids. model.set_task("detect") result = model.predict("street.jpg") print(result.boxes.xyxy) ``` **Phrase grounding** ```python from libreyolo import LibreMODUS model = LibreMODUS(task="detect") # set_classes() switches detection to phrase grounding: each phrase # runs independently and returns through the same Boxes contract. model.set_classes(["red bus", "cyclist"]) result = model.predict("street.jpg", conf=0.2) print(result.boxes.xyxy, result.boxes.cls) ``` **any2any()** ```python from libreyolo import LibreMODUS model = LibreMODUS() # One to three image-derived inputs (rgb, depth, normal, canny/edge), # plus optional auxiliary text, composed toward one target. result = model.any2any( inputs={"rgb": "room.jpg"}, target="normal", steps=10, cfg=2.0, seed=0, ) normals = result.normal_map.data # Grounding through any2any() needs a text input naming the phrase. result = model.any2any( {"rgb": "street.jpg", "text": "red bus"}, target="grounding", ) print(result.boxes.xyxy) ``` The standard task API covers four tasks, each mapped to one MODUS target: `depth` to relative depth (`result.depth_map`), `normal` to surface normals (`result.normal_map`), `edge` to Canny-style edges (`result.edges`), and `detect` to COCO-80 boxes (`result.boxes`) unless `set_classes()` switches it to phrase grounding. `set_task()` switches between them on the same loaded model. The released recipe uses ten flow-sampling steps with text guidance 4.0 and image guidance 2.0; override them with `inference_steps=`, `inference_cfg=` and `inference_image_cfg=` at construction. `any2any()` reaches the wider public analysis surface: one to three image-derived inputs (`rgb`, `depth`, `normal`, `canny`/`edge`), plus optional auxiliary text, composed toward any one of depth, normals, edges, SAM-derived edges, COCO detection or phrase grounding. All image-derived inputs must describe the same aligned canvas; LibreMODUS rejects mismatched widths and heights rather than resizing them independently. `chain=(...)` generates intermediate targets and feeds them back into the same context, within the checkpoint's three-condition training budget. `verify=N` (N >= 2) generates N candidates and keeps the one that scores highest on a constrained self-consistency check, exposed as `result.verification_score`. `dtype="bf16"` (the default) matches the released checkpoint precision; `dtype="fp8"` stores eligible decoder-trunk linear weights as E4M3 with a per-output-channel scale, converts once into a local cache under `~/.cache/libreyolo/modus/fp8`, and dequantizes to the input dtype per matrix multiply, so it trades memory rather than trading accuracy at the activation level. `train()`, `val()` and `export()` all raise: LibreMODUS is inference-only, dataset validation is not offered, and there is no ONNX, TensorRT or TFLite export path. Batched `predict()` and test-time augmentation are also not supported; each call handles one image. ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: MODUS, EPFL Visual Intelligence and Learning Lab (VILAB) - Upstream license: Apache-2.0 (code); research-only per the upstream model card (weights) - Upstream source: https://github.com/EPFL-VILAB/Modus - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0 (code); research-only per the upstream model card (weights), distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: Apache-2.0 covers the EPFL-VILAB/Modus source repository that LibreYOLO's adapter follows; LibreYOLO's own port is MIT, permissive and usable in commercial and closed-source products, requiring only that license text and attribution travel with any copy you redistribute. The MODUS-14B-A7B checkpoint is a separate artifact under a separate license: its Hugging Face model card currently declares license: other, describes it as bagel-derived, and requests research-only use, terms that bind you directly rather than through LibreYOLO. LibreYOLO does not bundle, mirror, or redistribute this checkpoint in any form, quantized or otherwise; loading it always downloads the required files directly from EPFL-VILAB/MODUS at a pinned revision, which needs the user's own authenticated Hugging Face account and acceptance of the current upstream terms, or a checkpoint_path to a snapshot the user already obtained. One upstream file, modeling/bagel/modeling_utils.py, carries an incompatible CC BY-NC 4.0 license inherited from Meta's DiT and is not copied, adapted, or paraphrased anywhere in this port; the small permissive routines LibreYOLO needed were re-derived independently from Hugging Face Transformers (Apache-2.0) and OpenAI's guided-diffusion (MIT) instead. Training is not offered for this family. LibreYOLO does not host or mirror the MODUS checkpoint anywhere, including on its own Hugging Face org: loading it always pulls the pinned revision directly from EPFL-VILAB/MODUS, or reads a snapshot already on disk at `checkpoint_path`. ## Citation ```bibtex @article{ye2026modus, title = {MODUS: Decoder-only Any-to-Any Modeling of Diverse Modalities}, author = {Ye, Mingqiao and An, Zhaochong and Gao, Zhitong and Liu, Xian and Fleuret, Fran\c{c}ois and Li, Chuan and Zadeh, Amir and Belongie, Serge and Dehghan, Afshin and Allardice, Jesse and Mizrahi, David and Kar, O\u{g}uzhan Fatih and Bachmann, Roman and Zamir, Amir}, journal = {arXiv preprint arXiv:2607.25948}, year = {2026}, } ``` Copied from https://github.com/EPFL-VILAB/Modus#citation --- # LingBot-Vision LingBot-Vision is a family of self-supervised vision transformer backbones trained with boundary-centric masked modeling for dense spatial perception, released by Robbyant. LibreYOLO pairs the backbone with a dense head and supports it for one task, semantic segmentation. Tasks: semantic. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install LingBot-Vision needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreLingBotVisions-sem.pt") result = model(SAMPLE_IMAGE, save=True) mask = result.semantic_mask print(mask.data.shape, mask.classes) ``` **CLI** ```bash libreyolo predict model=LibreLingBotVisions-sem.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` `result.semantic_mask` carries the dense class map: `.data` is an `(H, W)` tensor of class IDs on the original image size, and `.classes` lists the class IDs actually present. `result.boxes` is `None`, since there are no per-instance detections. `conf` and `iou` are accepted for API parity but do not change the output, since the model returns one class per pixel rather than detections to filter. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Three published sizes, s, b and l, distilled from a 1.1B-parameter ViT-g/16 teacher. The teacher itself, size `g`, loads and fine-tunes in LibreYOLO but LibreYOLO does not host a `g` checkpoint of its own. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreLingBotVisions-sem.pt` | | semantic | apache-2.0 | | `LibreLingBotVisionb-sem.pt` | | semantic | apache-2.0 | | `LibreLingBotVisionl-sem.pt` | | semantic | apache-2.0 | ## Train `train()` fine-tunes a published checkpoint. The default recipe is the upstream report's linear probe: the ViT backbone is frozen and only the 1x1 dense head trains, matching how the LibreYOLO-hosted weights above were produced. Pass `freeze_backbone=False` to fine-tune the whole network instead, and expect to lower `lr0` accordingly. **Python (linear probe)** ```python from libreyolo import LibreYOLO # Backbone frozen by default, matching the upstream evaluation # protocol: only the 1x1 dense head trains. model = LibreYOLO("LibreLingBotVisions-sem.pt") model.train(data="my-dataset.yaml", epochs=20, imgsz=512, batch=16) ``` **CLI** ```bash libreyolo train model=LibreLingBotVisions-sem.pt data=my-dataset.yaml \ epochs=20 imgsz=512 batch=16 ``` **Full fine-tune** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreLingBotVisions-sem.pt") model.train( data="my-dataset.yaml", epochs=20, imgsz=512, batch=16, freeze_backbone=False, ) ``` **Multi-GPU** ```bash libreyolo train model=LibreLingBotVisions-sem.pt data=my-dataset.yaml \ epochs=20 device=0,1 batch=32 ``` See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary of `metrics/` keys: mIoU and pixel accuracy, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreLingBotVisions-sem.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mIoU"]) print(metrics["metrics/pixel_accuracy"]) ``` **CLI** ```bash libreyolo val model=LibreLingBotVisions-sem.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | semantic | yes | yes | yes | yes | yes | | | | | | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreLingBotVisions-sem.pt") model.export(format="onnx", imgsz=512) model.export(format="coreai", imgsz=512) ``` **CLI** ```bash libreyolo export model=LibreLingBotVisions-sem.pt format=onnx imgsz=512 ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreLingBotVisions-sem.onnx") result = model(SAMPLE_IMAGE) print(result.semantic_mask.data.shape) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreLingBotVisions-sem.pt` | | semantic | apache-2.0 | | `LibreLingBotVisionb-sem.pt` | | semantic | apache-2.0 | | `LibreLingBotVisionl-sem.pt` | | semantic | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: LingBot-Vision, Robbyant (Ant Group) - Upstream license: Apache-2.0 - Upstream source: https://github.com/robbyant/lingbot-vision - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. The LibreYOLO-hosted checkpoints combine Robbyant's Apache-2.0 backbone weights with a dense segmentation head LibreYOLO trained itself, so the whole checkpoint file carries the same permissive terms end to end. The upstream release documents its ViT as built on the DINOv2/DINOv3 architecture published by Meta AI. Robbyant distributes their implementation under Apache-2.0, and this LibreYOLO port was made only from the Robbyant repository, never from Meta's DINOv2 or DINOv3 code. ## Citation ```bibtex @article{lingbot-vision2026, title={Vision Pretraining for Dense Spatial Perception}, author={Fu, Zelin and Tan, Bin and Sun, Changjiang and Liu, Shaohui and Zheng, Kecheng and Xu, Yinghao and Zhu, Xing and Shen, Yujun and Xue, Nan}, journal={arXiv preprint arXiv:2607.05247}, year={2026} } ``` Copied from https://github.com/robbyant/lingbot-vision#-citation --- # LocateAnything LocateAnything is a vision-language grounding model released by NVIDIA that decodes bounding boxes and points in parallel rather than one coordinate token at a time. LibreYOLO wraps it as an open-vocabulary detector and pointer: any list of text labels becomes the class set, with no fixed head and no fine-tuning required. Tasks: Detection, point. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install LocateAnything needs the `vlm` extra, which pulls in `transformers` plus the `decord`, `lmdb` and `peft` packages its Hugging Face remote code imports at load time. ```bash pip install "libreyolo[vlm]" ``` ## Predict `LibreLocateAnything` is a Python class, not a `.pt` checkpoint: it is not loaded through the `LibreYOLO()` factory, and the `libreyolo` CLI does not resolve it. The `LibreVLM(...)` factory (`from libreyolo import LibreVLM`) also reaches this family by alias, e.g. `LibreVLM("locate-anything")`; the class used below is what it constructs. Loading it downloads and executes NVIDIA's own remote model code from Hugging Face, so LibreYOLO pins the download to one fixed commit revision rather than the mutable `main` branch, and logs a one-time license notice before the first download. **Python** ```python from libreyolo import LibreLocateAnything, SAMPLE_IMAGE model = LibreLocateAnything(size="3b") # Open vocabulary: any words work, not a fixed class head. Sticky # across every later predict()/track() call until set again. model.set_classes(["person", "bicycle", "dog"]) result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **Point prompting** ```python from libreyolo import LibreLocateAnything, SAMPLE_IMAGE # task="point" returns one point per matched object instead of a box. # Switch tasks on an already-loaded model with model.set_task("point"). model = LibreLocateAnything(size="3b", task="point") model.set_classes(["the person closest to the camera"]) result = model(SAMPLE_IMAGE, save=True) for pt in result.points: print(pt.cls, pt.conf, pt.xy) ``` **Raw chat** ```python from libreyolo import LibreLocateAnything, SAMPLE_IMAGE model = LibreLocateAnything(size="3b") # The escape hatch beneath the detection convenience: free-form # questions, counting, or any prompt the boxes wrapper doesn't cover. text = model.chat(SAMPLE_IMAGE, "Describe the scene in one sentence.") print(text) ``` `result.boxes` (task `detect`) and `result.points` (task `point`) carry the parsed output like any other family. Confidence is a placeholder: LocateAnything emits no per-box score, so every detection gets the same constant confidence, and `conf=` only drops rows below that constant, it does not rank them. Skip `set_classes()` and the vocabulary defaults to the COCO-80 names. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants One published size, 3b. Two tasks share the same weights: `detect` (the default) returns boxes, and `task="point"` returns a single point per matched object instead, in `result.points`; switch between them on an already-loaded model with `model.set_task("point")`. LibreYOLO's benchmark harness has not measured this family, so there are no published accuracy numbers to compare against. LibreYOLO exposes this family for prediction only. `train()`, `val()` and `export()` all raise `NotImplementedError`: fine-tune upstream and load the result instead, dataset validation is skipped because a placeholder confidence would make COCO mAP misleading, and export is out of scope for a generative model with no state dict to trace. ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: LocateAnything, NVIDIA - Upstream license: NVIDIA License (non-commercial) - Upstream source: https://github.com/NVlabs/Eagle/tree/main/Embodied - LibreYOLO code: MIT - Weights: NVIDIA License (non-commercial), distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: The NVIDIA License permits use, reproduction and modification, but Section 3.3 restricts the Work and any derivative to non-commercial use, research or evaluation only, for anyone other than NVIDIA and its affiliates: there is no revenue threshold or paid exception. Redistribution must keep a complete copy of the license and every attribution notice. LocateAnything-3B also composes two other licensed components: a Qwen2.5-3B-Instruct language backbone under the Qwen Research License, and a MoonViT-SO-400M vision encoder under MIT. Because loading this model requires trusting NVIDIA's own Hugging Face remote code, LibreYOLO pins the exact commit revision it downloads rather than the mutable main branch, and logs a one-time notice before that download. LibreYOLO does not host, mirror or redistribute any of it. The NVIDIA License permits use, reproduction and modification, but restricts the model and any derivative to non-commercial use, research or evaluation only, for anyone other than NVIDIA and its affiliates: there is no revenue threshold or paid exception. LocateAnything-3B also composes two other licensed components: a Qwen2.5-3B-Instruct language backbone under the Qwen Research License, and a MoonViT-SO-400M vision encoder under MIT. LibreYOLO does not host, mirror or redistribute any of it: `LibreLocateAnything` downloads the weights and the required remote code directly from `nvidia/LocateAnything-3B` on Hugging Face, pinned to one fixed commit, the first time it runs. ## Citation ```bibtex @article{wang2025locateanything, title = {LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding}, author = {Shihao Wang and Shilong Liu and Yuanguo Kuang and Xinyu Wei and Yangzhou Liu and Zhiqi Li and Yunze Man and Guo Chen and Andrew Tao and Guilin Liu and Jan Kautz and Lei Zhang and Zhiding Yu}, journal = {arXiv:2605.27365}, year = {2026}, } ``` Copied from https://github.com/NVlabs/Eagle/blob/main/Embodied/README.md#-citation --- # LW-DETR A plain-ViT detection transformer that Baidu positioned as a real-time alternative to YOLO detectors. LibreYOLO ships five sizes for detection, inference only. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install LW-DETR needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreLWDETRt.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreLWDETRt.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. `conf` and `max_det` filter the query selection; `iou` is accepted for API parity but has no effect, because the decoder is a set predictor with no NMS step. See [prediction](/docs/predict) for sources, streaming and result handling. LW-DETR is inference-only in LibreYOLO. Upstream trains with Group-DETR one-to-many supervision across multiple query groups and an IoU-aware classification loss; that recipe is not wired here, so `train()` raises `NotImplementedError`. ## Variants Five sizes, all sharing the plain-ViT encoder, multi-scale projector and deformable DETR decoder, and all running at the same input resolution. The two smallest share an encoder width and split by block depth; the next two share a wider encoder and split by how many projector levels feed the decoder; the largest steps up to the widest encoder. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreLWDETRt.pt") # val() returns a plain dict, not an object metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) print(metrics["metrics/precision"], metrics["metrics/recall"]) ``` **CLI** ```bash libreyolo val model=LibreLWDETRt.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreLWDETRt.pt") model.export(format="onnx", imgsz=640) model.export(format="tensorrt", imgsz=640, half=True) ``` **CLI** ```bash libreyolo export model=LibreLWDETRt.pt format=onnx imgsz=640 libreyolo export model=LibreLWDETRt.pt format=tensorrt imgsz=640 half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreLWDETRt.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreLWDETRt.pt` | 640 | Detection | apache-2.0 | | `LibreLWDETRs.pt` | 640 | Detection | apache-2.0 | | `LibreLWDETRm.pt` | 640 | Detection | apache-2.0 | | `LibreLWDETRl.pt` | 640 | Detection | apache-2.0 | | `LibreLWDETRx.pt` | 640 | Detection | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: LW-DETR, Baidu - Upstream license: Apache-2.0 - Upstream source: https://github.com/Atten4Vis/LW-DETR - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours. The published checkpoints are converted from the upstream Apache-2.0 Hugging Face release (xbsu/LW-DETR) and rehosted under the same license. ## Citation ```bibtex @article{chen2024lw, title={LW-DETR: A Transformer Replacement to YOLO for Real-Time Detection}, author={Chen, Qiang and Su, Xiangbo and Zhang, Xinyu and Wang, Jian and Chen, Jiahui and Shen, Yunpeng and Han, Chuchu and Chen, Ziliang and Xu, Weixiang and Li, Fanrong and others}, journal={arXiv preprint arXiv:2406.03459}, year={2024} } ``` Copied from https://github.com/Atten4Vis/LW-DETR#10-citation --- # Mask R-CNN Mask R-CNN adds a per-region mask branch to Faster R-CNN, predicting a segmentation mask alongside each box it detects. LibreYOLO ports the torchvision implementation for detection and instance segmentation. Tasks: Detection, Instance segmentation. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install Mask R-CNN needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreMaskRCNNr50.pt") result = model(SAMPLE_IMAGE, save=True) print(result.masks.data.shape) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreMaskRCNNr50.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **Boxes only** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # task="detect" skips the mask head and returns boxes from the same # checkpoint, with no masks in the result. model = LibreYOLO("LibreMaskRCNNr50.pt", task="detect") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. Loading the checkpoint with no `task` argument returns instance masks, since segmentation is this family's default task; `result.masks` then carries them alongside the boxes. Passing `task="detect"` loads the same weights without the mask head and returns boxes only. `conf` and `iou` set the confidence and NMS thresholds; Mask R-CNN keeps its upstream NMS step, unlike a query-based detector. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants One backbone: ResNet-50 with a feature pyramid, using torchvision's v2 Mask R-CNN builder. The published checkpoint carries a BSD-3-Clause license and serves both tasks in this family, so there is no size to choose between. ## Validate `val()` returns a dictionary of `metrics/` keys. Against this checkpoint's default segmentation task, the plain `metrics/mAP50-95` key holds the mask score, and the same run reports boxes under the `(B)` suffix so both are available from one pass. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreMaskRCNNr50.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) # masks print(metrics["metrics/mAP50-95(B)"]) # boxes ``` **CLI** ```bash libreyolo val model=LibreMaskRCNNr50.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | | | | | | | | | | | | | Instance segmentation | yes | | | | | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. Mask R-CNN exports to ONNX only, at batch size 1. The exported graph keeps the upstream resize and mask-paste steps inside it, so LibreYOLO forces `dynamic=True` regardless of what is passed, to keep the graph valid for sources that are not square. An exported `.onnx` file loads back through `LibreYOLO()` on its file suffix and returns the same `Results`. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreMaskRCNNr50.pt") model.export(format="onnx", imgsz=800) ``` **CLI** ```bash libreyolo export model=LibreMaskRCNNr50.pt format=onnx imgsz=800 ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreMaskRCNNr50.onnx") result = model(SAMPLE_IMAGE) print(result.masks.data.shape) ``` ## Checkpoints Every published weight file for this family. The one checkpoint below is listed under detect, but the same file loads for segmentation too: pass no `task` argument and it returns masks by default. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreMaskRCNNr50.pt` | 800 | Detection | bsd-3-clause | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: Mask R-CNN, PyTorch - Upstream license: BSD-3-Clause - Upstream source: https://github.com/pytorch/vision - LibreYOLO code: BSD-3-Clause - Weights: BSD-3-Clause, republished at https://huggingface.co/LibreYOLO - Interpretation: BSD-3-Clause is a permissive license, so this code can be used in commercial and closed-source products with no obligation on your own application code. It asks only that you keep the copyright notice and disclaimer with any copy you redistribute, and it carries no patent grant. The published checkpoint used for parity testing is not distributed in the LibreYOLO source tree: torchvision's own documentation notes that a pretrained model's terms may depend on its training data, so the Hugging Face mirror ships the BSD text on that implied basis and repeats the caveat rather than issuing an explicit checkpoint-specific grant. Mask R-CNN is built as a subclass of LibreYOLO's Faster R-CNN wrapper: it shares the same torchvision source and BSD-3-Clause license, and adds the mask predictor and mask RoI head from the same ported commit. --- # MiDaS MiDaS is monocular relative depth estimation trained with scale-and-shift invariant loss across mixed datasets, the line of work that established the zero-shot depth transfer protocol later families reuse. LibreYOLO supports it for the depth task: predict and zero-shot validation, with no training path. Tasks: depth. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install MiDaS needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict MiDaS is the one depth family LibreYOLO does not republish on its own Hugging Face organization. Requesting a checkpoint by its LibreYOLO filename downloads the matching official asset directly from the `isl-org/MiDaS` GitHub releases, checks it against a pinned SHA-256, and wraps it with LibreYOLO's checkpoint metadata before first use; later runs reuse the cached local file. See Licensing for why. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # Not on disk yet: LibreYOLO downloads it from the official isl-org/MiDaS # GitHub release and checks it against a pinned SHA-256 before use. model = LibreYOLO("LibreMiDaSl-depth.pt") result = model(SAMPLE_IMAGE, save=True) depth = result.depth_map print(depth.min, depth.max, depth.mean) ``` **CLI** ```bash libreyolo predict model=LibreMiDaSl-depth.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **Small variant** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # EfficientNet-Lite3 encoder, smaller and faster than the DPT-Large l size. model = LibreYOLO("LibreMiDaSs-depth.pt") result = model(SAMPLE_IMAGE, save=True) ``` `result.depth_map` carries a dense relative inverse-depth map: higher values mean closer to the camera, and the values have no metric unit or cross-image scale. `save=True` writes a colormapped visualization of that map to disk; `Results.plot()` does not cover this family, since it is defined for surface normals and edges only. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Two variants with different encoders, not just different scales of the same one. `s` is MiDaS v2.1 Small, an EfficientNet-Lite3 encoder. `l` is DPT-Large, a ViT-L/16 encoder with the DPT decoder MiDaS introduced for dense prediction. They also preprocess differently: `s` uses an upper-bound aspect resize with ImageNet mean/std normalization, `l` uses a minimal aspect resize with mean and std of 0.5. Pick `s` for a lighter CNN, `l` for the transformer decoder's accuracy. Training is not offered for this family. `LibreMiDaS.train()` raises `NotImplementedError` unconditionally. ## Validate `val()` runs the shared depth validator: it aligns each prediction to its ground truth with a per-image least-squares scale and shift, then reports the standard zero-shot relative-depth metrics, AbsRel, RMSE and the three delta thresholds. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreMiDaSl-depth.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/abs_rel"]) print(metrics["metrics/rmse"]) print(metrics["metrics/delta1"]) ``` **CLI** ```bash libreyolo val model=LibreMiDaSl-depth.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | depth | yes | yes | yes | yes | yes | | | | yes | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`, with `depth_map` in place of boxes. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreMiDaSl-depth.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreMiDaSl-depth.pt format=onnx libreyolo export model=LibreMiDaSl-depth.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreMiDaSl-depth.onnx") result = model(SAMPLE_IMAGE) print(result.depth_map.data.shape) ``` ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: MiDaS, Intel Intelligent Systems Lab (Intel ISL) - Upstream license: MIT - Upstream source: https://github.com/isl-org/MiDaS - LibreYOLO code: MIT - Weights: MIT, distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: The isl-org/MiDaS repository, code and released checkpoints included, is MIT. LibreYOLO's own port code is MIT as well. LibreYOLO does not republish the checkpoints on its own Hugging Face organization, though: the family downloads the two official release assets directly from GitHub and checks them against a pinned SHA-256 before wrapping them, because an internal LibreYOLO policy (ADR 0006) requires the training-dataset mixture's commercial-redistribution terms to be cleared before LibreYOLO hosts a depth checkpoint itself, and that clearance has not happened for MiDaS. The bytes you get are upstream's own MIT-licensed release either way. ## Citation ```bibtex @ARTICLE {Ranftl2022, author = "Ren\'{e} Ranftl and Katrin Lasinger and David Hafner and Konrad Schindler and Vladlen Koltun", title = "Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer", journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence", year = "2022", volume = "44", number = "3" } @article{Ranftl2021, author = {Ren\'{e} Ranftl and Alexey Bochkovskiy and Vladlen Koltun}, title = {Vision Transformers for Dense Prediction}, journal = {ICCV}, year = {2021}, } ``` Copied from https://github.com/isl-org/MiDaS#citation --- # MobileNetV4 MobileNetV4 is an image classifier built for mobile and edge hardware, using the Universal Inverted Bottleneck block to unify several prior mobile block designs into one searchable structure. LibreYOLO supports it for one task: classification. Tasks: classify. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install MobileNetV4 needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreMobileNetV4s-cls.pt") result = model(SAMPLE_IMAGE, save=True) print(result.probs.top1, result.probs.top1conf) print(result.probs.top5) ``` **CLI** ```bash libreyolo predict model=LibreMobileNetV4s-cls.pt source=cat.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different model is a one line change. A classifier carries no boxes or masks: `result.probs` holds the whole-image prediction, with `top1`, `top5`, `top1conf` and `top5conf`. `conf`, `iou` and `max_det` are accepted for API parity but have no effect, since there is nothing to threshold or suppress on a single probability vector. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Three sizes, small/medium/large, all conv-only: this family excludes the hybrid variants that add Mobile MQA attention. Picking a size is a straight parameter-count-for-accuracy trade. The task is fixed: every size covers classification only. The weights filename ends `-cls.pt` on every size, and that suffix is what the factory reads to route to this family; no `task=` argument is needed. ## Train Fine-tuning starts from the published ImageNet backbone and rebuilds the final classifier layer to the target dataset's class count automatically. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreMobileNetV4s-cls.pt") model.train(data="imagenette160", epochs=5) ``` **CLI** ```bash libreyolo train model=LibreMobileNetV4s-cls.pt data=imagenette160 epochs=5 ``` **Multi-GPU** ```bash libreyolo train model=LibreMobileNetV4s-cls.pt data=imagenette160 \ epochs=50 device=0,1 batch=-1 ``` Left alone, the trainer runs 100 epochs at `lr0=1e-3` with AdamW, a batch of 64 and early stopping after 50 epochs without improvement. `data` accepts a dataset root (`train/` and `val/`, one folder per class), a known short name such as `imagenette160`, or a `.zip` URL. `lora=True` is not supported here; passing it raises, since LoRA in LibreYOLO targets transformer components with `nn.Linear` layers and this family's UIB blocks have none. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary of `metrics/` keys. For classification that is top-1 and top-5 accuracy over the validation split. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreMobileNetV4s-cls.pt") metrics = model.val(data="imagenette160") print(metrics["metrics/accuracy_top1"]) print(metrics["metrics/accuracy_top5"]) ``` **CLI** ```bash libreyolo val model=LibreMobileNetV4s-cls.pt data=imagenette160 ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | classify | yes | yes | yes | yes | yes | | | | yes | yes | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts and the extras a few of them add. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreMobileNetV4s-cls.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreMobileNetV4s-cls.pt format=onnx libreyolo export model=LibreMobileNetV4s-cls.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreMobileNetV4s-cls.onnx") result = model(SAMPLE_IMAGE) print(result.probs.top1) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreMobileNetV4s-cls.pt` | 224 | classify | apache-2.0 | | `LibreMobileNetV4m-cls.pt` | 224 | classify | apache-2.0 | | `LibreMobileNetV4l-cls.pt` | 256 | classify | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: MobileNetV4, Google - Upstream license: Apache-2.0 - Upstream source: https://github.com/huggingface/pytorch-image-models - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours. The architecture is Google's design, whose official code lives in the tensorflow/models repository; LibreYOLO's implementation follows the conv-only MobileNetV4 (small/medium/large) block definitions, channel rounding and naming in timm, whose mobilenetv4_conv_{small,medium,large} ImageNet-1k weights are licensed Apache-2.0 and are what LibreYOLO ships. The hybrid variants, which add Mobile MQA attention, are not part of this family. --- # MobileSAM MobileSAM replaces SAM's ViT-H image encoder with a distilled TinyViT encoder, so the same promptable point-and-box workflow runs on lighter hardware. LibreYOLO carries a native port of it through a dedicated LibreSAM factory, separate from the LibreYOLO() detector factory. Tasks: Instance segmentation. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install MobileSAM needs the `sam` extra: LibreYOLO's own weight download still goes through `transformers`' Hugging Face snapshot tooling, even though inference runs on a native, non-`transformers` decoder. ```bash pip install "libreyolo[sam]" ``` ## Predict `LibreSAM(...)` (or the family-specific `LibreMobileSAM(...)`) is a separate entry point from `LibreYOLO(...)`: it returns a promptable segmenter rather than a detector, because a forward pass here is meaningless without a spatial prompt. There is no `libreyolo predict` CLI command for this family; use the Python API. **Point and box prompts** ```python from libreyolo import LibreSAM, SAMPLE_IMAGE # MobileSAM has a single size, "tiny", so no other alias is needed. model = LibreSAM("mobilesam") # A point prompt: [x, y] in pixel coordinates, label 1 = foreground. result = model.predict(SAMPLE_IMAGE, points=[640, 420], labels=[1]) print(result.masks.xy) # polygon per mask print(result.boxes.xyxy) # tight box derived from the mask # A box prompt instead of a point. result = model.predict(SAMPLE_IMAGE, bboxes=[300, 200, 900, 700]) # No prompt at all segments the whole image (a simplified automatic # mask generator, not the exhaustive reference one). result = model.predict(SAMPLE_IMAGE) ``` **Encode once, prompt many** ```python from libreyolo import LibreMobileSAM, SAMPLE_IMAGE model = LibreMobileSAM() # The image encoder is the expensive part. set_image() runs it once; # every predict() call after that reuses the cached embedding. model.set_image(SAMPLE_IMAGE) a = model.predict(points=[640, 420], labels=[1]) b = model.predict(bboxes=[300, 200, 900, 700]) model.reset_image() ``` A point prompt accepts `[x, y]` for one object, `[[x, y], ...]` for several, or numpy arrays; `labels` marks each point `1` (foreground) or `0` (background) and defaults to all foreground. A box prompt takes `[x1, y1, x2, y2]` or a list of boxes, one mask per box. Omitting both prompts segments the whole image by prompting a dense grid and keeping the confident, non-overlapping masks; this "segment everything" mode is simplified against the reference automatic mask generator and can under-segment crowded scenes, so a real point or box prompt is the precise path. `conf` filters by predicted mask quality (IoU), not a detection confidence: pass `0.0` to keep every candidate. `multimask=True` returns all three of SAM's whole-versus-part ambiguity masks per prompt instead of the single best one. `device=` moves the model and, if a `set_image()` session is active, its cached embedding. Every mask carries class id `0`, named `"object"`, since a promptable mask has no fixed class set. `train()`, `val()`, `export()` and `track()` all raise `NotImplementedError` for this family: MobileSAM is predict-only in LibreYOLO. See [prediction](/docs/predict) for source types. ## Variants One size, tiny, at a fixed 1024 px input: MobileSAM ships a single TinyViT encoder rather than the base/large/huge ladder SAM-1 offers. ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreMobileSAM.pt` | | Instance segmentation | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: MobileSAM, Kyung Hee University - Upstream license: Apache-2.0 - Upstream source: https://github.com/ChaoningZhang/MobileSAM - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so this code and these weights can be used in commercial and closed-source products. It asks you to keep the license text and attribution notices with any copy you redistribute, and it grants a patent license. LibreYOLO carries a native port of the TinyViT image encoder, prompt encoder, two-way transformer and mask decoder rather than vendoring upstream files unmodified, checked for bit-exact parity against the original Apache-2.0 implementation, with a NOTICE recording that provenance. The converted checkpoint is hosted as LibreMobileSAM.pt on the LibreYOLO Hugging Face org, tagged Apache-2.0 there as well. ## Citation ```bibtex @article{mobile_sam, title={Faster Segment Anything: Towards Lightweight SAM for Mobile Applications}, author={Zhang, Chaoning and Han, Dongshen and Qiao, Yu and Kim, Jung Uk and Bae, Sung-Ho and Lee, Seungkyu and Hong, Choong Seon}, journal={arXiv preprint arXiv:2306.14289}, year={2023} } ``` Copied from https://github.com/ChaoningZhang/MobileSAM#bibtex-of-our-mobilesam --- # MoGe-2 MoGe-2 is a single-forward monocular geometry model that predicts a dense surface-normal field from one RGB image. LibreYOLO supports it for normal estimation only, through the official ViT-S, ViT-B and ViT-L checkpoints. Tasks: normal. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install MoGe-2 needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download automatically on first use: LibreYOLO fetches the matching size directly from the official checkpoints and caches it locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreMoGe2s-normal.pt") result = model(SAMPLE_IMAGE, save=True) normal = result.normal_map print(normal.array.shape) # (H, W, 3) float32 unit vectors ``` **CLI** ```bash libreyolo predict model=LibreMoGe2s-normal.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` MoGe-2 returns a dense field rather than a set of detections, so `result.boxes` is empty and `conf`, `iou` and `max_det` have no effect. `result.normal_map` holds the result: an `(H, W, 3)` array of unit vectors in the OpenCV camera frame, where `+x` is right, `+y` is down, `+z` is into the scene, and a surface facing the camera reads `(0, 0, -1)`. Predicting a list of images runs one forward pass per image; this family has no stacked-batch fast path. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Three encoder sizes ship as separate checkpoints: ViT-S, ViT-B and ViT-L, all at the same input resolution. LibreYOLO's benchmark harness has not measured this family, so there are no published accuracy numbers to compare them by; pick a size against your own compute budget. ## Validate `val()` measures angular error against a paired normal-map dataset: images beside same-stem 16-bit normal PNGs, with an optional validity mask so padded and invalid pixels never count. It returns the mean and median angular error in degrees, plus the percentage of pixels within 11.25, 22.5 and 30 degrees. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreMoGe2s-normal.pt") metrics = model.val(data="my-dataset.yaml", imgsz=518) print(metrics["metrics/mean_angular_error"]) # degrees print(metrics["metrics/median_angular_error"]) print(metrics["metrics/within_11_25"]) # percent of pixels ``` **CLI** ```bash libreyolo val model=LibreMoGe2s-normal.pt data=my-dataset.yaml imgsz=518 ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | normal | yes | yes | yes | yes | yes | | | | yes | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. Normal export uses a fixed-resolution, batch-1 runtime contract: `dynamic` and a `batch` other than 1 are rejected, and `imgsz` must be divisible by the ViT encoder's patch size, which LibreYOLO checks before the run starts. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` file behaves like a checkpoint and returns the same `Results`. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreMoGe2s-normal.pt") model.export(format="onnx", imgsz=518) model.export(format="tensorrt", imgsz=518, half=True) ``` **CLI** ```bash libreyolo export model=LibreMoGe2s-normal.pt format=onnx imgsz=518 libreyolo export model=LibreMoGe2s-normal.pt format=tensorrt imgsz=518 half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreMoGe2s-normal.onnx") result = model(SAMPLE_IMAGE) print(result.normal_map.array.shape) ``` ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: MoGe-2, Microsoft - Upstream license: MIT - Upstream source: https://github.com/microsoft/MoGe - LibreYOLO code: MIT - Weights: MIT, distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: MIT is a permissive license: the code and the official ViT-S, ViT-B and ViT-L checkpoints can be used in commercial and closed-source products. It asks that you keep the license text and copyright notice with any copy you redistribute, and it places no obligation on your own application code. LibreYOLO downloads these checkpoints directly from the official Hugging Face repositories at a pinned revision rather than copying them into its own organization, and verifies each file against a recorded SHA-256 checksum before use. The DINOv2 encoder MoGe-2 builds on is separately licensed Apache-2.0 by Meta AI; LibreYOLO reuses the DINOv2 implementation already bundled for its Depth Anything V2 family rather than copying it again here. LibreYOLO does not copy these checkpoints into its own organization. `LibreYOLO("LibreMoGe2s-normal.pt")` downloads the matching size directly from the official Hugging Face repositories at a pinned revision, and verifies the file against a recorded SHA-256 checksum before use. ## Citation ```bibtex @inproceedings{wang2025moge, title={Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision}, author={Wang, Ruicheng and Xu, Sicheng and Dai, Cassie and Xiang, Jianfeng and Deng, Yu and Tong, Xin and Yang, Jiaolong}, booktitle={Proceedings of the Computer Vision and Pattern Recognition Conference}, pages={5261--5271}, year={2025} } @misc{wang2025moge2, title={MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Details}, author={Ruicheng Wang and Sicheng Xu and Yue Dong and Yu Deng and Jianfeng Xiang and Zelong Lv and Guangzhong Sun and Xin Tong and Jiaolong Yang}, year={2025}, eprint={2507.02546}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2507.02546}, } ``` Copied from https://github.com/microsoft/MoGe#-citation --- # NAFNet NAFNet is a convolutional network for image restoration that removes the nonlinear activation functions from a typical UNet block, replacing them with elementwise multiplication. LibreYOLO supports it for one task, restoration, with a published real-image denoising checkpoint trained on SIDD. Tasks: restore. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install NAFNet needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreNAFNetl-restore-sidd.pt") result = model("noisy.jpg", save=True) restored = result.restored print(restored.array.shape) ``` **CLI** ```bash libreyolo predict model=LibreNAFNetl-restore-sidd.pt source=noisy.jpg save=True ``` **Save the restored image** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreNAFNetl-restore-sidd.pt") result = model.predict("noisy.jpg") result.restored.save("denoised.png") ``` The returned `Results` object carries one field for this family, `restored`, a dense HWC uint8 RGB image on the original canvas; there are no boxes to iterate. `save=True` writes that restored image straight to disk rather than drawing an annotation over the input. `conf`, `iou` and `max_det` are accepted for signature parity with every other family but have no effect, since restoration produces no detections to filter. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Two widths share this architecture: `s` (width 32) and `l` (width 64), both built around a 256 px training patch. Predict and validate run at native image resolution regardless of size, padding only to the network's downsample factor. Only the `l` width is currently published, as a real-image denoising checkpoint trained on SIDD. ## Train NAFNet fine-tunes on your own paired degraded/clean images: a dataset YAML pointing at an `inputs//` folder of degraded images and a `targets//` folder of clean targets, matched by filename stem. `degradation` and `dataset` are optional strings recorded on the saved checkpoint for provenance; they take no part in training. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreNAFNetl-restore-sidd.pt") model.train(data="my-dataset.yaml", epochs=100, imgsz=256, batch=16, lr0=1e-3) ``` **CLI** ```bash libreyolo train model=LibreNAFNetl-restore-sidd.pt data=my-dataset.yaml \ epochs=100 imgsz=256 batch=16 lr0=1e-3 ``` **Checkpoint provenance** ```python from libreyolo import LibreYOLO # degradation and dataset are recorded on the saved checkpoint; they # don't change what is trained. model = LibreYOLO("LibreNAFNetl-restore-sidd.pt") model.train( data="my-dataset.yaml", epochs=100, degradation="denoise", dataset="MyDataset", ) ``` **Multi-GPU** ```bash libreyolo train model=LibreNAFNetl-restore-sidd.pt data=my-dataset.yaml \ epochs=100 device=0,1 batch=32 ``` Left alone, the trainer runs 100 epochs with AdamW at `lr0=1e-3`, a batch of 16, 256 px crops, and early stopping after 50 epochs without PSNR improvement. There is no LoRA path for this family: `lora=True` raises an error rather than running, since `NAFNetTrainer` never opts in to adapter fine-tuning. During training the network runs with plain global-average pooling. NAFNet's inference-only windowed local pooling (Test-time Local Converter) is detached before the first epoch and reattached once training finishes, since backpropagating through a fixed-window local pool would not match how the checkpoint is used at inference. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary with `metrics/PSNR` and `metrics/SSIM`, computed in RGB over the full valid canvas: SSIM uses an 11x11 Gaussian window with sigma 1.5, and `fitness` for best-checkpoint selection is the PSNR value. `data` points at the same paired-image dataset format used for training. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreNAFNetl-restore-sidd.pt") # val() returns a plain dict, not an object metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/PSNR"]) print(metrics["metrics/SSIM"]) ``` **CLI** ```bash libreyolo val model=LibreNAFNetl-restore-sidd.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | restore | yes | yes | yes | yes | yes | | | | yes | | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`, with `restored` carrying the output image. NAFNet exports at a fixed spatial resolution: `imgsz` must be divisible by the network's downsample factor (16 for both architecture widths), and only the batch dimension is dynamic when `dynamic=True`; height and width are fixed at export time. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreNAFNetl-restore-sidd.pt") model.export(format="onnx", imgsz=256) model.export(format="tensorrt", imgsz=256, half=True) ``` **CLI** ```bash libreyolo export model=LibreNAFNetl-restore-sidd.pt format=onnx imgsz=256 libreyolo export model=LibreNAFNetl-restore-sidd.pt format=tensorrt imgsz=256 half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreNAFNetl-restore-sidd.onnx") result = model("noisy.jpg") result.restored.save("denoised.png") ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreNAFNetl-restore-sidd.pt` | | restore | mit | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: NAFNet, Megvii - Upstream license: MIT - Upstream source: https://github.com/megvii-research/NAFNet - LibreYOLO code: MIT - Weights: MIT, republished at https://huggingface.co/LibreYOLO - Interpretation: MIT is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep the copyright notice and license text with any copy you redistribute, and places no other obligation on your own application code. Part of the training pipeline is ported from BasicSR under Apache-2.0, which additionally grants a patent license. The published checkpoint is trained on the Smartphone Image Denoising Dataset (SIDD), itself MIT-licensed. ## Citation ```bibtex @article{chen2022simple, title={Simple Baselines for Image Restoration}, author={Chen, Liangyu and Chu, Xiaojie and Zhang, Xiangyu and Sun, Jian}, journal={arXiv preprint arXiv:2204.04676}, year={2022} } ``` Copied from https://github.com/megvii-research/NAFNet#citations --- # OMDet-Turbo OMDet-Turbo is a real-time open-vocabulary object detector, developed by Om AI Lab, that decouples class embeddings from a language task prompt. LibreYOLO wraps it as a predict-only family in its open-vocabulary detector tier. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install OMDet-Turbo loads through LibreYOLO's open-vocabulary detector tier, which needs the `openvocab` extra: ```bash pip install "libreyolo[openvocab]" ``` That extra pulls in `transformers` and `timm`, the Hugging Face libraries this tier calls into; OMDet-Turbo's Swin backbone loads through `transformers`' `TimmBackbone` wrapper. ## Predict OMDet-Turbo is not a checkpoint LibreYOLO loads through `LibreYOLO()`. It loads through the sibling `LibreOpenVocab` factory, which downloads a Hugging Face snapshot on first use and caches it under `weights/`. **Python** ```python from libreyolo import LibreOpenVocab, SAMPLE_IMAGE model = LibreOpenVocab("omdet-turbo") model.set_classes(["person", "dog", "skateboard"]) result = model.predict(SAMPLE_IMAGE, conf=0.3) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **Custom NMS threshold** ```python from libreyolo import LibreOpenVocab, SAMPLE_IMAGE model = LibreOpenVocab("omdet-turbo") model.set_classes(["traffic light", "bicycle"]) # OMDet-Turbo is the one family in this tier that honours iou=: its # own post-processing takes the suppression threshold as an argument, # defaulting to 0.5 when iou= is left unset. result = model.predict(SAMPLE_IMAGE, conf=0.3, iou=0.7) print(result.names, len(result)) ``` `set_classes()` sets a sticky text vocabulary: call it again to replace the list outright, or skip it to keep the default COCO-80 labels, and an empty result is a valid outcome rather than an error. Unlike Grounding DINO, OMDet-Turbo decouples its class embeddings from the language task prompt, so `transformers`' post-processing returns labels that map straight back to the queried class list with no phrase-disambiguation step. OMDet-Turbo has no text-token threshold: only `conf` filters detections, and passing `text_threshold` raises. It is the one family in this tier that runs its own non-maximum suppression inside `post_process_grounded_object_detection`, so `iou` is honoured here rather than warned about. `imgsz` and `augment=True` are rejected outright: the `transformers` processor owns resizing, and test-time augmentation is out of scope for this tier. `predict()` on a single image returns one `Results`, not a list; pass a directory, a list of images, or `stream=True` for a video source to get several. There is no CLI path for this family, `libreyolo predict` only loads `.pt` checkpoints through `LibreYOLO()`, so `LibreOpenVocab` families run from Python. See [prediction](/docs/predict) for source types and streaming. ## Variants One checkpoint, `t`, the tier's only size. It mirrors `omlab/omdet-turbo-swin-tiny-hf` at a pinned upstream revision through `transformers`' `OmDetTurboForObjectDetection`; the mirrored weight file is byte-identical to that upstream snapshot. No accuracy or latency numbers are published for this family yet. Training, dataset validation and export are all out of scope for this tier: `train()`, `val()` and `export()` all raise `NotImplementedError` unconditionally. This is a predict-only wrapper around a published checkpoint. ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreOMDetTurbot.pt` | 640 | Detection | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: OMDet-Turbo, Om AI Lab - Upstream license: Apache-2.0 - Upstream source: https://github.com/om-ai-lab/OmDet - LibreYOLO code: MIT - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so this checkpoint can be used in commercial and closed-source products. It asks you to keep the license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. LibreYOLO vendors no OMDet-Turbo model source of its own: LibreOMDetTurbo calls the Apache-2.0 `transformers` implementation, `OmDetTurboForObjectDetection`, directly. The mirrored checkpoint is a byte-identical copy of `omlab/omdet-turbo-swin-tiny-hf` at a pinned upstream revision, verified against a recorded SHA-256 checksum before it is used. ## Citation ```bibtex @article{zhao2024real, title={Real-time Transformer-based Open-Vocabulary Detection with Efficient Fusion Head}, author={Zhao, Tiancheng and Liu, Peng and He, Xuan and Zhang, Lu and Lee, Kyusong}, journal={arXiv preprint arXiv:2403.06892}, year={2024} } ``` Copied from https://github.com/om-ai-lab/OmDet#citation --- # OV-DEIM OV-DEIM is a DETR-style open-vocabulary object detector that matches decoder queries to text embeddings from a bundled MobileCLIP text tower. LibreYOLO ports it natively as a predict-only family in its open-vocabulary detector tier. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install OV-DEIM loads through LibreYOLO's open-vocabulary detector tier, which needs the `openvocab` extra: ```bash pip install "libreyolo[openvocab]" ``` Unlike the rest of this tier, OV-DEIM is a native LibreYOLO port rather than a `transformers` wrapper, no `transformers` model class exists for it, but the same extra covers the `huggingface_hub`, `safetensors`, `regex` and `ftfy` packages it needs at predict time. ## Predict OV-DEIM is not a checkpoint LibreYOLO loads through `LibreYOLO()`. It loads through the sibling `LibreOpenVocab` factory, which downloads a Hugging Face snapshot on first use and caches it under `weights/`. **Python** ```python from libreyolo import LibreOpenVocab, SAMPLE_IMAGE model = LibreOpenVocab("ov-deim-s") model.set_classes(["person", "dog", "skateboard"]) result = model.predict(SAMPLE_IMAGE, conf=0.25) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **Replace the vocabulary** ```python from libreyolo import LibreOpenVocab, SAMPLE_IMAGE model = LibreOpenVocab("ov-deim-l") model.set_classes(["traffic light", "bicycle"]) first = model.predict(SAMPLE_IMAGE, conf=0.3) # A second call to set_classes() replaces the vocabulary outright and # re-embeds it through the text tower; an empty result is a valid # outcome rather than an error. model.set_classes(["giraffe"]) second = model.predict(SAMPLE_IMAGE, conf=0.5) print(second.names, len(second)) ``` `set_classes()` sets a sticky text vocabulary: call it again to replace the list outright, or skip it to keep the default COCO-80 labels, and an empty result is a valid outcome rather than an error. Each decoder query is scored by cosine similarity against text embeddings from a bundled MobileCLIP-B(LT) text tower, computed online for whatever vocabulary is set and cached until it changes, so arbitrary prompts work without any precomputed embedding file. OV-DEIM has no text-token threshold: only `conf` filters detections, and passing `text_threshold` raises. Matching is one-to-one top-K selection, so nothing here runs non-maximum suppression, and `iou` is accepted for API compatibility but warns and does nothing. `imgsz` and `augment=True` are rejected outright: the model owns a fixed letterboxed input, and test-time augmentation is out of scope for this tier. `predict()` on a single image returns one `Results`, not a list; pass a directory, a list of images, or `stream=True` for a video source to get several. There is no CLI path for this family, `libreyolo predict` only loads `.pt` checkpoints through `LibreYOLO()`, so `LibreOpenVocab` families run from Python. See [prediction](/docs/predict) for source types and streaming. Every call to `predict()` also runs the bundled MobileCLIP-B(LT) text tower to embed the current vocabulary; see Licensing for what that adds to the terms. ## Variants Three checkpoints, `s`, `m` and `l`. `s` is this tier's default size when none is given. Unlike the rest of this tier, OV-DEIM is a native port rather than a `transformers` wrapper: LibreYOLO vendors the detector modules under the same Apache-2.0 license as the upstream code and reuses the DINOv3 backbone adapter already built for the DEIMv2 family. The `l` checkpoint's backbone is a DINOv3-S fine-tune, licensed separately under Meta's DINOv3 License. No accuracy or latency numbers are published for this family yet. Training, dataset validation and export are all out of scope for this tier: `train()`, `val()` and `export()` all raise `NotImplementedError` unconditionally. This is a predict-only wrapper around a published checkpoint. ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreOVDEIMs.pt` | 640 | Detection | cc-by-nc-4.0 | | `LibreOVDEIMm.pt` | 640 | Detection | cc-by-nc-4.0 | | `LibreOVDEIMl.pt` | 640 | Detection | cc-by-nc-4.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: OV-DEIM, Leilei Wang et al. - Upstream license: CC BY-NC 4.0 - Upstream source: https://github.com/wleilei/OV-DEIM - LibreYOLO code: Apache-2.0 - Weights: CC BY-NC 4.0, republished at https://huggingface.co/LibreYOLO - Interpretation: OV-DEIM's code is Apache-2.0; LibreYOLO's port keeps that license and preserves the original RT-DETR and DEIMv2 attribution headers. The published S, M and L checkpoints carry a separate CC BY-NC 4.0 license: redistribution and format conversion are permitted with attribution, but only for non-commercial use, confirmed directly by the upstream author. Every prediction also runs a bundled MobileCLIP-B(LT) text tower, loaded unchanged from Apple's own release, to embed the vocabulary online; those weights carry the Apple Machine Learning Research Model license, which permits redistribution with the license text, an attribution notice and a record of modifications, but restricts use to research, a stricter term than CC BY-NC 4.0 that applies to every call this family makes. The `l` checkpoint's DINOv3-S backbone fine-tune is separately subject to Meta's DINOv3 License. OV-DEIM layers three upstream licenses onto every prediction call: the detector weights under OV-DEIM's own CC BY-NC 4.0, the online text tower under Apple's Machine Learning Research Model license (research use only), and, for the `l` checkpoint, a DINOv3-S backbone fine-tune under Meta's DINOv3 License. All three license texts ship inside the LibreYOLO weight repository. ## Citation ```bibtex @misc{wang2026ovdeim, title={OV-DEIM: Real-time DETR-Style Open-Vocabulary Object Detection with GridSynthetic Augmentation}, author={Leilei Wang and Longfei Liu and Xi Shen and Xuanlong Yu and Ying Tiffany He and Fei Richard Yu and Yingyi Chen}, year={2026}, eprint={2603.07022}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2603.07022}, } ``` Copied from https://github.com/wleilei/OV-DEIM#4-citation --- # OWLv2 OWLv2 is an open-vocabulary object detector, developed by Google Research, that scores image regions against text embeddings from a CLIP-style encoder. LibreYOLO wraps it as a predict-only family in its open-vocabulary detector tier. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install OWLv2 loads through LibreYOLO's open-vocabulary detector tier, which needs the `openvocab` extra: ```bash pip install "libreyolo[openvocab]" ``` That extra pulls in `transformers` and `timm`, the Hugging Face libraries this tier calls into. ## Predict OWLv2 is not a checkpoint LibreYOLO loads through `LibreYOLO()`. It loads through the sibling `LibreOpenVocab` factory, which downloads a Hugging Face snapshot on first use and caches it under `weights/`. **Python** ```python from libreyolo import LibreOpenVocab, SAMPLE_IMAGE model = LibreOpenVocab("owlv2-b16") model.set_classes(["person", "dog", "skateboard"]) result = model.predict(SAMPLE_IMAGE, conf=0.1) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **Default vocabulary** ```python from libreyolo import LibreOpenVocab, SAMPLE_IMAGE # Skipping set_classes() keeps the tier's default COCO-80 vocabulary. model = LibreOpenVocab("owlv2-l14") result = model.predict(SAMPLE_IMAGE, conf=0.1) print(result.names) ``` `set_classes()` sets a sticky text vocabulary: call it again to replace the list, or skip it to keep the default COCO-80 labels. Each label is wrapped in a fixed prompt template before it reaches the text tower, matching how `transformers`' `Owlv2ForObjectDetection` was trained. OWLv2 has no text-token threshold: only `conf` filters detections, and passing `text_threshold` raises. `iou` is accepted for API compatibility but warns and does nothing, since nothing here runs non-maximum suppression. `imgsz` and `augment=True` are rejected outright: the `transformers` processor owns resizing, and test-time augmentation is out of scope for this tier. `predict()` on a single image returns one `Results`, not a list; pass a directory, a list of images, or `stream=True` for a video source to get several. There is no CLI path for this family, `libreyolo predict` only loads `.pt` checkpoints through `LibreYOLO()`, so `LibreOpenVocab` families run from Python. See [prediction](/docs/predict) for source types and streaming. ## Variants Two checkpoints, `b16` (base, patch size 16) and `l14` (large, patch size 14). `b16` is this tier's default size when none is given. Both mirror the official Google Research release through `transformers`' `Owlv2ForObjectDetection`, downloaded once into a LibreYOLO-hosted Hugging Face snapshot that preserves the upstream files. No accuracy or latency numbers are published for this family yet. Training, dataset validation and export are all out of scope for this tier: `train()`, `val()` and `export()` all raise `NotImplementedError` unconditionally. This is a predict-only wrapper around a published checkpoint. ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreOWLv2b16.pt` | 960 | Detection | apache-2.0 | | `LibreOWLv2l14.pt` | 1008 | Detection | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: OWLv2, Google Research - Upstream license: Apache-2.0 - Upstream source: https://github.com/google-research/scenic/tree/main/scenic/projects/owl_vit - LibreYOLO code: MIT - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these checkpoints can be used in commercial and closed-source products. It asks you to keep the license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. LibreYOLO vendors no OWLv2 model source of its own: LibreOWLv2 calls the Apache-2.0 `transformers` implementation, `Owlv2ForObjectDetection`, directly, and downloads the official checkpoints into a LibreYOLO-hosted mirror repository that preserves the upstream snapshot files. ## Citation ```bibtex @article{minderer2023scaling, title={Scaling Open-Vocabulary Object Detection}, author={Matthias Minderer, Alexey Gritsenko, Neil Houlsby}, journal={NeurIPS}, year={2023}, } ``` Copied from https://github.com/google-research/scenic/blob/main/scenic/projects/owl_vit/README.md#references --- # PicoDet PicoDet is a single-stage detector built for mobile and edge CPUs: an ESNet backbone, a CSP-PAN neck and a shared Generalized Focal Loss head. LibreYOLO supports it for detection. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install PicoDet needs no extra beyond the base package. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibrePICODETs.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibrePICODETs.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. `conf` sets the confidence threshold and `iou` the NMS threshold. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Three sizes, each at its own fixed input resolution: `s` the smallest and `l` the largest. Resolution grows with the size, so larger checkpoints are also more expensive to run per image, on top of carrying more parameters. | Checkpoint | Input (px) | mAP 50-95 | Params (M) | | --- | --- | --- | --- | | `LibrePICODETs` | 320 | 29.5 | 0.99 | | `LibrePICODETm` | 416 | 37.6 | 2.15 | | `LibrePICODETl` | 640 | 44.2 | 3.31 | COCO val2017, 500 images. Published on https://www.visionanalysis.org/ Interactive chart: https://www.visionanalysis.org/embed/scatter?highlight=picodet-s%2Cpicodet-m%2Cpicodet-l&title=PicoDet%20on%20COCO&subtitle=Accuracy%20against%20latency%2C%20every%20benchmarked%20model ## Train **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibrePICODETs.pt") model.train( data="my-dataset.yaml", epochs=300, batch=16, lr0=0.01, ) ``` **CLI** ```bash # imgsz is worth setting: the CLI defaults it to 640, while the s # checkpoint is native at 320. libreyolo train model=LibrePICODETs.pt data=my-dataset.yaml imgsz=320 epochs=300 batch=16 lr0=0.01 ``` The loss components and the assigner follow the upstream recipe: VFL, DFL, GIoU and SimOTA, with classification-quality weighting and dynamic-IoU VFL targets. Inference is bit-equivalent to upstream on the same checkpoint. What has not been checked, per `train()`'s own docstring: full-dataset convergence, multi-GPU behavior, and any augmentation beyond horizontal flip. The `s` checkpoint at its native 320 has also not reliably cleared LibreYOLO's accuracy floor on the 30-image, two-class fixture the library tests small fine-tunes with. That size is a better fit at full-COCO scale. `train()` also accepts a `pretrained` argument, but the value is never read inside the method: training always continues from whatever weights the model was constructed with, so `pretrained=False` does not reinitialize the network. Leave `imgsz` unset in Python and it takes the loaded checkpoint's native resolution, 320 for `s`, 416 for `m` and 640 for `l`. The CLI always sends an `imgsz`, defaulting to 640, so set it there to match the checkpoint. Left alone otherwise, the trainer runs 300 epochs with SGD at `lr0=0.01`, momentum 0.9, weight decay 4e-5 and a 1-epoch warmup on a cosine schedule. Horizontal flip is the only augmentation applied. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibrePICODETs.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibrePICODETs.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | | | yes | yes | | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Running the graph in a bare runtime, with no LibreYOLO installed, is also supported, but then preprocessing and postprocessing are yours to write. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibrePICODETs.pt") model.export(format="onnx", imgsz=320) model.export(format="tensorrt", imgsz=320, half=True) ``` **CLI** ```bash libreyolo export model=LibrePICODETs.pt format=onnx imgsz=320 libreyolo export model=LibrePICODETs.pt format=tensorrt imgsz=320 half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibrePICODETs.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibrePICODETs.pt` | 320 | Detection | apache-2.0 | | `LibrePICODETm.pt` | 416 | Detection | apache-2.0 | | `LibrePICODETl.pt` | 640 | Detection | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: PP-PicoDet, PaddlePaddle (Baidu) - Upstream license: Apache-2.0 - Upstream source: https://github.com/PaddlePaddle/PaddleDetection - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours. LibreYOLO's port follows Bo396543018/Picodet_Pytorch, a PyTorch re-implementation of PaddleDetection's original PP-PicoDet, and both carry the same Apache-2.0 terms as the paper's authors. LibreYOLO's port follows Bo396543018/Picodet_Pytorch, a PyTorch re-implementation of PaddleDetection's original PP-PicoDet, with mmcv stripped out and every activation matched exactly so PaddlePaddle checkpoints converted through Bo's pipeline load with no numerical drift. Both sources carry the same Apache-2.0 terms as the paper's authors. ## Citation ```bibtex @misc{yu2021pppicodet, title={PP-PicoDet: A Better Real-Time Object Detector on Mobile Devices}, author={Guanghua Yu and Qinyao Chang and Wenyu Lv and Chang Xu and Cheng Cui and Wei Ji and Qingqing Dang and Kaipeng Deng and Guanzhong Wang and Yuning Du and Baohua Lai and Qiwen Liu and Xiaoguang Hu and Dianhai Yu and Yanjun Ma}, year={2021}, eprint={2111.00902}, archivePrefix={arXiv}, primaryClass={cs.CV} } ``` Copied from https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.8/configs/picodet/README_en.md#cite-pp-picodet --- # PicoSAM3 PicoSAM3 is a compact CNN distilled from SAM 2.1 and SAM 3, built for box-prompted region-of-interest segmentation on sensors like the Sony IMX500. LibreYOLO supports it through a dedicated LibreSAM factory, separate from the LibreYOLO() detector factory, with box prompts only. Tasks: Instance segmentation. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install PicoSAM3 needs the `sam` extra: LibreYOLO's own weight download still goes through `transformers`' Hugging Face tooling, even though inference runs on a native, non-`transformers` CNN. ```bash pip install "libreyolo[sam]" ``` ## Predict `LibreSAM(...)` (or the family-specific `LibrePicoSAM3(...)`) is a separate entry point from `LibreYOLO(...)`: it returns a promptable segmenter rather than a detector, because a forward pass here is meaningless without a prompt. There is no `libreyolo predict` CLI command for this family; use the Python API. **Box prompt** ```python from libreyolo import LibreSAM, SAMPLE_IMAGE # PicoSAM3 has a single size, "pico", so no other alias is needed. model = LibreSAM("picosam3") # bboxes= is the only supported prompt: [x1, y1, x2, y2] or a list of # boxes, one mask per box. Each box is expanded 10%, made square, # clipped to the image and resized to 96x96 before the CNN runs. result = model.predict(SAMPLE_IMAGE, bboxes=[300, 200, 900, 700]) print(result.masks.xy) # polygon per mask print(result.boxes.xyxy) # tight box derived from the mask ``` **Encode once, prompt many** ```python from libreyolo import LibrePicoSAM3, SAMPLE_IMAGE model = LibrePicoSAM3() # set_image() caches the source image; PicoSAM3 runs one full CNN # forward per box, so this saves the image load/decode, not an # encoder pass the way it does for the other SAM families. model.set_image(SAMPLE_IMAGE) a = model.predict(bboxes=[300, 200, 900, 700]) b = model.predict(bboxes=[100, 100, 400, 400]) model.reset_image() ``` PicoSAM3 accepts only `bboxes=`; passing `points=`, `labels=`, `masks=`, `text=`, `multimask=True` or omitting the box to segment everything all raise a clear `ValueError`, since none of those modes exist in the upstream model. `conf` filters by predicted mask quality (IoU), not a detection confidence, and must be between `0.0` and `1.0`. Every mask carries class id `0`, named `"object"`. `train()`, `val()` and `track()` raise `NotImplementedError`; use LibreSAM2 or LibreSAM3 for point, text, mask or segment-everything prompts. See [prediction](/docs/predict) for source types. ## Variants One size, pico, at a fixed 96 px ROI input: PicoSAM3 runs one full CNN forward per box rather than encoding the whole image once. ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Instance segmentation | yes | | | | | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. PicoSAM3 is the only family in the SAM tier that exports: it ships its raw 96x96 ROI CNN to ONNX, `roi_image -> mask_logits`, with no NMS or mask post-processing baked in. The other SAM families raise `NotImplementedError` on `export()`, since their encoder/decoder split has no defined runtime export contract yet. An exported PicoSAM3 graph does not load back through `LibreYOLO()`; run it directly with a runtime such as `onnxruntime`, applying the same 10%-padded square-ROI preprocessing shown above. **Python** ```python from libreyolo import LibrePicoSAM3 model = LibrePicoSAM3() model.export(format="onnx", output_path="LibrePicoSAM3pico.onnx") # opset (default 13) and dynamic (default True, batch axis only) are # the only export arguments this family accepts. ``` **Use the exported file** ```python import numpy as np import onnxruntime as ort # PicoSAM3 exports its raw 96x96 ROI CNN: roi_image -> mask_logits. # There is no LibreYOLO-side pre/postprocessing to reuse here, since # export() is not routed back through LibreYOLO() the way a detector # checkpoint is. session = ort.InferenceSession("LibrePicoSAM3pico.onnx") name = session.get_inputs()[0].name outputs = session.run(None, {name: np.zeros((1, 3, 96, 96), dtype=np.float32)}) for meta, array in zip(session.get_outputs(), outputs): print(meta.name, array.shape) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibrePicoSAM3.pt` | | Instance segmentation | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: PicoSAM3, ETH Zurich - Upstream license: Apache-2.0 - Upstream source: https://github.com/pbonazzi/picosam3 - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so this code and these weights can be used in commercial and closed-source products. It asks you to keep the license text and attribution notices with any copy you redistribute, and it grants a patent license. LibreYOLO carries a native port of the compact ROI CNN rather than vendoring upstream files unmodified, and downloads LibrePicoSAM3pico.pt from the LibreYOLO Hugging Face org, converted with unchanged tensor values from the pinned pietrobonazzi/picosam3 revision af49e4322b6b7cf448499fee5c073d4576f59444 and tagged Apache-2.0 there. PicoSAM3 is distilled from SAM 2.1 and SAM 3 as teacher models; LibreYOLO does not vendor or redistribute either teacher's code or weights in this family. PicoSAM3 is distilled from SAM 2.1 and SAM 3 as teacher models. LibreYOLO does not vendor or redistribute either teacher's code or weights in this family; only the compact student CNN and its converted checkpoint are shipped. ## Citation ```bibtex @article{picosam3_2026, title={PicoSAM3: Real-Time In-Sensor Region-of-Interest Segmentation}, author={Pietro Bonazzi and Nicola Farronato and Stefan Zihlmann and Haotong Qin and Michele Magno}, journal={IEEE Sensors Journal}, year={2026} } ``` Copied from https://github.com/pbonazzi/picosam3#readme --- # PIDNet A three-branch semantic segmentation network that adds a dedicated boundary branch to a proportional-integral-derivative-inspired design, aimed at real-time inference. LibreYOLO ships it for semantic segmentation only. Tasks: semantic. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install PIDNet needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. The `-sem` filename suffix is required for this family. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibrePIDNets-sem.pt") result = model(SAMPLE_IMAGE, save=True) mask = result.semantic_mask print(mask.data.shape) # (H, W) class ids print(mask.classes) # sorted class ids present in the image ``` **CLI** ```bash libreyolo predict model=LibrePIDNets-sem.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` Semantic segmentation returns one class id per pixel, not boxes, so `result.semantic_mask` carries a `(H, W)` array on `.data` and the list of class ids present in the image on `.classes`. `conf`, `iou` and `max_det` are accepted for API parity but have no effect: the model assigns a class to every pixel by argmax, with no confidence threshold or NMS step. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Three sizes, all at a fixed 1024 px input. The published checkpoints are conversions of the official PIDNet Cityscapes weights, 19 classes. LibreYOLO does not train PIDNet: `train()` raises `NotImplementedError` for this family, which the [support tier](/docs/models) above marks as inference only. ## Validate `val()` returns `metrics/mIoU` and `metrics/pixel_accuracy`, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibrePIDNets-sem.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mIoU"]) print(metrics["metrics/pixel_accuracy"]) ``` **CLI** ```bash libreyolo val model=LibrePIDNets-sem.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | semantic | yes | yes | yes | yes | yes | | | | yes | yes | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibrePIDNets-sem.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibrePIDNets-sem.pt format=onnx libreyolo export model=LibrePIDNets-sem.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibrePIDNets-sem.onnx") result = model(SAMPLE_IMAGE) print(result.semantic_mask.data.shape) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibrePIDNets-sem.pt` | 1024 | semantic | mit | | `LibrePIDNetm-sem.pt` | 1024 | semantic | mit | | `LibrePIDNetl-sem.pt` | 1024 | semantic | mit | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: PIDNet, Jiacong Xu - Upstream license: MIT - Upstream source: https://github.com/XuJiacong/PIDNet - LibreYOLO code: MIT - Weights: MIT, republished at https://huggingface.co/LibreYOLO - Interpretation: MIT is a permissive license, so this code and these weights can be used in commercial and closed-source products. It asks only that you keep the copyright and license notice with any copy you redistribute. LibreYOLO's checkpoints are conversions of the official PIDNet Cityscapes weights, which upstream licenses as MIT. The Cityscapes dataset itself carries separate research-oriented terms and is not redistributed by LibreYOLO. ## Citation ```bibtex @misc{xu2022pidnet, title={PIDNet: A Real-time Semantic Segmentation Network Inspired from PID Controller}, author={Jiacong Xu and Zixiang Xiong and Shankar P. Bhattacharyya}, year={2022}, eprint={2206.02066}, archivePrefix={arXiv}, primaryClass={cs.CV} } ``` Copied from https://github.com/XuJiacong/PIDNet#bibtex-citation --- # PP-OCRv5 PP-OCRv5 is PaddleOCR's text detection and recognition pipeline: a differentiable-binarization detector locates text quads and an SVTR/CTC recognizer reads them. LibreYOLO ports it to PyTorch for two tiers. Tasks: ocr. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install PP-OCRv5 needs no extra beyond the base package. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibrePPOCRl-ocr.pt") result = model(SAMPLE_IMAGE, save=True) for text, conf in zip(result.ocr.texts, result.ocr.conf): print(text, float(conf)) ``` **CLI** ```bash libreyolo predict model=LibrePPOCRl-ocr.pt source=receipt.jpg save=True ``` **Quads** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibrePPOCRl-ocr.pt") result = model(SAMPLE_IMAGE) # (N, 4, 2) polygons in reading order: top-left, top-right, # bottom-right, bottom-left. Detection quads are genuine polygons # (rotated text), so they populate result.ocr, not result.boxes. print(result.ocr.data.shape) print(result.ocr.det_conf) ``` Each checkpoint bundles both stages, detection and recognition, under one `.pt` file, with the recognition charset and pipeline defaults carried in the checkpoint metadata. The recognizer reads Simplified and Traditional Chinese, English, Japanese and pinyin with one dictionary. `result.ocr` is an `OCRRegions` payload: `.data` holds the four-point polygons, `.texts` the transcripts, `.conf` the per-region recognition score, and `.det_conf` the detection score. Multi-image sources run sequentially: the two-stage pipeline does not batch across images. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Two tiers: `t`, built on lighter PP-LCNetV3/PP-OCRv5_mobile backbones for CPU use, and `l`, built on PP-HGNetV2 server backbones for higher accuracy. Both tiers run detection at a fixed long-side limit and recognize crops in batches; `rec_batch` controls how many crops go through the recognizer per forward pass. ## Validate `val()` measures the pipeline against a directory of images plus a `labels/.jsonl` file, or the equivalent dataset YAML, each label listing per-image text-region polygons and their transcripts. It reports detection hmean (IoU-matched precision/recall/F1), end-to-end F1 (hmean plus an exact transcript match after normalization, the checkpoint's fitness metric), and 1-NED, the mean normalized edit distance over matched pairs. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibrePPOCRl-ocr.pt") metrics = model.val(data="my-dataset") print(metrics["metrics/det_hmean"]) print(metrics["metrics/e2e_f1"]) # headline metric print(metrics["metrics/rec_1-NED"]) ``` **CLI** ```bash libreyolo val model=LibrePPOCRl-ocr.pt data=my-dataset ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | ocr | | | | | | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. PP-OCRv5 is a two-network pipeline, detection and recognition moving together, not one traceable graph, and export is not implemented for it: no format is supported yet. Fine-tune the Apache-2.0 upstream training code directly and convert the result with `weights/convert_ppocr_weights.py` if you need a checkpoint outside this format. ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibrePPOCRt-ocr.pt` | | ocr | apache-2.0 | | `LibrePPOCRl-ocr.pt` | | ocr | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: PaddleOCR (PP-OCRv5), PaddlePaddle Authors - Upstream license: Apache-2.0 - Upstream source: https://github.com/PaddlePaddle/PaddleOCR - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license: the architecture, the officially released PP-OCRv5 checkpoints this port converts, and LibreYOLO's own PyTorch port may all be used, modified and redistributed, including commercially, provided the license text and attribution notices travel with any copy. It grants a patent license and places no obligation on your own application code. ## Citation ```bibtex @misc{cui2025paddleocr30technicalreport, title={PaddleOCR 3.0 Technical Report}, author={Cheng Cui and Ting Sun and Manhui Lin and Tingquan Gao and Yubo Zhang and Jiaxuan Liu and Xueqing Wang and Zelun Zhang and Changda Zhou and Hongen Liu and Yue Zhang and Wenyu Lv and Kui Huang and Yichao Zhang and Jing Zhang and Jun Zhang and Yi Liu and Dianhai Yu and Yanjun Ma}, year={2025}, eprint={2507.05595}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2507.05595}, } ``` Copied from https://github.com/PaddlePaddle/PaddleOCR#citation --- # Qwen3-VL Qwen3-VL is Alibaba's vision-language model with native 2D grounding. LibreYOLO wraps it as an open-vocabulary object detector and exposes its free-form chat directly: supply a class list to detect, or ask it a question. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install Qwen3-VL belongs to LibreYOLO's VLM-as-detector tier, a separate product surface from the checkpoint-based families with its own factory. It needs the `vlm` extra. ```bash pip install "libreyolo[vlm]" ``` ## Predict Weights download from Hugging Face on first use and are cached locally. `LibreVLM()` called with no argument defaults to Qwen3-VL-4B. **Python** ```python from libreyolo import LibreVLM, SAMPLE_IMAGE model = LibreVLM("qwen3-vl-4b") model.set_classes(["forklift", "pallet", "safety vest"]) result = model.predict(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **Chat** ```python from libreyolo import LibreVLM, SAMPLE_IMAGE model = LibreVLM("qwen3-vl-4b") # The escape hatch beneath the detection convenience: any question, # not just a bounding-box query. answer = model.chat(SAMPLE_IMAGE, "How many people are wearing a safety vest?") print(answer) ``` This family loads through the `LibreVLM()` factory, not `LibreYOLO()`: VLM families declare no checkpoint loader, so the file-suffix routing described on other model pages does not apply here. `set_classes()` sets the vocabulary Qwen3-VL is asked to find; it is sticky, so it stays in effect across every later `predict()`/`track()` call until you set it again. Every detection carries the same placeholder confidence, so `conf` filtering is all-or-nothing rather than a ranking; `iou` does have an effect for this family, dropping a later same-class box once it overlaps an already-kept one past the threshold, since a repeating generator can otherwise emit near-duplicate boxes for one object. Unlike Florence-2 and Kosmos-2, Qwen3-VL also answers free-form questions through `chat()`, the same escape hatch documented on the `LibreVLM` factory. LibreYOLO's CLI does not cover this tier: there is no `libreyolo predict model=...` form for it. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Three sizes: Qwen3-VL-2B-Instruct, Qwen3-VL-4B-Instruct and Qwen3-VL-8B-Instruct, loaded as `LibreVLM("qwen3-vl-2b")`, `LibreVLM("qwen3-vl-4b")` and `LibreVLM("qwen3-vl-8b")`. All three declare a nominal 1024 px input, but the Qwen processor's own smart-resize decides the actual canvas passed to the network, so that figure is not a fixed operating resolution the way it is for the other families on this site. LibreYOLO has not published a benchmark comparing accuracy across the three sizes. LibreYOLO does not train, validate or export Qwen3-VL: `train()`, `val()` and `export()` all raise `NotImplementedError` for every family in this tier (see the support tier above). Fine-tune Qwen3-VL upstream and load the resulting weights if you need a custom vocabulary baked in; check `predict()` output by eye instead of a COCO-style validation pass, since every detection carries the same placeholder confidence. ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: Qwen3-VL, Alibaba (Qwen Team) - Upstream license: Apache-2.0 - Upstream source: https://github.com/QwenLM/Qwen3-VL - LibreYOLO code: MIT - Weights: Apache-2.0, distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. All three sizes LibreYOLO downloads, Qwen3-VL-2B-Instruct, Qwen3-VL-4B-Instruct and Qwen3-VL-8B-Instruct, carry this license on their Hugging Face repository. ## Citation ```bibtex @article{Qwen3-VL, title={Qwen3-VL Technical Report}, author={Shuai Bai and Yuxuan Cai and Ruizhe Chen and Keqin Chen and Xionghui Chen and Zesen Cheng and Lianghao Deng and Wei Ding and Chang Gao and Chunjiang Ge and Wenbin Ge and Zhifang Guo and Qidong Huang and Jie Huang and Fei Huang and Binyuan Hui and Shutong Jiang and Zhaohai Li and Mingsheng Li and Mei Li and Kaixin Li and Zicheng Lin and Junyang Lin and Xuejing Liu and Jiawei Liu and Chenglong Liu and Yang Liu and Dayiheng Liu and Shixuan Liu and Dunjie Lu and Ruilin Luo and Chenxu Lv and Rui Men and Lingchen Meng and Xuancheng Ren and Xingzhang Ren and Sibo Song and Yuchong Sun and Jun Tang and Jianhong Tu and Jianqiang Wan and Peng Wang and Pengfei Wang and Qiuyue Wang and Yuxuan Wang and Tianbao Xie and Yiheng Xu and Haiyang Xu and Jin Xu and Zhibo Yang and Mingkun Yang and Jianxin Yang and An Yang and Bowen Yu and Fei Zhang and Hang Zhang and Xi Zhang and Bo Zheng and Humen Zhong and Jingren Zhou and Fan Zhou and Jing Zhou and Yuanzhi Zhu and Ke Zhu}, journal={arXiv preprint arXiv:2511.21631}, year={2025} } ``` Copied from https://github.com/QwenLM/Qwen3-VL#citation --- # Real-ESRGAN A practical blind super-resolution upscaler trained on synthetic degradations rather than only bicubic downscaling. LibreYOLO ships inference and validation for its 4x, 2x and fast 4x checkpoints. Tasks: restore. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install Real-ESRGAN needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreRealESRGANx4-restore.pt") result = model(SAMPLE_IMAGE, save=True) restored = result.restored print(restored.array.shape, restored.array.dtype) ``` **CLI** ```bash libreyolo predict model=LibreRealESRGANx4-restore.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **Tiled, for large images** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRealESRGANx4-restore.pt") # tile splits the forward pass into overlapping tiles and blends the # seams back together; tile_pad is the halo added around each tile # before it is cropped back out. Both are Python-only keyword # arguments, not CLI flags. result = model("large-photo.jpg", tile=512, tile_pad=10, save=True) ``` A restore result carries no boxes; `result.restored` is a dense `(H, W, 3)` uint8 RGB image, on a canvas `Results.restore_scale` times the input in each dimension. `save=True` writes that image directly rather than an annotated plot. Input is converted to RGB and any alpha channel is dropped. A source larger than memory allows can be split with `tile` and `tile_pad`, which blend the tile seams back together in the output. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Three checkpoints, named for their upscale factor. `x4` is RRDBNet (`RealESRGAN_x4plus`), 23 residual-in-residual dense blocks, the quality default at 4x. `x2` is the same RRDBNet architecture at 2x. `x4t` is SRVGGNetCompact (`realesr-general-x4v3`), a smaller, faster generator built for video and lower-latency use at 4x. The upstream general-purpose model also ships a paired denoise-strength network blended in at inference time; that strength knob is not part of this port, which runs the base `x4t` generator. ## Validate `val()` measures PSNR and SSIM between the restored output and a clean target image, both computed in RGB on the original canvas with no border crop and no resizing. SSIM uses an 11x11 Gaussian window with sigma 1.5, averaged over the three color channels. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRealESRGANx4-restore.pt") metrics = model.val(data="my-restore-dataset.yaml") print(metrics["metrics/PSNR"]) print(metrics["metrics/SSIM"]) ``` **CLI** ```bash libreyolo val model=LibreRealESRGANx4-restore.pt data=my-restore-dataset.yaml ``` The dataset argument is a YAML pairing a directory of degraded input images with a directory of clean target images of matching resolution; see [dataset formats](/docs/reference/dataset-formats) for the exact keys. ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | restore | yes | yes | yes | yes | yes | | | | yes | yes | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts and the extras a few of them add. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRealESRGANx4-restore.pt") # imgsz defaults to a small internal patch size when omitted, not # your working resolution, so pass the size your deployment actually # feeds the model. model.export(format="onnx", imgsz=512) model.export(format="tensorrt", imgsz=512, half=True) ``` **CLI** ```bash libreyolo export model=LibreRealESRGANx4-restore.pt format=onnx imgsz=512 ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreRealESRGANx4-restore.onnx") result = model(SAMPLE_IMAGE) print(result.restored.array.shape) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreRealESRGANx4t-restore.pt` | | restore | bsd-3-clause | | `LibreRealESRGANx4-restore.pt` | | restore | bsd-3-clause | | `LibreRealESRGANx2-restore.pt` | | restore | bsd-3-clause | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: Real-ESRGAN, Tencent ARC Lab and Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences - Upstream license: BSD-3-Clause - Upstream source: https://github.com/xinntao/Real-ESRGAN - LibreYOLO code: Apache-2.0 and BSD-3-Clause - Weights: BSD-3-Clause, republished at https://huggingface.co/LibreYOLO - Interpretation: BSD-3-Clause is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep the copyright notice, the condition list and the disclaimer with any copy you redistribute, in source or compiled form, and its third clause forbids using the names of Xintao Wang or the project's contributors to endorse or promote a derived product without separate written permission. It carries no patent grant, unlike Apache-2.0. LibreYOLO's checkpoints are format conversions of the official pretrained generators, with the learned parameters unchanged; Real-ESRGAN's training is a GAN over a synthetic degradation pipeline that is not wired into this library, so there is no LibreYOLO-trained variant to license separately. ## Citation ```bibtex @InProceedings{wang2021realesrgan, author = {Xintao Wang and Liangbin Xie and Chao Dong and Ying Shan}, title = {Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data}, booktitle = {International Conference on Computer Vision Workshops (ICCVW)}, date = {2021} } ``` Copied from https://github.com/xinntao/Real-ESRGAN#bibtex --- # ResNet ResNet is an image classifier built from residual blocks, skip connections that let a network add many more layers without the accuracy loss deep plain convolutional stacks otherwise suffer. LibreYOLO supports it for one task: classification. Tasks: classify. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install ResNet needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreResNet50-cls.pt") result = model(SAMPLE_IMAGE, save=True) print(result.probs.top1, result.probs.top1conf) print(result.probs.top5) ``` **CLI** ```bash libreyolo predict model=LibreResNet50-cls.pt source=cat.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different model is a one line change. A classifier carries no boxes or masks: `result.probs` holds the whole-image prediction, with `top1`, `top5`, `top1conf` and `top5conf`. `conf`, `iou` and `max_det` are accepted for API parity but have no effect, since there is nothing to threshold or suppress on a single probability vector. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Four depths, all trained and evaluated the same way, so picking one is a straight parameter-count-for-accuracy trade. The task is fixed: every size covers classification only. The weights filename ends `-cls.pt` on every size, and that suffix is what the factory reads to route to this family; no `task=` argument is needed. ## Train Fine-tuning starts from the published ImageNet backbone and rebuilds the final classifier layer to the target dataset's class count automatically. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreResNet50-cls.pt") model.train(data="imagenette160", epochs=5) ``` **CLI** ```bash libreyolo train model=LibreResNet50-cls.pt data=imagenette160 epochs=5 ``` **Multi-GPU** ```bash libreyolo train model=LibreResNet50-cls.pt data=imagenette160 \ epochs=50 device=0,1 batch=-1 ``` Left alone, the trainer runs 100 epochs at `lr0=1e-3` with AdamW, a batch of 64 and early stopping after 50 epochs without improvement. `data` accepts a dataset root (`train/` and `val/`, one folder per class), a known short name such as `imagenette160`, or a `.zip` URL. `lora=True` is not supported here; passing it raises, since LoRA in LibreYOLO targets transformer components with `nn.Linear` layers and ResNet has none. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary of `metrics/` keys. For classification that is top-1 and top-5 accuracy over the validation split. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreResNet50-cls.pt") metrics = model.val(data="imagenette160") print(metrics["metrics/accuracy_top1"]) print(metrics["metrics/accuracy_top5"]) ``` **CLI** ```bash libreyolo val model=LibreResNet50-cls.pt data=imagenette160 ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | classify | yes | yes | yes | yes | yes | | | | yes | yes | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts and the extras a few of them add. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreResNet50-cls.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreResNet50-cls.pt format=onnx libreyolo export model=LibreResNet50-cls.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreResNet50-cls.onnx") result = model(SAMPLE_IMAGE) print(result.probs.top1) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreResNet18-cls.pt` | 224 | classify | apache-2.0 | | `LibreResNet34-cls.pt` | 224 | classify | apache-2.0 | | `LibreResNet50-cls.pt` | 224 | classify | apache-2.0 | | `LibreResNet101-cls.pt` | 224 | classify | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: ResNet, Microsoft Research Asia - Upstream license: Apache-2.0 - Upstream source: https://github.com/huggingface/pytorch-image-models - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours. The architecture is the original Microsoft Research Asia design; the pretrained weights LibreYOLO ships are timm's resnet{18,34,50,101}.a1_in1k reproduction (the "ResNet Strikes Back" A1 recipe), trained by Ross Wightman and the timm contributors on ImageNet-1k and licensed Apache-2.0 there. LibreYOLO's module and attribute names mirror timm so the state dict loads unchanged and inference matches bit-identically. ## Citation ```bibtex @article{He2015, author = {Kaiming He and Xiangyu Zhang and Shaoqing Ren and Jian Sun}, title = {Deep Residual Learning for Image Recognition}, journal = {arXiv preprint arXiv:1512.03385}, year = {2015} } ``` Copied from https://github.com/KaimingHe/deep-residual-networks#citation --- # RetinaNet RetinaNet is a one-stage detector trained with focal loss, which down-weights easy negatives so a dense grid of anchors no longer needs a separate proposal stage to stay accurate. LibreYOLO ports the torchvision implementation for detection. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install RetinaNet needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreRetinaNetr50v2.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreRetinaNetr50v2.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. `conf` and `iou` set the confidence and NMS thresholds; RetinaNet keeps its upstream NMS step over the dense anchor grid. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Two sizes, both ResNet-50 with a feature pyramid: `r50` is the original head, and `r50v2` replaces it with a GroupNorm head and a wider P6 block fed from the backbone's last stage instead of the FPN output. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRetinaNetr50v2.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreRetinaNetr50v2.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | | | | | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. RetinaNet exports to ONNX only, at batch size 1. RetinaNet resizes to a variable, aspect-preserved input, so LibreYOLO forces `dynamic=True` regardless of what is passed, to keep the graph valid for sources of different shapes. An exported `.onnx` file loads back through `LibreYOLO()` on its file suffix and returns the same `Results`. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRetinaNetr50v2.pt") model.export(format="onnx", imgsz=800) ``` **CLI** ```bash libreyolo export model=LibreRetinaNetr50v2.pt format=onnx imgsz=800 ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreRetinaNetr50v2.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreRetinaNetr50.pt` | 800 | Detection | bsd-3-clause | | `LibreRetinaNetr50v2.pt` | 800 | Detection | bsd-3-clause | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: RetinaNet, PyTorch - Upstream license: BSD-3-Clause - Upstream source: https://github.com/pytorch/vision - LibreYOLO code: BSD-3-Clause - Weights: BSD-3-Clause, republished at https://huggingface.co/LibreYOLO - Interpretation: BSD-3-Clause is a permissive license, so this code can be used in commercial and closed-source products with no obligation on your own application code. It asks only that you keep the copyright notice and disclaimer with any copy you redistribute, and it carries no patent grant. The two published checkpoints used for parity testing are not distributed in the LibreYOLO source tree: torchvision's own documentation notes that a pretrained model's terms may depend on its training data, so each Hugging Face mirror ships the BSD text on that implied basis and repeats the caveat rather than issuing an explicit checkpoint-specific grant. --- # RF-DETR A detection transformer that predicts a fixed set of objects instead of a dense grid, so it needs no NMS at inference. LibreYOLO supports it for four tasks. Tasks: Detection, Instance segmentation, Pose, Oriented boxes. Install: pip install "libreyolo[rfdetr]". Verified against LibreYOLO v1.5.0. ## Install RF-DETR needs its own extra, which pulls in `transformers` for the backbone. ```bash pip install "libreyolo[rfdetr]" ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreRFDETRs.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreRFDETRs.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **Video** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRFDETRs.pt") # Any source the library accepts: file, folder, URL, webcam index, # RTSP stream, or a .streams list for result in model.predict("clip.mp4", stream=True, save=True): print(len(result.boxes)) ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. `conf` and `max_det` filter the query selection; there is no NMS step to tune. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Four sizes, and four tasks that share one architecture: segmentation, pose and oriented boxes reuse the detection decoder with a different head, so they take the same arguments. The sizes carry similar parameter counts and differ mainly in input resolution. | Checkpoint | Input (px) | mAP 50-95 | Params (M) | | --- | --- | --- | --- | | `LibreRFDETRn` | 384 | 51.4 | 30.47 | | `LibreRFDETRs` | 512 | 55.1 | 32.11 | | `LibreRFDETRm` | 576 | 57.4 | 33.69 | | `LibreRFDETRl` | 704 | 58.6 | 33.93 | COCO val2017, 500 images. Published on https://www.visionanalysis.org/ Interactive chart: https://www.visionanalysis.org/embed/scatter?highlight=rfdetr-n%2Crfdetr-s%2Crfdetr-m%2Crfdetr-l%2Crfdetr-seg-n&title=RF-DETR%20on%20COCO&subtitle=Accuracy%20against%20latency%2C%20every%20benchmarked%20model ## Train Training starts from a published checkpoint, for all four tasks. RF-DETR lists `pretrained` among the arguments its native trainer ignores, so passing `pretrained=False` does not give you a randomly initialized model here. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRFDETRs.pt") model.train(data="my-dataset.yaml", epochs=50, imgsz=512, batch=8, lr0=1e-4) ``` **CLI** ```bash libreyolo train model=LibreRFDETRs.pt data=my-dataset.yaml \ epochs=50 imgsz=512 batch=8 lr0=1e-4 ``` **LoRA** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRFDETRs.pt") model.train(data="my-dataset.yaml", epochs=50, lora=True) ``` **Multi-GPU** ```bash libreyolo train model=LibreRFDETRs.pt data=my-dataset.yaml \ epochs=50 device=0,1 batch=-1 ``` Two arguments matter more here than on a CNN detector. Keep `lr0` at or below `1e-4`, since transformer detectors diverge at learning rates a YOLO model tolerates. Leave `imgsz` at the checkpoint's native resolution unless you have a reason to change it. The input must divide evenly by the backbone patch size times the window count; LibreYOLO checks this before the run starts and names the nearest valid sizes. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRFDETRs.pt") # val() returns a plain dict, not an object metrics = model.val(data="my-dataset.yaml", imgsz=512) print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) print(metrics["metrics/precision"], metrics["metrics/recall"]) ``` **CLI** ```bash libreyolo val model=LibreRFDETRs.pt data=my-dataset.yaml imgsz=512 ``` **Against COCO** ```bash # The bundled COCO yaml carries an embedded download script, so it # needs explicit permission unless the dataset is already local. libreyolo val model=LibreRFDETRn.pt data=coco.yaml imgsz=384 \ allow_download_scripts=True ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | | yes | | | | yes | yes | | Instance segmentation | yes | yes | yes | yes | yes | | | | | | | | | Pose | yes | yes | yes | yes | yes | | | | | | | | | Oriented boxes | yes | yes | yes | yes | yes | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Running the graph in a bare runtime, with no LibreYOLO installed, is also supported, but then preprocessing and postprocessing are yours to write. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRFDETRs.pt") model.export(format="onnx", imgsz=512) model.export(format="tensorrt", imgsz=512, half=True) # Arguments accepted for every format: # # format "onnx" | "torchscript" | "executorch" | "tensorrt" # | "openvino" | "paddle" | "mnn" | "rknn" | "ncnn" # | "tflite" | "coreml" | "coreai". # "engine" is an alias for tensorrt, "litert" for tflite. # imgsz int, or (height, width). Defaults to the checkpoint's # native resolution. # batch int, default 1. # half bool, export in FP16. Default False. # int8 bool, export in INT8. Default False. Needs `data`. # data path to a dataset YAML, used to calibrate int8. # fraction float, share of that calibration set to use. Default 1.0. # dynamic bool, dynamic axes. Default True. # simplify bool, run ONNX graph simplification. Default True. # opset int, ONNX opset. Chosen per family when not given. # device str, device to trace on. Defaults to the model's device. # output_path str, defaults to a name derived from the checkpoint. # verbose bool, default False. # allow_download_scripts bool, default False. Permits embedded # Python in a dataset YAML that has to be downloaded. # # A few formats take extra arguments of their own, such as an RKNN # target platform. Those are documented on each format's page. ``` **CLI** ```bash libreyolo export model=LibreRFDETRs.pt format=onnx imgsz=512 libreyolo export model=LibreRFDETRs.pt format=tensorrt imgsz=512 half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreRFDETRs.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` **Without LibreYOLO** ```python import numpy as np import onnxruntime as ort # Running the graph directly means doing your own preprocessing and # postprocessing. Inspect the signature before wiring anything up. session = ort.InferenceSession("LibreRFDETRs.onnx") name = session.get_inputs()[0].name outputs = session.run(None, {name: np.zeros((1, 3, 512, 512), dtype=np.float32)}) for meta, array in zip(session.get_outputs(), outputs): print(meta.name, array.shape) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreRFDETRl.pt` | 704 | Detection | apache-2.0 | | `LibreRFDETRm.pt` | 576 | Detection | apache-2.0 | | `LibreRFDETRn.pt` | 384 | Detection | apache-2.0 | | `LibreRFDETRs.pt` | 512 | Detection | apache-2.0 | | `LibreRFDETRn-seg.pt` | 312 | Instance segmentation | apache-2.0 | | `LibreRFDETRs-seg.pt` | 384 | Instance segmentation | apache-2.0 | | `LibreRFDETRm-seg.pt` | 432 | Instance segmentation | apache-2.0 | | `LibreRFDETRl-seg.pt` | 504 | Instance segmentation | apache-2.0 | | `LibreRFDETRx-seg.pt` | 624 | Instance segmentation | apache-2.0 | | `LibreRFDETRxx-seg.pt` | 768 | Instance segmentation | apache-2.0 | | `LibreRFDETRn-obb.pt` | 384 | Oriented boxes | cc-by-4.0 | | `LibreRFDETRn-pose.pt` | | Pose | apache-2.0 | | `LibreRFDETRs-pose.pt` | | Pose | apache-2.0 | | `LibreRFDETRm-pose.pt` | | Pose | apache-2.0 | | `LibreRFDETRl-pose.pt` | | Pose | apache-2.0 | | `LibreRFDETRs-obb.pt` | 512 | Oriented boxes | cc-by-4.0 | | `LibreRFDETRm-obb.pt` | 576 | Oriented boxes | cc-by-4.0 | | `LibreRFDETRl-obb.pt` | 704 | Oriented boxes | cc-by-4.0 | | `LibreRFDETRx-pose.pt` | 576 | Pose | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: RF-DETR, Roboflow - Upstream license: Apache-2.0 - Upstream source: https://github.com/roboflow/rf-detr - LibreYOLO code: MIT - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours. ## Citation ```bibtex @inproceedings{robinson2026rfdetr, title = {RF-DETR: Real-Time Detection Transformer}, author = {Robinson, Isaac and Robicheaux, Peter and Popov, Matvei and Ramanan, Deva and Peri, Neehar}, booktitle = {International Conference on Learning Representations (ICLR)}, year = {2026}, url = {https://arxiv.org/abs/2511.09554} } ``` Copied from https://github.com/roboflow/rf-detr#citation --- # RT-DETR A detection transformer built for real-time inference: it decodes a fixed set of queries rather than a dense grid, so it runs no NMS. LibreYOLO carries three versions of it, told apart by the checkpoint you load, and version 2 also serves oriented boxes. Tasks: Detection, Oriented boxes. Install: pip install "libreyolo[rtdetr]". Verified against LibreYOLO v1.5.0. ## Install RT-DETR needs no optional extra. Everything it imports is in the base install, and the `rtdetr` extra is a stable name that adds nothing to it. ```bash pip install libreyolo ``` Adapter fine-tuning with `lora=True` is the exception, and needs the `lora` extra. ```bash pip install "libreyolo[lora]" ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreRTDETRr18.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreRTDETRr18.pt save=True \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` **Video** ```python from libreyolo import LibreYOLO # The version is part of the file name, and the factory routes on the # checkpoint, so all three load the same way. model = LibreYOLO("LibreRTDETRv4s.pt") # Any source the library accepts: file, folder, URL, webcam index, # RTSP stream, or a .streams list for result in model.predict("clip.mp4", stream=True, save=True): print(len(result.boxes)) ``` **Oriented boxes** ```python from libreyolo import LibreYOLO # Version 2 only. The -obb suffix selects the task, and the checkpoint # is recognized as oriented from its own tensors, so no task argument # is needed. These weights are DOTA v1.0, 15 aerial classes at 1024 px. model = LibreYOLO("LibreRTDETRv2n-obb.pt") result = model("aerial.png", save=True) obb = result.obb print(obb.xywhr) # (N, 5): cx, cy, w, h, radians print(obb.xyxyxyxy) # the same rows as four corner points print(result.boxes.xyxy) # enclosing axis-aligned boxes ``` **Oriented boxes, CLI** ```bash libreyolo predict model=LibreRTDETRv2n-obb.pt source=aerial.png save=True ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. `conf` and `max_det` filter a top-k decode over queries and classes; there is no NMS step to tune, and `iou` is accepted but unused. An oriented checkpoint fills `result.obb` natively and also fills `result.boxes` with the enclosing axis-aligned rectangles. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Three versions, two tasks between them, and the size codes do not run in a single series. Version 1 names its sizes after the backbone, ResNet or HGNetv2. Version 2 reuses the ResNet names only: version 1 already ships the two HGNetv2 sizes, and version 2's results there were close enough that LibreYOLO publishes no duplicate weights for them. Version 4 uses a plain letter series, which collides with version 1's HGNetv2 names, so a size code on its own does not identify a model. The version is written into the checkpoint file name. | Checkpoint | Input (px) | mAP 50-95 | Params (M) | | --- | --- | --- | --- | | `LibreRTDETRl` | 640 | 55.8 | 32.93 | | `LibreRTDETRr101` | 640 | 56.8 | 76.56 | | `LibreRTDETRr18` | 640 | 49.7 | 20.18 | | `LibreRTDETRr34` | 640 | 52.2 | 31.44 | | `LibreRTDETRr50` | 640 | 55.9 | 42.89 | | `LibreRTDETRr50m` | 640 | 53.8 | 36.59 | | `LibreRTDETRx` | 640 | 57.9 | 67.37 | | `LibreRTDETRv2r101` | 640 | 56.8 | 76.56 | | `LibreRTDETRv2r18` | 640 | 50.8 | 20.18 | | `LibreRTDETRv2r34` | 640 | 53.2 | 31.44 | | `LibreRTDETRv2r50` | 640 | 55.7 | 42.89 | | `LibreRTDETRv2r50m` | 640 | 54.8 | 36.59 | | `LibreRTDETRv4l` | 640 | 57.8 | 31.24 | | `LibreRTDETRv4m` | 640 | 56.5 | 19.59 | | `LibreRTDETRv4s` | 640 | 52.8 | 10.32 | | `LibreRTDETRv4x` | 640 | 60.0 | 62.62 | COCO val2017, 500 images. Published on https://www.visionanalysis.org/ Interactive chart: https://www.visionanalysis.org/embed/scatter?highlight=rtdetr-l%2Crtdetr-r101%2Crtdetr-r18%2Crtdetr-r34%2Crtdetr-r50%2Crtdetr-r50m%2Crtdetr-x%2Crtdetrv2-r101%2Crtdetrv2-r18%2Crtdetrv2-r34%2Crtdetrv2-r50%2Crtdetrv2-r50m%2Crtdetrv4-l%2Crtdetrv4-m%2Crtdetrv4-s%2Crtdetrv4-x&title=RT-DETR%20on%20COCO&subtitle=Accuracy%20against%20latency%2C%20every%20benchmarked%20model Version 2 keeps version 1's architecture and state dict layout and changes how the deformable attention samples, which is why the two are told apart by the metadata in the checkpoint rather than by shape. Version 4 is a different lineage: it reuses D-FINE's architecture and trainer, and its weights come from distilling a DINOv3 vision foundation model teacher into an HGNetv2 student. In LibreYOLO `LibreRTDETRv4` is a subclass of `LibreDFINE` with the mask head pinned off, so it stays detection only. ### Oriented boxes on version 2 Version 2 is the one version that carries a second task. Its supported tasks are `detect` and `obb`, and the two do not share a graph or a size series. Detection uses the ResNet sizes at 640 px; oriented detection uses an HGNetv2 series, n, s, m, l and x, at 1024 px, and the input size resolves per task rather than per family. A checkpoint is recognized as oriented from its own tensors, by the five-coordinate box heads and the version 2 sampling parameters, so `-obb` weights load into the oriented graph without a `task` argument and a mismatch between the two is a hard error rather than a silent reinterpretation. The published files are `LibreRTDETRv2n-obb.pt` through `LibreRTDETRv2x-obb.pt`. They are the official DOTA v1.0 single-scale checkpoints converted into LibreYOLO's format, 15 aerial classes from plane and ship through harbor and helicopter, and their class names are stamped into the checkpoint. Unlike the detection side, the oriented task is inference only: prediction, validation and export work, and `train()` on an oriented model raises. Tracking and test-time augmentation do not support oriented boxes either. [Oriented detection](/docs/tasks/oriented-detection) covers the task, the label format and the metrics. ## Train Training starts from a published checkpoint. `pretrained` is accepted and then dropped on all three versions, so `pretrained=False` does not give you a randomly initialized model. Everything in this section is about detection: version 2's oriented task is inference only, and there is no transfer path from detection weights to it, because the two use different backbones. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRTDETRr18.pt") # coco128.yaml downloads a 128-image sample on first use. Point `data` # at your own dataset YAML for a real run. model.train(data="coco128.yaml", epochs=50, batch=4, lr0=1e-4) ``` **CLI** ```bash libreyolo train model=LibreRTDETRr18.pt data=coco128.yaml \ epochs=50 batch=4 lr0=1e-4 ``` **LoRA** ```python # Needs the lora extra: pip install "libreyolo[lora]" from libreyolo import LibreYOLO model = LibreYOLO("LibreRTDETRr18.pt") model.train(data="coco128.yaml", epochs=50, lora=True) ``` **Multi-GPU** ```bash libreyolo train model=LibreRTDETRr18.pt data=coco128.yaml \ epochs=50 device=0,1 ``` Learning rate is the argument to get right, and each version carries its own default rather than the library-wide one. The Python `train()` signature reads it from that version's training config, and the CLI resolves the same value when `lr0` is not passed. Versions 1 and 2 also take `lr_backbone` and default it to a twentieth of `lr0`, following the original recipe; version 4 runs through the D-FINE trainer, which scales the backbone parameter group with `backbone_lr_mult` instead. Leave `imgsz` at the checkpoint's native size unless you have a reason to change it. Validation and prediction at other sizes work, with one residual: a rectangular size whose token count matches the native size still reuses an embedding built for the wrong aspect ratio. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRTDETRr18.pt") # val() returns a plain dict, not an object metrics = model.val(data="coco128.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) print(metrics["metrics/precision"], metrics["metrics/recall"]) ``` **CLI** ```bash libreyolo val model=LibreRTDETRr18.pt data=coco128.yaml ``` **Against COCO** ```bash # coco-val-only.yaml fetches the 5000 val2017 images and skips the # training set. It carries an embedded download script, so it needs # explicit permission unless the dataset is already local. libreyolo val model=LibreRTDETRr18.pt data=coco-val-only.yaml \ allow_download_scripts=True ``` **Oriented boxes** ```python from libreyolo import LibreYOLO # Oriented validation matches with rotated IoU, so a prediction in the # right place at the wrong angle counts as a miss. model = LibreYOLO("LibreRTDETRv2n-obb.pt") metrics = model.val(data="my-obb-dataset.yaml") print(metrics["metrics/mAP50-95(OBB)"]) print(metrics["metrics/mAP50(OBB)"]) ``` The rows in the benchmark table above come from the LibreYOLO benchmark harness; the note under that table records which dataset produced them and links the run records. Oriented validation runs through the same call and reports the same keys, plus four repeated under an `(OBB)` suffix. Matching uses rotated IoU rather than the IoU of the enclosing rectangles, so an angle error is a miss. `augment=True` is rejected on this task. ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | | yes | | | | | yes | | Oriented boxes | yes | yes | yes | yes | yes | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. The matrix covers the lineage as one page: where the three versions disagree about a format, the cell shows the weakest of the three, so nothing here is oversold for whichever version you load. The oriented row belongs to version 2 alone. ONNX and TorchScript are validated there, at FP32, batch 1 and a fixed 1024 by 1024 canvas; OpenVINO, TensorRT and ExecuTorch convert and reload but have not met raw-output parity across the full query set, so the top boxes agree to a fraction of a pixel while the tail drifts. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. **Python** ```python # Needs the onnx extra: pip install "libreyolo[onnx]" from libreyolo import LibreYOLO model = LibreYOLO("LibreRTDETRr18.pt") path = model.export(format="onnx") print(path) ``` **CLI** ```bash libreyolo export model=LibreRTDETRr18.pt format=onnx ``` **Oriented boxes** ```bash # ONNX and TorchScript are the validated targets for the oriented task, # at FP32, batch 1, on a fixed 1024 by 1024 canvas. libreyolo export model=LibreRTDETRv2n-obb.pt format=onnx imgsz=1024 libreyolo export model=LibreRTDETRv2n-obb.pt format=torchscript imgsz=1024 ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreRTDETRr18.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreRTDETRr34.pt` | 640 | Detection | apache-2.0 | | `LibreRTDETRr18.pt` | 640 | Detection | apache-2.0 | | `LibreRTDETRr50.pt` | 640 | Detection | apache-2.0 | | `LibreRTDETRr50m.pt` | 640 | Detection | apache-2.0 | | `LibreRTDETRr101.pt` | 640 | Detection | apache-2.0 | | `LibreRTDETRl.pt` | 640 | Detection | apache-2.0 | | `LibreRTDETRx.pt` | 640 | Detection | apache-2.0 | | `LibreRTDETRv2r18.pt` | 640 | Detection | apache-2.0 | | `LibreRTDETRv2r34.pt` | 640 | Detection | apache-2.0 | | `LibreRTDETRv2r50m.pt` | 640 | Detection | apache-2.0 | | `LibreRTDETRv2r50.pt` | 640 | Detection | apache-2.0 | | `LibreRTDETRv2r101.pt` | 640 | Detection | apache-2.0 | | `LibreRTDETRv4s.pt` | 640 | Detection | apache-2.0 | | `LibreRTDETRv4m.pt` | 640 | Detection | apache-2.0 | | `LibreRTDETRv4l.pt` | 640 | Detection | apache-2.0 | | `LibreRTDETRv4x.pt` | 640 | Detection | apache-2.0 | | `LibreRTDETRv2n-obb.pt` | 1024 | Oriented boxes | apache-2.0 | | `LibreRTDETRv2s-obb.pt` | 1024 | Oriented boxes | apache-2.0 | | `LibreRTDETRv2m-obb.pt` | 1024 | Oriented boxes | apache-2.0 | | `LibreRTDETRv2l-obb.pt` | 1024 | Oriented boxes | apache-2.0 | | `LibreRTDETRv2x-obb.pt` | 1024 | Oriented boxes | apache-2.0 | The file name carries the version, then the size, then the task. Detection weights are `LibreRTDETR.pt`, `LibreRTDETRv2.pt` and `LibreRTDETRv4.pt`, all at 640 px. Oriented weights exist for version 2 only and add the task suffix, `LibreRTDETRv2n-obb.pt` through `LibreRTDETRv2x-obb.pt`, all at 1024 px and trained on DOTA v1.0 rather than COCO. ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: RT-DETR, RT-DETRv2 and RT-DETRv4, Baidu (versions 1 and 2), Peking University and Tsinghua University (version 4) - Upstream license: Apache-2.0 - Upstream source: https://github.com/lyuwenyu/RT-DETR - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours. All three versions carry it, across two repositories and three papers: RT-DETR and RT-DETRv2 at github.com/lyuwenyu/RT-DETR, and RT-DETRv4 at github.com/RT-DETRs/RT-DETRv4, which is cited separately (arXiv 2510.25257). RT-DETRv4 distills from a DINOv3 teacher while training only; the released student weights hold no DINOv3 parameters, and the tensors that fed the teacher are dropped when a checkpoint is converted, so Meta's DINOv3 license does not reach them. ## Citation ```bibtex @misc{lv2023detrs, title={DETRs Beat YOLOs on Real-time Object Detection}, author={Yian Zhao and Wenyu Lv and Shangliang Xu and Jinman Wei and Guanzhong Wang and Qingqing Dang and Yi Liu and Jie Chen}, year={2023}, eprint={2304.08069}, archivePrefix={arXiv}, primaryClass={cs.CV} } @misc{lv2024rtdetrv2improvedbaselinebagoffreebies, title={RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer}, author={Wenyu Lv and Yian Zhao and Qinyao Chang and Kui Huang and Guanzhong Wang and Yi Liu}, year={2024}, eprint={2407.17140}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2407.17140}, } ``` Copied from https://github.com/lyuwenyu/RT-DETR#citation The block above is what the authors publish for versions 1 and 2 detection. Version 2's oriented weights have a third upstream, the Apache-2.0 RiO-DETR repository at [github.com/RicePasteM/RiO-DETR](https://github.com/RicePasteM/RiO-DETR), which is where the DOTA checkpoints come from; cite that project if you used one. Version 4 is a separate paper by a different group and has its own citation block at [github.com/RT-DETRs/RT-DETRv4](https://github.com/RT-DETRs/RT-DETRv4#4-citation); cite that one if you used a version 4 checkpoint. --- # RTMDet RTMDet is a single-stage detector that predicts from one point-based prior per grid location, no anchors, through a head whose convolutions are shared across feature levels. LibreYOLO supports it for detection and RTMDet-Ins instance segmentation. Tasks: Detection, Instance segmentation. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install RTMDet needs no extra beyond the base package. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreRTMDets.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreRTMDets.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **Instance segmentation** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The -seg suffix in the filename selects the RTMDet-Ins mask head, # so no task argument is needed here. model = LibreYOLO("LibreRTMDets-seg.pt") result = model(SAMPLE_IMAGE, save=True) print(result.masks.data.shape) ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. A `-seg` filename resolves to the RTMDet-Ins task on its own, and `result.masks` then carries the instance masks alongside the boxes. `conf` sets the confidence threshold and `iou` the NMS threshold. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Five sizes, `t` through `x`, share one architecture at a common input resolution. This family carries no benchmark table here: compare sizes by checkpoint file size in the table below. ## Train **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRTMDets.pt") model.train( data="my-dataset.yaml", epochs=300, imgsz=640, batch=16, lr0=0.004, ) ``` **CLI** ```bash libreyolo train model=LibreRTMDets.pt data=my-dataset.yaml imgsz=640 epochs=300 batch=16 lr0=0.004 ``` Detection trains through `train()`. The QualityFocalLoss, GIoU and DynamicSoftLabelAssigner components are ported from upstream mmdetection, and the forward pass and ONNX export are bit-equivalent to it, with postprocessing matching mmdet's output within 0.001 mAP on val2017 subsets. What has not been checked, per `train()`'s own docstring: small-dataset fine-tune convergence, from-scratch paper parity, multi-GPU behavior, cached Mosaic and MixUp throughput, the strict upstream two-stage pipeline switch, and the paramwise weight-decay overrides that zero decay on norm and bias parameters. RTMDet-Ins has no training path. Calling `train()` on a `-seg` checkpoint, or with `task="segment"`, raises `NotImplementedError`; instance segmentation supports inference and validation only. `train()` also accepts a `pretrained` argument, but the value is never read inside the method: training always continues from whatever weights the model was constructed with, so `pretrained=False` does not reinitialize the network. Left alone otherwise, the trainer runs 300 epochs with AdamW at `lr0=0.004` and `weight_decay=0.05`, a 1-epoch warmup on a cosine schedule, and Mosaic and MixUp switched off for the final 20 epochs. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRTMDets.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreRTMDets.pt data=my-dataset.yaml ``` **Instance segmentation** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRTMDets-seg.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95(M)"]) # masks print(metrics["metrics/mAP50-95(B)"]) # boxes ``` Against a `-seg` checkpoint the plain `metrics/mAP50-95` key holds the mask score, and the same run also reports boxes under `(B)` and masks under `(M)` so both are available from one pass. ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | | | | | | | yes | | Instance segmentation | | | | | | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. Detection exports to most formats; instance segmentation currently exports to none of them; the matrix above reflects that split. An exported detection artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Running the graph in a bare runtime, with no LibreYOLO installed, is also supported, but then preprocessing and postprocessing are yours to write. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRTMDets.pt") model.export(format="onnx", imgsz=640) model.export(format="tensorrt", imgsz=640, half=True) ``` **CLI** ```bash libreyolo export model=LibreRTMDets.pt format=onnx imgsz=640 libreyolo export model=LibreRTMDets.pt format=tensorrt imgsz=640 half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreRTMDets.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreRTMDett.pt` | 640 | Detection | apache-2.0 | | `LibreRTMDets.pt` | 640 | Detection | apache-2.0 | | `LibreRTMDetm.pt` | 640 | Detection | apache-2.0 | | `LibreRTMDetl.pt` | 640 | Detection | apache-2.0 | | `LibreRTMDetx.pt` | 640 | Detection | apache-2.0 | | `LibreRTMDett-seg.pt` | 640 | Instance segmentation | apache-2.0 | | `LibreRTMDets-seg.pt` | 640 | Instance segmentation | apache-2.0 | | `LibreRTMDetm-seg.pt` | 640 | Instance segmentation | apache-2.0 | | `LibreRTMDetl-seg.pt` | 640 | Instance segmentation | apache-2.0 | | `LibreRTMDetx-seg.pt` | 640 | Instance segmentation | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: RTMDet, OpenMMLab - Upstream license: Apache-2.0 - Upstream source: https://github.com/open-mmlab/mmdetection - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours. The published RTMDet and RTMDet-Ins checkpoints are converted from mmdetection's own COCO weights, trained by OpenMMLab under the same license. ## Citation ```bibtex @misc{lyu2022rtmdet, title={RTMDet: An Empirical Study of Designing Real-Time Object Detectors}, author={Chengqi Lyu and Wenwei Zhang and Haian Huang and Yue Zhou and Yudong Wang and Yanyi Liu and Shilong Zhang and Kai Chen}, year={2022}, eprint={2212.07784}, archivePrefix={arXiv}, primaryClass={cs.CV} } ``` Copied from https://github.com/open-mmlab/mmdetection/tree/main/configs/rtmdet#citation --- # SAM SAM (Segment Anything) turns a point or box click into an object mask. LibreYOLO loads it through a dedicated LibreSAM factory, separate from the LibreYOLO() detector factory, because a promptable model needs a different call shape. Tasks: Instance segmentation. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install SAM needs the `sam` extra, which pulls in `transformers` and `timm`. ```bash pip install "libreyolo[sam]" ``` ## Predict `LibreSAM(...)` is a separate entry point from `LibreYOLO(...)`: it returns a promptable segmenter rather than a detector, because a forward pass here is meaningless without a spatial prompt. There is no `libreyolo predict` CLI command for this family; use the Python API. **Point and box prompts** ```python from libreyolo import LibreSAM, SAMPLE_IMAGE # "base" autodownloads facebook/sam-vit-base on first use. # Other sizes: "large", "huge" (also "b"/"l"/"h"). model = LibreSAM("base") # A point prompt: [x, y] in pixel coordinates, label 1 = foreground. result = model.predict(SAMPLE_IMAGE, points=[640, 420], labels=[1]) print(result.masks.xy) # polygon per mask print(result.boxes.xyxy) # tight box derived from the mask # A box prompt instead of a point. result = model.predict(SAMPLE_IMAGE, bboxes=[300, 200, 900, 700]) # No prompt at all segments the whole image (a simplified automatic # mask generator, not the exhaustive reference one). result = model.predict(SAMPLE_IMAGE) ``` **Encode once, prompt many** ```python from libreyolo import LibreSAM, SAMPLE_IMAGE model = LibreSAM("base") # The image encoder is the expensive part. set_image() runs it once; # every predict() call after that reuses the cached embedding. model.set_image(SAMPLE_IMAGE) a = model.predict(points=[640, 420], labels=[1]) b = model.predict(bboxes=[300, 200, 900, 700]) model.reset_image() ``` A point prompt accepts `[x, y]` for one object, `[[x, y], ...]` for several, or numpy arrays; `labels` marks each point `1` (foreground) or `0` (background) and defaults to all foreground. A box prompt takes `[x1, y1, x2, y2]` or a list of boxes, one mask per box. Omitting both prompts segments the whole image by prompting a dense grid and keeping the confident, non-overlapping masks; this "segment everything" mode is simplified against the reference automatic mask generator and can under-segment crowded scenes, so a real point or box prompt is the precise path. `conf` filters by predicted mask quality (IoU), not a detection confidence: pass `0.0` to keep every candidate. `multimask=True` returns all three of SAM's whole-versus-part ambiguity masks per prompt instead of the single best one. `device=` moves the model and, if a `set_image()` session is active, its cached embedding. Every mask carries class id `0`, named `"object"`, since a promptable mask has no fixed class set. `train()`, `val()`, `export()` and `track()` all raise `NotImplementedError` for this family: SAM is predict-only in LibreYOLO, and video tracking is out of scope. See [prediction](/docs/predict) for source types. ## Variants Three ViT image-encoder sizes: base, large and huge, all at a fixed 1024 px input. No accuracy or latency benchmark is published for this family yet, so choosing a size trades encoder weight for mask quality directly: base is the fastest to encode, huge the heaviest. ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: SAM (Segment Anything), Meta AI Research (FAIR) - Upstream license: Apache-2.0 - Upstream source: https://github.com/facebookresearch/segment-anything - LibreYOLO code: MIT - Weights: Apache-2.0, distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: Apache-2.0 is a permissive license, so this code and these weights can be used in commercial and closed-source products. It asks you to keep the license text and attribution notices with any copy you redistribute, and it grants a patent license. LibreYOLO does not mirror SAM-1 weights on its own Hugging Face org: LibreSAM downloads the base, large and huge checkpoints directly from Meta's own facebook/sam-vit-base, facebook/sam-vit-large and facebook/sam-vit-huge repositories, each tagged Apache-2.0 there as well. LibreYOLO does not host its own copy of the SAM-1 weights. `LibreSAM("base")`, `"large"` and `"huge"` download straight from Meta's own `facebook/sam-vit-base`, `facebook/sam-vit-large` and `facebook/sam-vit-huge` repositories on Hugging Face, each tagged Apache-2.0 there independently of LibreYOLO. ## Citation ```bibtex @article{kirillov2023segany, title={Segment Anything}, author={Kirillov, Alexander and Mintun, Eric and Ravi, Nikhila and Mao, Hanzi and Rolland, Chloe and Gustafson, Laura and Xiao, Tete and Whitehead, Spencer and Berg, Alexander C. and Lo, Wan-Yen and Doll{\'a}r, Piotr and Girshick, Ross}, journal={arXiv:2304.02643}, year={2023} } ``` Copied from https://github.com/facebookresearch/segment-anything#citing-segment-anything --- # SAM 2 SAM 2 extends SAM with a streaming-memory architecture built for video, and turns a point or box click into an object mask. LibreYOLO supports its image segmentation path through a dedicated LibreSAM factory, separate from the LibreYOLO() detector factory. Tasks: Instance segmentation. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install SAM 2 needs the `sam` extra, which pulls in `transformers` and `timm`. ```bash pip install "libreyolo[sam]" ``` ## Predict `LibreSAM(...)` (or the family-specific `LibreSAM2(...)`) is a separate entry point from `LibreYOLO(...)`: it returns a promptable segmenter rather than a detector, because a forward pass here is meaningless without a spatial prompt. There is no `libreyolo predict` CLI command for this family; use the Python API. Only image segmentation is supported; SAM 2's video-memory tracking is out of scope here. **Point and box prompts** ```python from libreyolo import LibreSAM, SAMPLE_IMAGE # Size aliases: "sam2-tiny", "sam2-small", "sam2-base-plus", # "sam2-large" (also the short forms "sam2-t"/"sam2-s"/"sam2-bp"/"sam2-l"). model = LibreSAM("sam2-large") # A point prompt: [x, y] in pixel coordinates, label 1 = foreground. result = model.predict(SAMPLE_IMAGE, points=[640, 420], labels=[1]) print(result.masks.xy) # polygon per mask print(result.boxes.xyxy) # tight box derived from the mask # A box prompt instead of a point. result = model.predict(SAMPLE_IMAGE, bboxes=[300, 200, 900, 700]) # No prompt at all segments the whole image (a simplified automatic # mask generator, not the exhaustive reference one). result = model.predict(SAMPLE_IMAGE) ``` **Encode once, prompt many** ```python from libreyolo import LibreSAM2, SAMPLE_IMAGE # The family-specific class takes the size without the "sam2-" prefix. model = LibreSAM2("large") # The image encoder is the expensive part. set_image() runs it once; # every predict() call after that reuses the cached embedding. model.set_image(SAMPLE_IMAGE) a = model.predict(points=[640, 420], labels=[1]) b = model.predict(bboxes=[300, 200, 900, 700]) model.reset_image() ``` A point prompt accepts `[x, y]` for one object, `[[x, y], ...]` for several, or numpy arrays; `labels` marks each point `1` (foreground) or `0` (background) and defaults to all foreground. A box prompt takes `[x1, y1, x2, y2]` or a list of boxes, one mask per box. Omitting both prompts segments the whole image by prompting a dense grid and keeping the confident, non-overlapping masks; this "segment everything" mode is simplified against the reference automatic mask generator and can under-segment crowded scenes, so a real point or box prompt is the precise path. `conf` filters by predicted mask quality (IoU), not a detection confidence: pass `0.0` to keep every candidate. `multimask=True` returns all three of SAM's whole-versus-part ambiguity masks per prompt instead of the single best one. `device=` moves the model and, if a `set_image()` session is active, its cached embedding. Every mask carries class id `0`, named `"object"`, since a promptable mask has no fixed class set. `train()`, `val()`, `export()` and `track()` all raise `NotImplementedError` for this family: image inference is what LibreYOLO supports here. See [prediction](/docs/predict) for source types. ## Variants Four Hiera-backbone sizes: tiny, small, base-plus and large, all at the same input resolution. No accuracy or latency benchmark is published for this family yet, so choosing a size trades encoder weight for mask quality directly: tiny is the fastest to encode, large the heaviest. ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreSAM2tiny.pt` | | Instance segmentation | apache-2.0 | | `LibreSAM2small.pt` | | Instance segmentation | apache-2.0 | | `LibreSAM2base-plus.pt` | | Instance segmentation | apache-2.0 | | `LibreSAM2large.pt` | | Instance segmentation | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: SAM 2, Meta FAIR - Upstream license: Apache-2.0 - Upstream source: https://github.com/facebookresearch/sam2 - LibreYOLO code: MIT - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so this code and these weights can be used in commercial and closed-source products. It asks you to keep the license text and attribution notices with any copy you redistribute, and it grants a patent license. LibreYOLO loads SAM 2 through the Apache-2.0 Transformers implementation and republishes Transformers-compatible snapshots of the tiny, small, base-plus and large checkpoints on its own Hugging Face org, tagged Apache-2.0 there as well. LibreYOLO ships image inference only; SAM 2's video-memory tracking is out of scope for this family. ## Citation ```bibtex @article{ravi2024sam2, title={SAM 2: Segment Anything in Images and Videos}, author={Ravi, Nikhila and Gabeur, Valentin and Hu, Yuan-Ting and Hu, Ronghang and Ryali, Chaitanya and Ma, Tengyu and Khedr, Haitham and R{\"a}dle, Roman and Rolland, Chloe and Gustafson, Laura and Mintun, Eric and Pan, Junting and Alwala, Kalyan Vasudev and Carion, Nicolas and Wu, Chao-Yuan and Girshick, Ross and Doll{\'a}r, Piotr and Feichtenhofer, Christoph}, journal={arXiv preprint arXiv:2408.00714}, url={https://arxiv.org/abs/2408.00714}, year={2024} } ``` Copied from https://github.com/facebookresearch/sam2#citing-sam-2 --- # SAM 3 SAM 3 extends SAM with a text-concept prompt on top of the usual points and boxes, so a phrase like "yellow school bus" returns every matching instance. LibreYOLO supports its image path through a dedicated LibreSAM factory, separate from the LibreYOLO() detector factory. Tasks: Instance segmentation. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install SAM 3 needs the `sam` extra, which pulls in `transformers` and `timm`. ```bash pip install "libreyolo[sam]" ``` The weights are gated: visit [huggingface.co/facebook/sam3](https://huggingface.co/facebook/sam3), accept Meta's SAM License, then run `hf auth login` (or set `HF_TOKEN`) before the first download. LibreYOLO logs a license notice the first time it downloads this family. ## Predict `LibreSAM(...)` (or the family-specific `LibreSAM3(...)`) is a separate entry point from `LibreYOLO(...)`: it returns a promptable segmenter rather than a detector, because a forward pass here is meaningless without a prompt. There is no `libreyolo predict` CLI command for this family; use the Python API. Only image inference is supported; SAM 3's video models are out of scope here. **Point and box prompts** ```python from libreyolo import LibreSAM, SAMPLE_IMAGE # "sam3" is the only size ("large"); aliases: "sam3", "sam-3", "sam3-large". model = LibreSAM("sam3") # A point prompt: [x, y] in pixel coordinates, label 1 = foreground. result = model.predict(SAMPLE_IMAGE, points=[640, 420], labels=[1]) print(result.masks.xy) # polygon per mask print(result.boxes.xyxy) # tight box derived from the mask # A box prompt instead of a point. result = model.predict(SAMPLE_IMAGE, bboxes=[300, 200, 900, 700]) ``` **Text (concept) prompt** ```python from libreyolo import LibreSAM3, SAMPLE_IMAGE model = LibreSAM3("large") # Finds every instance matching the phrase, not just one object. # text= is mutually exclusive with points, bboxes, labels and masks. result = model.predict(SAMPLE_IMAGE, text="a person") print(result.names) # {0: "a person"} print(result.boxes.conf) # the PCS detection score per instance ``` **Encode once, prompt many** ```python from libreyolo import LibreSAM3, SAMPLE_IMAGE model = LibreSAM3("large") # The image encoder is the expensive part. set_image() runs it once; # every predict() call after that reuses the cached embedding. A # text= call re-encodes internally, since the tracker and the # concept-segmentation encoder do not share a cache. model.set_image(SAMPLE_IMAGE) a = model.predict(points=[640, 420], labels=[1]) b = model.predict(bboxes=[300, 200, 900, 700]) model.reset_image() ``` The point and box path matches the rest of the SAM family: a point prompt accepts `[x, y]` for one object or `[[x, y], ...]` for several, `labels` marks each point `1` (foreground) or `0` (background), and a box prompt takes `[x1, y1, x2, y2]` or a list of boxes. `conf` on this path filters by predicted mask quality (IoU), not a detection confidence. The `text=` path is SAM 3's addition: a concept string returns every matching instance in the image through Promptable Concept Segmentation, and cannot be combined with points, boxes, labels or masks. `conf` there is the PCS detection score instead of mask IoU; leaving it at the default applies the model's own 0.3 threshold, and `conf=0.0` keeps every candidate. The returned `names` maps class id `0` to the requested concept string, since a promptable mask has no fixed class set otherwise. `device=` moves the model and, if a `set_image()` session is active, its cached embedding. `train()`, `val()`, `export()` and `track()` all raise `NotImplementedError` for this family: SAM 3 is predict-only in LibreYOLO, and video tracking is out of scope. See [prediction](/docs/predict) for source types. ## Variants One size, large, at a fixed 1008 px input. SAM 3.1 is not supported: its implementation carries a custom license that cannot be vendored into this MIT repository, and the Transformers version LibreYOLO depends on does not yet load its checkpoint format. ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: SAM 3, Meta FAIR - Upstream license: SAM License (Meta custom, gated) - Upstream source: https://github.com/facebookresearch/sam3 - LibreYOLO code: MIT - Weights: SAM License (Meta custom, gated), distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: The SAM License is Meta's own agreement, not an OSI-approved license like MIT or Apache-2.0. It grants a worldwide, royalty-free license to use, reproduce, distribute and create derivative works, and does not itself say the weights are non-commercial. It does add terms Apache and MIT do not: publications that use the results must acknowledge SAM Materials, reverse engineering is prohibited, use is subject to export-control and sanctions compliance, filing IP litigation against Meta over the materials terminates your license, and Meta may amend the agreement. Weights are gated on facebook/sam3 on Hugging Face: you must accept the terms there and authenticate before downloading. LibreYOLO calls SAM 3 through the Apache-2.0 Transformers implementation and vendors none of Meta's model source, but does not redistribute the weights themselves; read the license yourself before relying on it commercially. LibreYOLO does not host its own copy of the SAM 3 weights and does not redistribute them. `LibreSAM("sam3")` downloads directly from Meta's gated `facebook/sam3` repository on Hugging Face, which requires accepting Meta's SAM License and authenticating before the first download. ## Citation ```bibtex @misc{carion2025sam3segmentconcepts, title={SAM 3: Segment Anything with Concepts}, author={Nicolas Carion and Laura Gustafson and Yuan-Ting Hu and Shoubhik Debnath and Ronghang Hu and Didac Suris and Chaitanya Ryali and Kalyan Vasudev Alwala and Haitham Khedr and Andrew Huang and Jie Lei and Tengyu Ma and Baishan Guo and Arpit Kalla and Markus Marks and Joseph Greer and Meng Wang and Peize Sun and Roman Rädle and Triantafyllos Afouras and Effrosyni Mavroudi and Katherine Xu and Tsung-Han Wu and Yu Zhou and Liliane Momeni and Rishi Hazra and Shuangrui Ding and Sagar Vaze and Francois Porcher and Feng Li and Siyuan Li and Aishwarya Kamath and Ho Kei Cheng and Piotr Dollár and Nikhila Ravi and Kate Saenko and Pengchuan Zhang and Christoph Feichtenhofer}, year={2025}, eprint={2511.16719}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2511.16719}, } ``` Copied from https://github.com/facebookresearch/sam3#citing-sam-3 --- # SAM 3D Body SAM 3D Body is Meta's promptable model for recovering a full-body 3D mesh, including hands and feet, from a single image and person boxes. LibreYOLO wraps the upstream package rather than porting it. Tasks: mesh. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install ```bash pip install libreyolo ``` That gives you LibreYOLO's adapter only. SAM 3D Body itself is not bundled, because its license is not one LibreYOLO's own code may be derived from: clone the upstream repository and install its dependencies yourself, then point LibreYOLO at the clone. ```bash git clone https://github.com/facebookresearch/sam-3d-body pip install roma einops yacs omegaconf braceexpand pytorch-lightning timm ``` ```python from libreyolo.models.sam3dbody import LibreSAM3DBody model = LibreSAM3DBody( None, size="d3", sam_3d_body_path="/path/to/sam-3d-body", device="cuda", ) ``` or set the `SAM_3D_BODY_PATH` environment variable instead of passing `sam_3d_body_path` on every call. A user who never constructs this family never triggers the import, and never encounters the SAM License. This family is not wired into the `LibreYOLO()` factory or the `libreyolo predict` CLI command; `LibreSAM3DBody` is the only entry point. ## Predict **Python** ```python from libreyolo import SAMPLE_IMAGE from libreyolo.models.sam3dbody import LibreSAM3DBody # This family is not registered with the LibreYOLO() factory, so it # is constructed directly. model_path=None is what triggers the # gated Hugging Face download; a string is instead treated as an # existing local checkpoint path and is never fetched automatically. # Inference requires a CUDA device; there is no CPU path. model = LibreSAM3DBody(None, size="d3", device="cuda") result = model(SAMPLE_IMAGE, person_boxes=[[34, 12, 220, 400]]) meshes = result.meshes print(meshes.vertices.shape) # (N, V, 3), camera frame, meters print(meshes.joints3d.shape) # (N, J, 3) ``` **With a person detector** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE from libreyolo.models.sam3dbody import LibreSAM3DBody # No named-string shortcut here: pass a constructed LibreYOLO # detector, a plain callable, or a PersonDetector instance. detector = LibreYOLO("LibreRFDETRn.pt") model = LibreSAM3DBody(None, size="d3", device="cuda") result = model(SAMPLE_IMAGE, person_detector=detector) ``` The checkpoint download is gated: it requires accepting Meta's license on the Hugging Face model page and authenticating with `hf auth login` before the first download succeeds. Inference itself needs a CUDA device unconditionally: the upstream estimator moves its batch to the GPU without checking, so a CPU-only machine raises rather than falling back. `result.meshes` is a `Meshes` payload, row-aligned with `result.boxes` (one row per detected person): `vertices` and `joints3d` are metric and already include the estimated camera translation, `joints2d` is in pixels on the original image, and rotations follow MHR's convention, Euler angles rather than axis-angle. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Two backbones behind the same MHR body model: `d3` uses a DINOv3 ViT-H/16+ encoder, and `h` uses the original ViT-H encoder. ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | mesh | | | | | | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. Body-mesh export is not implemented: LibreYOLO has not yet defined an exported-graph contract for the mesh task, including how to represent the MHR parameter layout outside PyTorch. ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreSAM3DBodyd3-mesh.pt` | 512 | mesh | other | | `LibreSAM3DBodyh-mesh.pt` | 512 | mesh | other | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: SAM 3D Body, Meta Platforms, Inc. - Upstream license: SAM License (not OSI-approved) - Upstream source: https://github.com/facebookresearch/sam-3d-body - LibreYOLO code: MIT - Weights: SAM License (not OSI-approved), republished at https://huggingface.co/LibreYOLO - Interpretation: The SAM License is not one OSI recognizes as open source. Meta can amend its terms unilaterally, disclaims all warranty and liability, forbids reverse engineering the SAM Materials, and forbids any use touching military or warfare purposes, nuclear applications, espionage, weapons, or other export-controlled ends. It does not forbid commercial use outright, but a litigation clause ends your license the moment you sue Meta over the SAM Materials, so read it yourself before shipping a product on it. LibreYOLO wraps Meta's package rather than porting its code: none of it is vendored, it is an optional dependency the user installs themselves, and a user who never touches the mesh task never encounters these terms. The checkpoints are mirrored byte-identically behind the same Hugging Face access gate Meta uses, so accepting the license happens on Meta's own model page. LibreYOLO's own adapter code is MIT. The body model the checkpoints drive, MHR (Momentum Human Rig), is Meta's separate Apache-2.0 release and is fetched at runtime from its own public repository rather than mirrored. The body model the checkpoints drive, MHR (Momentum Human Rig), is a separate Meta release under Apache-2.0. LibreYOLO fetches its TorchScript asset from MHR's own public release at runtime and caches it locally; that file is not mirrored by LibreYOLO and carries its own Apache-2.0 terms, not the SAM License. ## Citation ```bibtex @article{yang2026sam3dbody, title={SAM 3D Body: Robust Full-Body Human Mesh Recovery}, author={Yang, Xitong and Kukreja, Devansh and Pinkus, Don and Sagar, Anushka and Fan, Taosha and Park, Jinhyung and Shin, Soyong and Cao, Jinkun and Liu, Jiawei and Ugrinovic, Nicolas and Feiszli, Matt and Malik, Jitendra and Dollar, Piotr and Kitani, Kris}, journal={arXiv preprint arXiv:2602.15989}, year={2026} } ``` Copied from https://github.com/facebookresearch/sam-3d-body#citing-sam-3d-body --- # SegFormer SegFormer is a semantic segmentation transformer that pairs a hierarchical Mix Transformer (MiT) encoder with a lightweight all-MLP decode head, avoiding the heavy decoders and fixed positional encodings earlier segmentation transformers needed. LibreYOLO supports it for one task, semantic segmentation, across six sizes. Tasks: semantic. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install SegFormer needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreSegformerb0-sem.pt") result = model(SAMPLE_IMAGE, save=True) mask = result.semantic_mask print(mask.data.shape, mask.classes) ``` **CLI** ```bash libreyolo predict model=LibreSegformerb0-sem.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` `result.semantic_mask` carries the dense class map: `.data` is an `(H, W)` tensor of class IDs on the original image size, and `.classes` lists the class IDs actually present. `result.boxes` is `None`, since there are no per-instance detections. `conf` and `iou` are accepted for API parity but do not change the output: the model returns one class per pixel, not per-instance detections to filter or de-duplicate. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Six sizes, b0 through b5, widening and deepening the Mix Transformer encoder at each step while keeping the same all-MLP decode head design. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreSegformerb0-sem.pt` | | semantic | other | | `LibreSegformerb1-sem.pt` | | semantic | other | | `LibreSegformerb2-sem.pt` | | semantic | other | | `LibreSegformerb3-sem.pt` | | semantic | other | | `LibreSegformerb4-sem.pt` | | semantic | other | | `LibreSegformerb5-sem.pt` | | semantic | other | ## Train `train()` fine-tunes a published checkpoint by default. Pass no `model_path` to `LibreSegformer(...)` instead and it builds with a randomly initialized encoder and head, training from scratch, the only route to weights that carry none of the pretrained checkpoints' non-commercial restriction (see [Licensing](#licensing)). **Python (fine-tune)** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreSegformerb0-sem.pt") model.train(data="my-dataset.yaml", epochs=160, imgsz=512, batch=8) ``` **CLI** ```bash libreyolo train model=LibreSegformerb0-sem.pt data=my-dataset.yaml \ epochs=160 imgsz=512 batch=8 ``` **From scratch** ```python from libreyolo.models.segformer.model import LibreSegformer # No model_path: random init, nothing downloaded. The only route to # weights free of the pretrained checkpoints' non-commercial term. model = LibreSegformer(size="b0", nb_classes=150) model.train(data="my-dataset.yaml", epochs=160, imgsz=512, batch=8) ``` **Multi-GPU** ```bash libreyolo train model=LibreSegformerb0-sem.pt data=my-dataset.yaml \ epochs=160 device=0,1 batch=16 ``` Left alone, the trainer follows the SegFormer paper's ADE20K recipe: AdamW at a backbone base learning rate with the decode head trained at 10x that rate, weight decay everywhere except LayerNorm and the Mix-FFN positional convolution, and a linear decay schedule with a warmup. Convergence for the larger sizes, b3 through b5, has not been validated end to end. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary of `metrics/` keys: mIoU and pixel accuracy, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreSegformerb0-sem.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mIoU"]) print(metrics["metrics/pixel_accuracy"]) ``` **CLI** ```bash libreyolo val model=LibreSegformerb0-sem.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | semantic | yes | yes | yes | yes | yes | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreSegformerb0-sem.pt") model.export(format="onnx", imgsz=512) model.export(format="tensorrt", imgsz=512, half=True) ``` **CLI** ```bash libreyolo export model=LibreSegformerb0-sem.pt format=onnx imgsz=512 libreyolo export model=LibreSegformerb0-sem.pt format=tensorrt imgsz=512 half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreSegformerb0-sem.onnx") result = model(SAMPLE_IMAGE) print(result.semantic_mask.data.shape) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreSegformerb0-sem.pt` | | semantic | other | | `LibreSegformerb1-sem.pt` | | semantic | other | | `LibreSegformerb2-sem.pt` | | semantic | other | | `LibreSegformerb3-sem.pt` | | semantic | other | | `LibreSegformerb4-sem.pt` | | semantic | other | | `LibreSegformerb5-sem.pt` | | semantic | other | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: SegFormer, NVIDIA - Upstream license: NVIDIA Source Code License (non-commercial, research or evaluation only) - Upstream source: https://github.com/NVlabs/SegFormer - LibreYOLO code: Apache-2.0 - Weights: NVIDIA Source Code License (non-commercial, research or evaluation only), republished at https://huggingface.co/LibreYOLO - Interpretation: The pretrained ADE20K checkpoints LibreYOLO hosts for this family are converted from NVIDIA's official SegFormer release under the NVIDIA Source Code License. That license permits redistributing the weights and derivative works, provided the license text and attribution notices travel with them, but it limits USE to non-commercial research or evaluation, a restriction its Section 3.2 carries forward into every derivative and that cannot be relicensed away: these weights are NOT covered by LibreYOLO's normal permissive terms, and that limitation binds you, not just LibreYOLO. LibreYOLO's own SegFormer implementation is a separate Apache-2.0 port of Hugging Face Transformers' code, unrelated to NVIDIA's repository, so a model you train from scratch with LibreSegformer(...).train(...) carries none of this restriction. LibreSegformer's encoder and decode head are a PyTorch port of Hugging Face Transformers' Apache-2.0 SegFormer implementation, not of NVlabs/SegFormer: NVIDIA's original repository was never read or copied, and is credited here only for attribution to the paper's authors. Only the pretrained checkpoints above carry NVIDIA's non-commercial restriction; the architecture and LibreYOLO's own code stay MIT throughout. ## Citation ```bibtex @inproceedings{xie2021segformer, title={SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers}, author={Xie, Enze and Wang, Wenhai and Yu, Zhiding and Anandkumar, Anima and Alvarez, Jose M and Luo, Ping}, booktitle={Neural Information Processing Systems (NeurIPS)}, year={2021} } ``` Copied from https://github.com/NVlabs/SegFormer#citation --- # SenseNova-Vision SenseNova-Vision is a unified multimodal model that casts vision tasks as prompted generation on a shared decoder: boxes, points, keypoints and OCR words come out as tagged text, and depth, mask and panoptic maps come out as images a decoder renders. LibreYOLO loads it through LibreVLM and supports seven tasks from the one 7B checkpoint. Tasks: Detection, Instance segmentation, panoptic, Pose, point, depth, ocr. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install SenseNova-Vision needs its own extra, which pulls in `accelerate` for the big-model dispatch this checkpoint needs and, on non-macOS platforms, `bitsandbytes` for 4-bit loading. ```bash pip install "libreyolo[sensenova]" ``` The checkpoint is mirrored on Hugging Face under LibreYOLO's own org and downloads automatically on first use; it is CC BY-NC 4.0, non-commercial use only, and the loader prints that notice before every automatic download. See Licensing below. ## Predict **Python** ```python from libreyolo import LibreVLM model = LibreVLM("sensenova-vision", task="detect") model.set_classes(["bird", "boat"]) result = model.predict("image.jpg") print(result.boxes.xyxy) # set_task() switches tasks on the same loaded model. model.set_task("depth") result = model.predict("image.jpg") depth = result.depth_map.data ``` **Referring segmentation and panoptic** ```python from libreyolo import LibreVLM model = LibreVLM("sensenova-vision", task="segment") # Segmentation is referring: it needs a target phrase, not a class list. model.set_classes(["the person furthest to the right"]) result = model.predict("street.jpg") mask = result.masks.data[0] model.set_task("panoptic") # With no custom vocabulary, panoptic falls back to the COCO panoptic # categories the checkpoint was tuned on. result = model.predict("street.jpg") segment_map = result.panoptic.data for segment in result.panoptic.segments_info: print(segment) ``` **Points, pose and OCR** ```python from libreyolo import LibreVLM model = LibreVLM("sensenova-vision", task="point") model.set_classes(["screw"]) result = model.predict("board.jpg") print(result.points.xy) # With no vocabulary set, pose falls back to "person". model.set_task("pose") result = model.predict("gym.jpg") print(result.boxes.xyxy, result.keypoints.data.shape) model.set_task("ocr") result = model.predict("sign.jpg") print(result.ocr.texts) ``` Every prediction is a diffusion decode over the shared Bagel-MoT backbone, so it is a capability model rather than a real-time one: expect noticeably higher per-image latency than a purpose-built detector or segmenter. `dtype="auto"` (the default) loads bf16 on a GPU with enough memory and falls back to 4-bit NF4 quantization elsewhere, which needs `bitsandbytes`; pass `dtype="bf16"` to force full precision on a large enough GPU. `noise_seed=42` at construction seeds the diffusion sampler for reproducible dense outputs; pass `noise_seed=None` to disable seeding. The seven tasks share one loaded checkpoint: `set_task()` switches between them without reloading. `set_classes()` sets the active vocabulary; detection, points, pose and panoptic accept a class list, while segmentation is referring and needs exactly the phrase to isolate. Each task returns the standard `Results` object with a different payload populated: `boxes` for detect, `points` for point, `boxes` and `keypoints` for pose, `ocr` for OCR, `depth_map` for depth, `masks` for segment, and `panoptic` (with `segments_info`) for panoptic. See [prediction](/docs/predict) for sources, streaming and result handling. ## Checkpoints | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `SenseNovaVision7b.pt` | 1024 | Detection | cc-by-nc-4.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: SenseNova-Vision, SenseTime (OpenSenseNova) - Upstream license: Apache-2.0 (code); CC BY-NC 4.0 (weights) - Upstream source: https://github.com/OpenSenseNova/SenseNova-Vision - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0 (code); CC BY-NC 4.0 (weights), republished at https://huggingface.co/LibreYOLO - Interpretation: LibreYOLO's SenseNova-Vision port is Apache-2.0 code adapted from SenseTime's OpenSenseNova/SenseNova-Vision release, which is itself built on Apache-2.0 sources: ByteDance's Bagel decoder, Hugging Face Transformers' Qwen2 and SigLIP modules, and Black Forest Labs' FLUX autoencoder. The SenseNova-Vision-7B-MoT checkpoint is a separate artifact under CC BY-NC 4.0, NON-COMMERCIAL USE ONLY. LibreYOLO mirrors it byte-identical, with attribution, at LibreYOLO/SenseNovaVision7b, but mirroring does not change the license: it stays non-commercial, and the loader prints that notice before every automatic download. One upstream file, modeling/bagel/modeling_utils.py, carries an incompatible CC BY-NC 4.0 license inherited from Meta's DiT; LibreYOLO did not port it. The small permissive routines it would have supplied were re-derived independently instead, from Hugging Face Transformers' ViTMAE implementation (Apache-2.0) and OpenAI's guided-diffusion (MIT). ## Citation ```bibtex @misc{sensenova2026sensenovavision, title={Vision as Unified Multimodal Generation}, author={Xiaoyang Han and Jianhua Li and Kewang Deng and Zukai Chen and Xuanke Shi and Sihan Wang and Boxuan Li and Linyan Wang and Siyi Xie and Xin You and Jinsheng Quan and Zhongang Cai and Haiwen Diao and Ziwei Liu and Lei Yang and Dahua Lin and Quan Wang}, year={2026}, eprint={2607.06560}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2607.06560}, } ``` Copied from https://github.com/OpenSenseNova/SenseNova-Vision#citation --- # SigLIP2 SigLIP2 is a dual-tower model that scores an image against text prompts with an independent sigmoid per class, instead of a shared softmax over a fixed label set. LibreYOLO supports it for zero-shot classification and image/text embedding, with no training step. Tasks: classify, embed. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install SigLIP2 needs its own extra, which pulls in the SentencePiece package its multilingual tokenizer uses. ```bash pip install "libreyolo[siglip2]" ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreSigLIP2b16-cls.pt") model.set_classes(["a forklift", "an empty aisle", "a spill"]) result = model(SAMPLE_IMAGE, save=True) print(model.names[result.probs.top1], float(result.probs.top1conf)) ``` **CLI** ```bash # With no set_classes() call, CLI predict uses the 1,000 ImageNet # class names the model loads with by default. libreyolo predict model=LibreSigLIP2b16-cls.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **Multi-label sigmoid scoring** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreSigLIP2b16-cls.pt") model.set_classes(["a dog", "a cat", "outdoors"], multi_label=True) r = model(SAMPLE_IMAGE) # Independent per-class probabilities: more than one, or none, can # score high at once. Softmax (the default) instead normalizes them # into a single-label distribution, matching LibreCLIP's behavior. for i, name in model.names.items(): print(name, float(r.probs.data[i])) ``` **Image and text embedding** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreSigLIP2b16-cls.pt", task="embed") image_embed = model(SAMPLE_IMAGE).embeddings.data text_embed = model.embed_text("a photo of a forklift") # Both are L2-normalized, so a plain dot product is cosine similarity. similarity = (image_embed @ text_embed.T).item() ``` `set_classes()` is the one primitive that makes this an open-vocabulary classifier: it renders each label into every prompt template, encodes and averages the results, and caches the resulting `[K, D]` matrix as the classifier head, so it is not recomputed per image. Call it again to change classes at any time. With no call, LibreSigLIP2 loads with the 1,000 ImageNet-1k class names already set. SigLIP scores each class independently: `logit = scale * (image . text) + bias`. By default that logit set is still passed through a softmax, giving a single-label distribution that matches LibreCLIP's `top1`/`top5` behavior. Passing `multi_label=True` to `set_classes()` (or at construction) switches to independent sigmoid probabilities instead, so more than one class, or none, can score high on the same image. The tokenizer is a multilingual SentencePiece model (Gemma vocabulary), so class names in languages other than English work the same way. With `task="embed"`, prediction returns one L2-normalized image vector per input instead of class probabilities, and `embed_text()` returns normalized text rows in the same vector space, so a plain dot product between them is cosine similarity. `iou` has no effect on either task; there is no NMS step. See [prediction](/docs/predict) for sources, streaming and result handling. ## Validate `val()` reads the class-folder names under an ImageFolder `train/` split, calls `set_classes()` with them, then measures zero-shot top-1 and top-5 accuracy under softmax scoring. Accuracy depends on how the class names read as prompts, not on any weight update, since there is nothing to train. Validation only covers `task="classify"`; `task="embed"` has no dataset validator. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreSigLIP2b16-cls.pt") # data is an ImageFolder root with a train/ split; its folder names # become the zero-shot class prompts for this run. metrics = model.val(data="imagenette160") print(metrics["metrics/accuracy_top1"]) print(metrics["metrics/accuracy_top5"]) ``` **CLI** ```bash libreyolo val model=LibreSigLIP2b16-cls.pt data=imagenette160 ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | classify | yes | yes | yes | yes | yes | | | | | yes | | yes | | embed | yes | yes | yes | yes | yes | | | | | yes | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. Export bakes the model's current state into a fixed graph. For `task="classify"`, whatever labels `set_classes()` last set, and the resolution at export time, are baked into a final linear layer with the learned scale and bias, so the exported graph is an ordinary `[B, K]` image classifier with no text tower and no tokenizer; export again after changing the classes or the size. Exporting in `multi_label=True` mode is not implemented; set it back to `False` first. `task="embed"` export traces the image tower alone. Both need ONNX opset 14 or higher, which the exporter sets by default. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreSigLIP2b16-cls.pt") model.set_classes(["a forklift", "an empty aisle", "a spill"]) model.export(format="onnx") # The current set_classes() labels and the input resolution are baked # into the graph. Re-export after changing either one. multi_label # must be False (the default) at export time. ``` **CLI** ```bash # No set_classes() call here, so this bakes in the default 1,000 # ImageNet classes the model loads with. libreyolo export model=LibreSigLIP2b16-cls.pt format=onnx ``` **Embedding export** ```python from libreyolo import LibreYOLO # task="embed" traces the image tower alone; no classes needed. model = LibreYOLO("LibreSigLIP2b16-cls.pt", task="embed") model.export(format="onnx") ``` ## Checkpoints Every published weight file for this family. Both are converted from Google's Apache-2.0 `siglip2-base-patch16-256` and `siglip2-so400m-patch14-384` checkpoints, not from any COCO training run. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreSigLIP2b16-cls.pt` | | classify | apache-2.0 | | `LibreSigLIP2so400m-cls.pt` | | classify | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: SigLIP 2, Google DeepMind - Upstream license: Apache-2.0 - Upstream source: https://github.com/huggingface/transformers - LibreYOLO code: MIT - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code. LibreSigLIP2's image and text towers are a clean-room re-implementation structured to match Hugging Face Transformers' SigLIP reference implementation, built without transformers as a runtime dependency; the multilingual SentencePiece tokenizer (Gemma vocabulary) is shipped verbatim from the Apache-2.0 google/siglip2-* release. The shipped checkpoints (b16, so400m) are converted from Google's Apache-2.0 siglip2-base-patch16-256 and siglip2-so400m-patch14-384 weights, a metadata wrap with the learned parameters unchanged. Training is not offered for this family: LibreSigLIP2 is zero-shot, and set_classes() replaces the fine-tuning step a trained classifier would otherwise need. ## Citation ```bibtex @misc{tschannen2025siglip2multilingualvisionlanguage, title={SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features}, author={Michael Tschannen and Alexey Gritsenko and Xiao Wang and Muhammad Ferjad Naeem and Ibrahim Alabdulmohsin and Nikhil Parthasarathy and Talfan Evans and Lucas Beyer and Ye Xia and Basil Mustafa and Olivier Hénaff and Jeremiah Harmsen and Andreas Steiner and Xiaohua Zhai}, year={2025}, eprint={2502.14786}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2502.14786}, } ``` Copied from https://huggingface.co/google/siglip2-base-patch16-256#bibtex-entry-and-citation-info --- # SmolVLM2 SmolVLM2 is Hugging Face's small vision-language model. LibreYOLO wraps it as an open-vocabulary object detector and exposes its free-form chat directly: supply a class list to detect, or ask it a question. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install SmolVLM2 belongs to LibreYOLO's VLM-as-detector tier, a separate product surface from the checkpoint-based families with its own factory. It needs the `vlm` extra, which also pulls in `num2words`, a dependency of SmolVLM2's own processor. ```bash pip install "libreyolo[vlm]" ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreVLM, SAMPLE_IMAGE model = LibreVLM("smolvlm2-500m") model.set_classes(["cat", "dog"]) result = model.predict(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **Chat** ```python from libreyolo import LibreVLM, SAMPLE_IMAGE model = LibreVLM("smolvlm2-500m") # The escape hatch beneath the detection convenience: any question, # not just a bounding-box query. answer = model.chat(SAMPLE_IMAGE, "What is the cat doing?") print(answer) ``` This family loads through the `LibreVLM()` factory, not `LibreYOLO()`: VLM families declare no checkpoint loader, so the file-suffix routing described on other model pages does not apply here. `set_classes()` sets the vocabulary SmolVLM2 is asked to find; it is sticky, so it stays in effect across every later `predict()`/`track()` call until you set it again. SmolVLM2 needs no parser override in LibreYOLO: it follows the same chat-template-plus-JSON output as the tier's shared default, so its detection prompt and box format are not family-specific. Every detection carries the same placeholder confidence, so `conf` filtering is all-or-nothing rather than a ranking; `iou` does have an effect, dropping a later same-class box once it overlaps an already-kept one past the threshold, since a repeating generator can otherwise emit near-duplicate boxes for one object. SmolVLM2 also answers free-form questions through `chat()`, the same escape hatch documented on the `LibreVLM` factory. LibreYOLO's CLI does not cover this tier: there is no `libreyolo predict model=...` form for it. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants One size in the registry: SmolVLM2-500M-Video-Instruct, loaded as `LibreVLM("smolvlm2-500m")`. SmolVLM2 is a weaker detector than the purpose-built grounding models in this tier; LibreYOLO's own wrapper describes it as a demonstration that a new family needs no special-case parsing to work here, not as its strongest open-vocabulary option. LibreYOLO does not train, validate or export SmolVLM2: `train()`, `val()` and `export()` all raise `NotImplementedError` for every family in this tier (see the support tier above). Fine-tune SmolVLM2 upstream and load the resulting weights if you need a custom vocabulary baked in; check `predict()` output by eye instead of a COCO-style validation pass, since every detection carries the same placeholder confidence. ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: SmolVLM2, Hugging Face (HuggingFaceTB) - Upstream license: Apache-2.0 - Upstream source: https://github.com/huggingface/smollm/tree/main/vision - LibreYOLO code: MIT - Weights: Apache-2.0, distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. Both sizes LibreYOLO downloads, SmolVLM2-500M-Video-Instruct and SmolVLM2-2.2B-Instruct, carry this license on their Hugging Face repository. ## Citation ```bibtex @article{marafioti2025smolvlm, title={SmolVLM: Redefining small and efficient multimodal models}, author={Andrés Marafioti and Orr Zohar and Miquel Farré and Merve Noyan and Elie Bakouch and Pedro Cuenca and Cyril Zakka and Loubna Ben Allal and Anton Lozhkov and Nouamane Tazi and Vaibhav Srivastav and Joshua Lochner and Hugo Larcher and Mathieu Morlon and Lewis Tunstall and Leandro von Werra and Thomas Wolf}, journal={arXiv preprint arXiv:2504.05299}, year={2025} } ``` Copied from https://huggingface.co/blog/smolvlm2 --- # SSD SSD (Single Shot MultiBox Detector) predicts every box and class score from a dense grid of default boxes in one forward pass, with no separate region-proposal stage. LibreYOLO ships the VGG16-backed SSD300 checkpoint as an inference-only detector. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install SSD needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreSSD300.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreSSD300.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. SSD decodes its default-box grid with per-class scores and then runs non-maximum suppression, so `conf`, `iou` and `max_det` all have a real effect here, unlike the query-based detectors in this library. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants SSD ships one checkpoint: the VGG16-backed SSD300 network at its fixed native canvas. There is no size or scale choice in this family; predict, validate and export all use that one graph. The weight file is `LibreSSD300.pt`, the family prefix followed by its only size key, `"300"`. The class behind it is `LibreSSD`, so a direct construction is `LibreSSD(size="300")` rather than a class named after the file. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreSSD300.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreSSD300.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | | | | | | | | | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. SSD exports to ONNX only; every other format is currently blocked for this family. Export always uses the checkpoint's native canvas, and the graph exposes SSD's raw packed head rather than a fused non-maximum-suppression output, so `nms=True` is not accepted at export time. LibreYOLO's own backends run the decode and suppression step after loading the graph back. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreSSD300.pt") # imgsz is left out here on purpose: SSD300 traces at its checkpoint's # native canvas, and any other value raises before export starts. model.export(format="onnx") ``` **CLI** ```bash libreyolo export model=LibreSSD300.pt format=onnx ``` **Use the exported file** ```python from libreyolo import LibreYOLO # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreSSD300.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreSSD300.pt` | 300 | Detection | bsd-3-clause | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: SSD, UNC Chapel Hill, Zoox, Google and University of Michigan - Upstream license: BSD-3-Clause - Upstream source: https://github.com/pytorch/vision - LibreYOLO code: BSD-3-Clause - Weights: BSD-3-Clause, republished at https://huggingface.co/LibreYOLO - Interpretation: BSD-3-Clause is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its copyright notice and disclaimer with any copy you redistribute, and it places no obligation on your own application code. The official SSD300 COCO checkpoint carries no separate, checkpoint-specific license: LibreYOLO's mirror applies BSD-3-Clause on an explicitly disclosed implied basis, the same basis torchvision states for its own pretrained SSD300 weights. The VGG16 backbone the graph is built on traces back to Oxford's fully convolutional reduced VGGNet, released under CC BY 4.0 by Karen Simonyan and Andrew Zisserman. LibreYOLO's SSD300 code is not ported from the paper authors' own Caffe release; it derives from torchvision's BSD-3-Clause SSD300 implementation, and that is the repository linked above as the upstream source. The backbone's VGG16 weights trace further back to Oxford's fully convolutional reduced VGGNet, released under CC BY 4.0 by Karen Simonyan and Andrew Zisserman. ## Citation ```bibtex @inproceedings{liu2016ssd, title = {{SSD}: Single Shot MultiBox Detector}, author = {Liu, Wei and Anguelov, Dragomir and Erhan, Dumitru and Szegedy, Christian and Reed, Scott and Fu, Cheng-Yang and Berg, Alexander C.}, booktitle = {ECCV}, year = {2016} } ``` Copied from https://github.com/weiliu89/caffe/tree/ssd#citing-ssd --- # Swin Transformer Swin Transformer V1: a hierarchical vision transformer that computes attention inside shifted local windows instead of over the whole image. LibreYOLO ships four sizes for image classification. Tasks: classify. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install Swin needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreSwint-cls.pt") result = model(SAMPLE_IMAGE, save=True) probs = result.probs print(probs.top1, probs.top1conf) print(probs.top5, probs.top5conf) ``` **CLI** ```bash libreyolo predict model=LibreSwint-cls.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` A classifier returns `result.probs` instead of `result.boxes`: `top1` and `top5` give class indices, `top1conf` and `top5conf` give their confidences. Every size is fixed to a 224px input, because the final attention stage is built for that resolution; predict, validate and export all raise if you pass a different `imgsz`. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Four sizes, tiny through large, built from the same shifted-window tower and differing in embedding width and stage depth. Large is pretrained on ImageNet-22k and fine-tuned on ImageNet-1k; the other three are trained on ImageNet-1k directly. LibreYOLO ships this family inference-only: prediction, ImageNet-style top-1/top-5 validation and export are supported, and the upstream ImageNet training recipe is not implemented. ## Validate `val()` runs against an ImageFolder-style split (a directory with `train/` and `val/` subfolders, one folder per class) and returns top-1 and top-5 accuracy. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreSwint-cls.pt") # data is a directory root with train/ and val/ class-folder splits # (ImageFolder layout), not a dataset YAML. metrics = model.val(data="imagenet-1k/") print(metrics["metrics/accuracy_top1"]) print(metrics["metrics/accuracy_top5"]) ``` **CLI** ```bash libreyolo val model=LibreSwint-cls.pt data=imagenet-1k/ ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | classify | yes | yes | yes | yes | yes | | | | yes | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts and the extras a few of them add. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreSwint-cls.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreSwint-cls.pt format=onnx libreyolo export model=LibreSwint-cls.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreSwint-cls.onnx") result = model(SAMPLE_IMAGE) print(result.probs.top1) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreSwint-cls.pt` | 224 | classify | mit | | `LibreSwins-cls.pt` | 224 | classify | mit | | `LibreSwinb-cls.pt` | 224 | classify | mit | | `LibreSwinl-cls.pt` | 224 | classify | mit | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: Swin Transformer, Microsoft Research - Upstream license: MIT - Upstream source: https://github.com/microsoft/Swin-Transformer - LibreYOLO code: Apache-2.0 - Weights: MIT, republished at https://huggingface.co/LibreYOLO - Interpretation: MIT is a permissive license, so these weights can be used in commercial and closed-source products. It asks only that you keep the license text and copyright notice with any copy you redistribute, and it carries no explicit patent grant. LibreYOLO's runtime code for this family is a derived port of the Apache-2.0 timm Swin implementation (Ross Wightman, huggingface/pytorch-image-models), kept parameter-name compatible so the tensors load unchanged; the four released Tiny/Small/Base/Large checkpoints are Microsoft's own MIT-licensed patch-4/window-7 classifiers. Code and weights therefore sit under two different permissive licenses, both of which allow commercial use. ## Citation ```bibtex @inproceedings{liu2021Swin, title={Swin Transformer: Hierarchical Vision Transformer using Shifted Windows}, author={Liu, Ze and Lin, Yutong and Cao, Yue and Hu, Han and Wei, Yixuan and Zhang, Zheng and Lin, Stephen and Guo, Baining}, booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)}, year={2021} } ``` Copied from https://github.com/microsoft/Swin-Transformer#citing-swin-transformer --- # SwinIR A Swin Transformer network for image restoration. LibreYOLO ships inference and validation for its 4x super-resolution checkpoints: the official lightweight, real-world medium and real-world large generators. Tasks: restore. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install SwinIR needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreSwinIRm-restore.pt") result = model(SAMPLE_IMAGE, save=True) restored = result.restored print(restored.array.shape, restored.array.dtype) ``` **CLI** ```bash libreyolo predict model=LibreSwinIRm-restore.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **Tiled, for large images** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreSwinIRl-restore.pt") # tile splits the forward pass into overlapping tiles and blends the # seams back together; tile_pad is the halo added around each tile # before it is cropped back out. Both are Python-only keyword # arguments, not CLI flags. result = model("large-photo.jpg", tile=512, tile_pad=16, save=True) ``` A restore result carries no boxes; `result.restored` is a dense `(H, W, 3)` uint8 RGB image, on a canvas 4x the input in each dimension. `save=True` writes that image directly rather than an annotated plot. The input is padded to a multiple of 8 rather than resized, so predict runs at the photo's own resolution; a source larger than memory allows can be split with `tile` and `tile_pad`, which blend the tile seams back together in the output. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Three sizes, all fixed at a 4x upscale. `s` is the official lightweight generator, with four residual Swin Transformer block (RSTB) stages and pixel-shuffle-direct upsampling. `m` and `l` are the real-world medium and large generators, with six and nine RSTB stages and a nearest-neighbor-plus- convolution upsampler built for real-world degradations rather than only bicubic downscaling. ## Validate `val()` measures PSNR and SSIM between the restored output and a clean target image, both computed in RGB on the original canvas with no border crop and no resizing. SSIM uses an 11x11 Gaussian window with sigma 1.5, averaged over the three color channels. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreSwinIRm-restore.pt") metrics = model.val(data="my-restore-dataset.yaml") print(metrics["metrics/PSNR"]) print(metrics["metrics/SSIM"]) ``` **CLI** ```bash libreyolo val model=LibreSwinIRm-restore.pt data=my-restore-dataset.yaml ``` The dataset argument is a YAML pairing a directory of degraded input images with a directory of clean target images of matching resolution; see [dataset formats](/docs/reference/dataset-formats) for the exact keys. ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | restore | yes | yes | | yes | yes | | | | | yes | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. ExecuTorch and every format the matrix marks blocked are not available for this family; ONNX, TorchScript, TensorRT, OpenVINO and TFLite are. [Export](/docs/export) lists the arguments every format accepts and the extras a few of them add. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreSwinIRm-restore.pt") # imgsz defaults to a small internal patch size when omitted, not # your working resolution, so pass the size your deployment actually # feeds the model. model.export(format="onnx", imgsz=512) model.export(format="tensorrt", imgsz=512, half=True) ``` **CLI** ```bash libreyolo export model=LibreSwinIRm-restore.pt format=onnx imgsz=512 ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreSwinIRm-restore.onnx") result = model(SAMPLE_IMAGE) print(result.restored.array.shape) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreSwinIRs-restore.pt` | | restore | apache-2.0 | | `LibreSwinIRm-restore.pt` | | restore | apache-2.0 | | `LibreSwinIRl-restore.pt` | | restore | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: SwinIR, Computer Vision Lab, ETH Zurich - Upstream license: Apache-2.0 - Upstream source: https://github.com/JingyunLiang/SwinIR - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code. LibreYOLO's checkpoints are format conversions of the official pretrained generators, with the learned parameters unchanged; SwinIR's real-world variants need a degradation and GAN training pipeline that is not wired into this library, so there is no LibreYOLO-trained variant to license separately. ## Citation ```bibtex @article{liang2021swinir, title={SwinIR: Image Restoration Using Swin Transformer}, author={Liang, Jingyun and Cao, Jiezhang and Sun, Guolei and Zhang, Kai and Van Gool, Luc and Timofte, Radu}, journal={arXiv preprint arXiv:2108.10257}, year={2021} } ``` Copied from https://github.com/JingyunLiang/SwinIR#citation --- # TEED TEED (Tiny and Efficient Edge Detector) is a small convolutional network that predicts a dense edge-probability map from one RGB image. LibreYOLO wraps its architecture for edge detection only; no checkpoint ships with the library. Tasks: edge. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install TEED needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict LibreYOLO ships no TEED checkpoint. The officially released weights are trained on BIPED, whose published dataset terms restrict use to non-commercial purposes, so LibreYOLO does not mirror them. Convert a checkpoint you are licensed to use with `weights/convert_teed_weights.py`, which checks the tensor keys against the runtime architecture before writing a file LibreYOLO can load directly: ```bash python weights/convert_teed_weights.py upstream.pth weights/LibreTEEDt-edge.pt --verify ``` **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("weights/LibreTEEDt-edge.pt") result = model(SAMPLE_IMAGE, save=True) edges = result.edges print(edges.array.shape) # (H, W) float32 in [0, 1] print(edges.binary(0.5).sum()) # thresholded edge-pixel count ``` **CLI** ```bash libreyolo predict model=weights/LibreTEEDt-edge.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` `result.edges` holds the result: an `(H, W)` float32 array in `[0, 1]`, with `.binary(threshold)` returning a boolean edge mask. There are no boxes, so `conf`, `iou` and `max_det` have no effect. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants TEED ships one size in LibreYOLO. LibreYOLO's benchmark harness has not measured this family, so there are no published numbers to compare it against. ## Validate `val()` reports BSDS-style ODS and OIS F-measures against a paired edge dataset: images beside same-stem edge maps, with an optional validity mask so padded pixels never count. `imgsz` must be divisible by the network's downsample stride, and LibreYOLO raises a clear error if it is not. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("weights/LibreTEEDt-edge.pt") metrics = model.val(data="my-dataset.yaml", imgsz=352) print(metrics["metrics/ODS"]) # optimal-dataset-scale F-measure print(metrics["metrics/OIS"]) # optimal-image-scale F-measure ``` **CLI** ```bash libreyolo val model=weights/LibreTEEDt-edge.pt data=my-dataset.yaml imgsz=352 ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | edge | yes | yes | yes | yes | yes | | | | | yes | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. Edge export uses a fixed-resolution, batch-1 runtime contract: `dynamic` and a `batch` other than 1 are rejected, and the exported graph outputs a single fused probability map. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` file behaves like a checkpoint and returns the same `Results`. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("weights/LibreTEEDt-edge.pt") model.export(format="onnx", imgsz=352) model.export(format="tensorrt", imgsz=352, half=True) ``` **CLI** ```bash libreyolo export model=weights/LibreTEEDt-edge.pt format=onnx imgsz=352 libreyolo export model=weights/LibreTEEDt-edge.pt format=tensorrt imgsz=352 half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("weights/LibreTEEDt-edge.onnx") result = model(SAMPLE_IMAGE) print(result.edges.array.shape) ``` ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: TEED, Xavier Soria - Upstream license: MIT - Upstream source: https://github.com/xavysp/TEED - LibreYOLO code: MIT - Weights: MIT, distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: MIT is a permissive license, so the TEED architecture LibreYOLO ports can be used in commercial and closed-source products, with the license text and copyright notice kept alongside any copy you redistribute. LibreYOLO ships no checkpoint for this family: the officially released weights are trained on BIPED, whose published dataset terms restrict use to non-commercial purposes, and mirroring them would carry that restriction into a nominally MIT-licensed download. Convert a checkpoint you hold a license for with `weights/convert_teed_weights.py`; the MIT code license does not change the terms attached to whatever checkpoint you convert. LibreYOLO publishes no TEED checkpoint. Nothing is mirrored under the LibreYOLO organization; convert a checkpoint you hold a license for with `weights/convert_teed_weights.py` instead. ## Citation ```bibtex @InProceedings{Soria_2023teed, author = {Soria, Xavier and Li, Yachuan and Rouhani, Mohammad and Sappa, Angel D.}, title = {Tiny and Efficient Model for the Edge Detection Generalization}, booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops}, month = {October}, year = {2023}, pages = {1364-1373} } ``` Copied from https://github.com/xavysp/TEED#citation --- # VGG VGG is a convolutional image classifier built from uniform stacks of small 3x3 convolutions instead of larger filters. LibreYOLO ships the 16- and 19-layer sizes, plain and with batch normalization, for image classification. Tasks: classify. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install VGG needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreVGG16-cls.pt") result = model(SAMPLE_IMAGE, save=True) probs = result.probs print(probs.top1, probs.top1conf) print(probs.top5, probs.top5conf) ``` **CLI** ```bash libreyolo predict model=LibreVGG16-cls.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` A classifier returns `result.probs` instead of `result.boxes`: `top1` and `top5` give class indices, `top1conf` and `top5conf` give their confidences. Prediction runs at a fixed 224px input and raises if you pass a different `imgsz`. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Four sizes: 16 and 19 convolutional layers, each with a plain and a batch-normalized variant. The shipped weights are torchvision's later from-scratch ImageNet training, not conversions of the Oxford group's original 2014 Caffe release. LibreYOLO ships this family inference-only: prediction, ImageNet-style top-1/top-5 validation and export are supported, and fine-tuning is not implemented. ## Validate `val()` runs against an ImageFolder-style split (a directory with `train/` and `val/` subfolders, one folder per class) and returns top-1 and top-5 accuracy. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreVGG16-cls.pt") # data is a directory root with train/ and val/ class-folder splits # (ImageFolder layout), not a dataset YAML. metrics = model.val(data="imagenet-1k/") print(metrics["metrics/accuracy_top1"]) print(metrics["metrics/accuracy_top5"]) ``` **CLI** ```bash libreyolo val model=LibreVGG16-cls.pt data=imagenet-1k/ ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | classify | yes | yes | yes | yes | yes | | | | yes | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts and the extras a few of them add. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreVGG16-cls.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreVGG16-cls.pt format=onnx libreyolo export model=LibreVGG16-cls.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreVGG16-cls.onnx") result = model(SAMPLE_IMAGE) print(result.probs.top1) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreVGG16-cls.pt` | 224 | classify | bsd-3-clause | | `LibreVGG19bn-cls.pt` | 224 | classify | bsd-3-clause | | `LibreVGG19-cls.pt` | 224 | classify | bsd-3-clause | | `LibreVGG16bn-cls.pt` | 224 | classify | bsd-3-clause | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: VGG, Visual Geometry Group, University of Oxford - Upstream license: BSD-3-Clause - Upstream source: https://github.com/pytorch/vision - LibreYOLO code: BSD-3-Clause - Weights: BSD-3-Clause, republished at https://huggingface.co/LibreYOLO - Interpretation: BSD-3-Clause is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep the copyright notice, the list of conditions and the disclaimer with any copy you redistribute, and it forbids using the contributors' names to endorse a derived product without permission; it carries no explicit patent grant. LibreYOLO's code and the four shipped checkpoints (16, 19, 16-BN, 19-BN) are both derived from torchvision, not from the Oxford group's original 2014 Caffe release, which is a separate model under Creative Commons Attribution and is not what LibreYOLO redistributes. Torchvision itself notes that BSD-3-Clause redistribution of a pretrained checkpoint is an implied basis rather than a grant written for that specific checkpoint, and that pretrained-model terms can depend on the data a model was trained on; LibreYOLO repeats that caveat on each hosted weights repository. --- # ViT The classic Vision Transformer: a pure transformer applied to fixed-size image patches, with a learned class token and no convolutions. LibreYOLO ships four AugReg-pretrained sizes for image classification. Tasks: classify. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install ViT needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreViTti-cls.pt") result = model(SAMPLE_IMAGE, save=True) probs = result.probs print(probs.top1, probs.top1conf) print(probs.top5, probs.top5conf) ``` **CLI** ```bash libreyolo predict model=LibreViTti-cls.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` A classifier returns `result.probs` instead of `result.boxes`: `top1` and `top5` give class indices, `top1conf` and `top5conf` give their confidences. Preprocessing resizes and center-crops to a fixed 224px input, using timm's AugReg evaluation recipe: bicubic interpolation at a 0.9 crop fraction. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Four sizes, tiny through large, sharing one fixed 224px, patch-16 graph and differing in embedding width and transformer depth. LibreYOLO ships this family inference-only: prediction, ImageNet-style top-1/top-5 validation and export are supported, and the AugReg fine-tuning recipe is not implemented. ## Validate `val()` runs against an ImageFolder-style split (a directory with `train/` and `val/` subfolders, one folder per class) and returns top-1 and top-5 accuracy. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreViTti-cls.pt") # data is a directory root with train/ and val/ class-folder splits # (ImageFolder layout), not a dataset YAML. metrics = model.val(data="imagenet-1k/") print(metrics["metrics/accuracy_top1"]) print(metrics["metrics/accuracy_top5"]) ``` **CLI** ```bash libreyolo val model=LibreViTti-cls.pt data=imagenet-1k/ ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | classify | yes | yes | yes | yes | yes | | | | yes | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. [Export](/docs/export) lists the arguments every format accepts and the extras a few of them add. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreViTti-cls.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreViTti-cls.pt format=onnx libreyolo export model=LibreViTti-cls.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreViTti-cls.onnx") result = model(SAMPLE_IMAGE) print(result.probs.top1) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreViTti-cls.pt` | 224 | classify | apache-2.0 | | `LibreViTs-cls.pt` | 224 | classify | apache-2.0 | | `LibreViTb-cls.pt` | 224 | classify | apache-2.0 | | `LibreViTl-cls.pt` | 224 | classify | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: ViT, Google Research - Upstream license: Apache-2.0 - Upstream source: https://github.com/google-research/vision_transformer - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code. LibreYOLO's runtime code for this family is a derived port of the Apache-2.0 timm Vision Transformer implementation (Ross Wightman, huggingface/pytorch-image-models), kept checkpoint-compatible with the shipped tensors. The four AugReg checkpoints themselves are timm's Apache-2.0 conversion of Google Research's own AugReg pretraining, so the code and the weights carry the same permissive terms end to end. ## Citation ```bibtex @article{dosovitskiy2020vit, title={An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale}, author={Dosovitskiy, Alexey and Beyer, Lucas and Kolesnikov, Alexander and Weissenborn, Dirk and Zhai, Xiaohua and Unterthiner, Thomas and Dehghani, Mostafa and Minderer, Matthias and Heigold, Georg and Gelly, Sylvain and Uszkoreit, Jakob and Houlsby, Neil}, journal={ICLR}, year={2021} } @article{steiner2021augreg, title={How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers}, author={Steiner, Andreas and Kolesnikov, Alexander and and Zhai, Xiaohua and Wightman, Ross and Uszkoreit, Jakob and Beyer, Lucas}, journal={arXiv preprint arXiv:2106.10270}, year={2021} } ``` Copied from https://github.com/google-research/vision_transformer#bibtex --- # YOLO-NAS A convolutional detector whose backbone and neck came out of Deci.AI's architecture search, built from quantization-aware RepVGG blocks. Its weights are Deci.AI's, licensed for non-commercial use only, and LibreYOLO publishes none of them. Tasks: Detection, Pose. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install YOLO-NAS needs no extra beyond the base package. ```bash pip install libreyolo ``` ## Predict A checkpoint name that is not already on disk is fetched from Deci's public CDN, not from the LibreYOLO org, which hosts none of these weights. Before the transfer starts the library prints Deci's license terms once per process, and before the downloaded file is opened its SHA-256 is checked against a pinned value. What those terms allow is in [licensing](#licensing). **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # A name not already on disk is fetched from Deci's CDN. The download # prints Deci's license terms first; taking the file accepts them. model = LibreYOLO("LibreYOLONASs.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreYOLONASs.pt save=True \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` **Pose** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The -pose suffix picks the pose head and its own set of weights. model = LibreYOLO("LibreYOLONASs-pose.pt") result = model(SAMPLE_IMAGE) print(result.keypoints.xy) ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. `conf` sets the confidence threshold and `iou` the NMS threshold. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Detection and pose are the same architecture under different heads, and they take the same arguments. The sizes in the table below are the detection ones; pose is published at those and at one smaller size. The pose head predicts the COCO keypoint set. | Checkpoint | Input (px) | mAP 50-95 | Params (M) | | --- | --- | --- | --- | | `LibreYOLONASl` | 640 | 56.3 | 66.98 | | `LibreYOLONASm` | 640 | 55.4 | 51.18 | | `LibreYOLONASs` | 640 | 51.8 | 19.05 | COCO val2017, 500 images. Published on https://www.visionanalysis.org/ Interactive chart: https://www.visionanalysis.org/embed/scatter?highlight=yolonas-l%2Cyolonas-m%2Cyolonas-s&title=YOLO-NAS%20on%20COCO&subtitle=Accuracy%20against%20latency%2C%20every%20benchmarked%20model ## Train **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLONASs.pt") model.train(data="my-dataset.yaml", epochs=100, imgsz=640, batch=16) ``` **CLI** ```bash libreyolo train model=LibreYOLONASs.pt data=my-dataset.yaml \ epochs=100 imgsz=640 batch=16 ``` **From scratch** ```python from libreyolo import LibreYOLONAS # No Deci checkpoint is touched: the model starts from random weights, # so what comes out of the run derives only from your data. model = LibreYOLONAS(None, size="s") model.train(data="my-dataset.yaml", imgsz=640, batch=16) ``` `epochs`, `lr0` and `amp` are resolved per task when you leave them out, so a pose run starts from different defaults than a detection run. The optimizer defaults to AdamW. The class count comes from the dataset YAML and the head is rebuilt for it before the first epoch; on the pose head the keypoint count is handled the same way, so a COCO pose checkpoint can be fine-tuned onto a skeleton of a different size. Fine-tuning starts from Deci's weights, which is what Deci's license covers. Training from a randomly initialized model involves no Deci checkpoint at all, and that is the third snippet above. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLONASs.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreYOLONASs.pt data=my-dataset.yaml ``` **Against COCO** ```bash # The bundled COCO yaml carries an embedded download script, so it # needs explicit permission unless the dataset is already local. libreyolo val model=LibreYOLONASl.pt data=coco.yaml imgsz=640 \ allow_download_scripts=True ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | yes | yes | yes | yes | yes | | yes | | Pose | yes | yes | yes | yes | yes | yes | | | yes | | | | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Running the graph in a bare runtime, with no LibreYOLO installed, is also supported, but then preprocessing and postprocessing are yours to write. Each format installs a different extra and takes a few arguments of its own. Both are on that format's page. An export is another copy of the same weights in a different container. Exporting a Deci checkpoint changes neither where the weights came from nor the license that covers them. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLONASs.pt") model.export(format="onnx", imgsz=640) ``` **CLI** ```bash libreyolo export model=LibreYOLONASs.pt format=onnx imgsz=640 ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreYOLONASs.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints There are none to list. Deci's license forbids redistribution, so the LibreYOLO org publishes no YOLO-NAS weights and the download resolves elsewhere: a name of the form `LibreYOLONAS.pt`, or `LibreYOLONAS-pose.pt` for pose, maps to the matching object on Deci's public CDN. Only the checkpoints whose SHA-256 the library pins can be fetched that way. Anything else fails closed rather than opening an unverified third-party pickle, and has to be downloaded by hand and passed as a path. A file already on disk loads from its path, with no download and no checksum gate. That includes a Deci `.pth` under its original name, which the loader recognizes. ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: YOLO-NAS, Deci.AI - Upstream license: Deci.AI proprietary, non-commercial - Upstream source: https://github.com/Deci-AI/super-gradients - LibreYOLO code: Apache-2.0 - Weights: Deci.AI proprietary, non-commercial, distributed by their authors. LibreYOLO does not host or mirror them. - Interpretation: Two licenses apply here and they are not the same. The SuperGradients source this port follows is Apache-2.0 and LibreYOLO's own implementation is MIT, so nothing restricts the architecture or the training code. Deci's published checkpoints are a different matter: the YOLO-NAS license grants a revocable, non-transferable right to use them and nothing more. It forbids redistribution and sublicensing, forbids offering them as a managed or remote service, restricts modification and reverse engineering, and rules out commercial use, including any model running in a production environment, unless you hold a separate agreement with Deci. LibreYOLO therefore does not mirror them, and prints those terms once before the download starts. Weights trained from a randomly initialized model on your own data derive from no Deci checkpoint. LibreYOLO neither hosts nor mirrors these weights: nothing for this family exists in the LibreYOLO Hugging Face org. Every auto-download goes to Deci's public CDN instead, prints Deci's terms once per process before it starts, and is checked against a pinned SHA-256 before the file is opened. Training from a randomly initialized model is the alternative. The architecture is Apache-2.0 upstream and MIT here, so a model trained that way on your own data derives from no Deci checkpoint. ## Citation YOLO-NAS was released without a paper. The entry below is the one its authors ask for, covering SuperGradients, the library it shipped in. ```bibtex @misc{supergradients, doi = {10.5281/ZENODO.7789328}, url = {https://zenodo.org/record/7789328}, author = {Aharon, Shay and {Louis-Dupont} and {Ofri Masad} and Yurkova, Kate and {Lotem Fridman} and {Lkdci} and Khvedchenya, Eugene and Rubin, Ran and Bagrov, Natan and Tymchenko, Borys and Keren, Tomer and Zhilko, Alexander and {Eran-Deci}}, title = {Super-Gradients}, publisher = {GitHub}, journal = {GitHub repository}, year = {2021}, } ``` Copied from https://github.com/Deci-AI/super-gradients#citing --- # YOLOv1 YOLOv1 is the original 2016 detector that gave the YOLO family its name: one convolutional network with a fully connected head predicts every box and class score in a single pass, with no anchor boxes. LibreYOLO carries it as a frozen, inference-only exhibit. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install YOLOv1 needs no extra beyond the base package. ```bash pip install libreyolo ``` ## Predict This family is inference-only: `train()` raises `NotImplementedError`, so this page has no Train section. Predict, validate and export are all supported. Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO1b.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreYOLO1b.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. Two things are specific to this family. The published checkpoint is trained on Pascal VOC (2007+2012), not COCO, so `box.cls` indexes the 20 VOC categories (aeroplane, bicycle, bird, boat, bottle, bus, car, cat, chair, cow, diningtable, dog, horse, motorbike, person, pottedplant, sheep, sofa, train, tvmonitor) rather than the 80 COCO ones. And the fully connected detection head accepts one image at a time, so a list of sources is looped rather than run as a true batch. See [prediction](/docs/predict) for sources, streaming and result handling. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against a dataset in the same VOC-style label space the checkpoint was trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO1b.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreYOLO1b.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | | | | yes | | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Running the graph in a bare runtime, with no LibreYOLO installed, is also supported, but then preprocessing and postprocessing are yours to write. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO1b.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreYOLO1b.pt format=onnx libreyolo export model=LibreYOLO1b.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreYOLO1b.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreYOLO1b.pt` | 448 | Detection | other | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: YOLOv1, Joseph Redmon - Upstream license: Public domain (Darknet "YOLO LICENSE") - Upstream source: https://github.com/pjreddie/darknet - LibreYOLO code: MIT - Weights: Public domain (Darknet "YOLO LICENSE"), republished at https://huggingface.co/LibreYOLO - Interpretation: Darknet's bundled NOTICE quotes its own license in full: "Darknet is public domain. Do whatever you want with it." That covers both the architecture and the pretrained weights LibreYOLO converts from it, with no attribution requirement and no restriction on commercial use. LibreYOLO's own code around this architecture is MIT. One thing to know before deploying this checkpoint: it is trained on Pascal VOC 2007+2012, not COCO, so it detects and names the 20 VOC categories rather than the 80 COCO ones. --- # YOLOv2 YOLOv2, also published as YOLO9000, is the Darknet-19 detector that introduced anchor boxes and a passthrough layer to the YOLO line. LibreYOLO carries it as a frozen, inference-only exhibit. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install YOLOv2 needs no extra beyond the base package. ```bash pip install libreyolo ``` ## Predict This family is inference-only: `train()` raises `NotImplementedError`, so this page has no Train section. Predict, validate and export are all supported. Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO2b.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreYOLO2b.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. `conf` filters the confidence threshold and `iou` the NMS threshold, applied against the `region` head's anchor-based predictions. See [prediction](/docs/predict) for sources, streaming and result handling. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you validate on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO2b.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreYOLO2b.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | | | | yes | | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Running the graph in a bare runtime, with no LibreYOLO installed, is also supported, but then preprocessing and postprocessing are yours to write. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO2b.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreYOLO2b.pt format=onnx libreyolo export model=LibreYOLO2b.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreYOLO2b.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreYOLO2t.pt` | 416 | Detection | other | | `LibreYOLO2b.pt` | 608 | Detection | other | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: YOLOv2, Joseph Redmon - Upstream license: Public domain (Darknet "YOLO LICENSE") - Upstream source: https://github.com/pjreddie/darknet - LibreYOLO code: MIT - Weights: Public domain (Darknet "YOLO LICENSE"), republished at https://huggingface.co/LibreYOLO - Interpretation: Darknet's bundled NOTICE quotes its own license in full: "Darknet is public domain. Do whatever you want with it." That covers both the architecture and the pretrained weights LibreYOLO converts from it, with no attribution requirement and no restriction on commercial use. LibreYOLO's own code around this architecture is MIT. --- # YOLOv3 YOLOv3 is the Darknet-53 detector that added multi-scale prediction and independent logistic classifiers to the YOLO line. LibreYOLO carries it as a frozen, inference-only exhibit in tiny, base and SPP sizes. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install YOLOv3 needs no extra beyond the base package. ```bash pip install libreyolo ``` ## Predict This family is inference-only: `train()` raises `NotImplementedError`, so this page has no Train section. Predict, validate and export are all supported. Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO3b.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreYOLO3b.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **SPP size** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The SPP variant adds a spatial pyramid pooling block before the # detection heads and runs at its own native input size. model = LibreYOLO("LibreYOLO3spp.pt") result = model(SAMPLE_IMAGE) ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. `conf` filters the confidence threshold and `iou` the NMS threshold, applied per scale before boxes from all three heads are merged. See [prediction](/docs/predict) for sources, streaming and result handling. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you validate on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO3b.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreYOLO3b.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | | | | yes | | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Running the graph in a bare runtime, with no LibreYOLO installed, is also supported, but then preprocessing and postprocessing are yours to write. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO3b.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreYOLO3b.pt format=onnx libreyolo export model=LibreYOLO3b.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreYOLO3b.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreYOLO3t.pt` | 416 | Detection | other | | `LibreYOLO3b.pt` | 416 | Detection | other | | `LibreYOLO3spp.pt` | 608 | Detection | other | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: YOLOv3, Joseph Redmon - Upstream license: Public domain (Darknet "YOLO LICENSE") - Upstream source: https://github.com/pjreddie/darknet - LibreYOLO code: MIT - Weights: Public domain (Darknet "YOLO LICENSE"), republished at https://huggingface.co/LibreYOLO - Interpretation: Darknet's bundled NOTICE quotes its own license in full: "Darknet is public domain. Do whatever you want with it." That covers both the architecture and the pretrained weights LibreYOLO converts from it, with no attribution requirement and no restriction on commercial use. LibreYOLO's own code around this architecture is MIT. --- # YOLOv4 YOLOv4 combines a CSPDarknet-53 backbone, an SPP block and a PANet neck with Mish activations. LibreYOLO carries it as a frozen, inference-only exhibit in tiny and base sizes. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install YOLOv4 needs no extra beyond the base package. ```bash pip install libreyolo ``` ## Predict This family is inference-only: `train()` raises `NotImplementedError`, so this page has no Train section. Predict, validate and export are all supported. Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO4b.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreYOLO4b.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. `conf` filters the confidence threshold and `iou` the NMS threshold, applied after each head's own `scale_x_y` center scaling. See [prediction](/docs/predict) for sources, streaming and result handling. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you validate on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO4b.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreYOLO4b.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | | | | yes | | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Running the graph in a bare runtime, with no LibreYOLO installed, is also supported, but then preprocessing and postprocessing are yours to write. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO4b.pt") model.export(format="onnx") model.export(format="tensorrt", half=True) ``` **CLI** ```bash libreyolo export model=LibreYOLO4b.pt format=onnx libreyolo export model=LibreYOLO4b.pt format=tensorrt half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreYOLO4b.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreYOLO4t.pt` | 416 | Detection | other | | `LibreYOLO4b.pt` | 608 | Detection | other | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: YOLOv4, Alexey Bochkovskiy - Upstream license: Public domain (Darknet "YOLO LICENSE") - Upstream source: https://github.com/AlexeyAB/darknet - LibreYOLO code: MIT - Weights: Public domain (Darknet "YOLO LICENSE"), republished at https://huggingface.co/LibreYOLO - Interpretation: Darknet's bundled NOTICE quotes its own license in full: "Darknet is public domain. Do whatever you want with it." That covers both the architecture and the pretrained weights LibreYOLO converts from it, with no attribution requirement and no restriction on commercial use. LibreYOLO's own code around this architecture is MIT. ## Citation ```bibtex @misc{bochkovskiy2020yolov4, title={YOLOv4: Optimal Speed and Accuracy of Object Detection}, author={Alexey Bochkovskiy and Chien-Yao Wang and Hong-Yuan Mark Liao}, year={2020}, eprint={2004.10934}, archivePrefix={arXiv}, primaryClass={cs.CV} } ``` Copied from https://github.com/AlexeyAB/darknet#citation --- # YOLOv7 YOLOv7 is an anchor-based, single-stage detector whose head adds learned implicit-knowledge offsets before the final convolution. LibreYOLO supports its single published size for detection. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install YOLOv7 needs no extra beyond the base package. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO7b.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreYOLO7b.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. `conf` sets the confidence threshold and `iou` the NMS threshold applied after the anchor-based head is decoded. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants LibreYOLO ships one size, `b`. Upstream publishes a single YOLOv7 model, so there is no size to choose between. ## Train **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO7b.pt") model.train(data="my-dataset.yaml", epochs=300, imgsz=640, batch=16, lr0=0.01) ``` **CLI** ```bash libreyolo train model=LibreYOLO7b.pt data=my-dataset.yaml \ epochs=300 imgsz=640 batch=16 lr0=0.01 ``` **Warm start from a fresh model** ```python from libreyolo import LibreYOLO7 # pretrained=True always loads the published LibreYOLO7b.pt checkpoint, # regardless of what this instance was constructed with. Constructing # the class directly, rather than through LibreYOLO(), starts with no # weights loaded at all. model = LibreYOLO7(None, size="b") model.train(data="my-dataset.yaml", epochs=300, pretrained=True) ``` `pretrained` is read, unlike the no-op of the same name on some other families here: pass `True` to warm-start from the published `LibreYOLO7b.pt` checkpoint (auto-downloaded), or a path or name for anything else. That published checkpoint is 80-class COCO, so requesting it on a model already rebuilt for a different class count first rebuilds back to 80, loads it, then transfers every shape-matching tensor into the target head count once the dataset's class count is read. `resume=True` cannot be combined with `pretrained`. Left at the default `None`, training continues from whatever the model was constructed with, or from a random initialization if nothing was loaded. Left alone otherwise, the trainer runs 300 epochs at `lr0=0.01` with SGD momentum 0.937, a 3-epoch warmup, and the same SimOTA assignment and final 15-epoch no-augmentation phase YOLOX uses, adapted to the anchor-based head. The one difference: YOLOX adds an L1 box-regression refinement during those final epochs that v7 skips, because v7's SimOTA loss carries no raw-offset L1 branch to refine. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO7b.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreYOLO7b.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | | | | yes | | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Running the graph in a bare runtime, with no LibreYOLO installed, is also supported, but then preprocessing and postprocessing are yours to write. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO7b.pt") model.export(format="onnx", imgsz=640) model.export(format="tensorrt", imgsz=640, half=True) ``` **CLI** ```bash libreyolo export model=LibreYOLO7b.pt format=onnx imgsz=640 libreyolo export model=LibreYOLO7b.pt format=tensorrt imgsz=640 half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreYOLO7b.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreYOLO7b.pt` | 640 | Detection | mit | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: YOLOv7, MultimediaTechLab - Upstream license: MIT - Upstream source: https://github.com/MultimediaTechLab/YOLO - LibreYOLO code: MIT - Weights: MIT, republished at https://huggingface.co/LibreYOLO - Interpretation: MIT is a permissive license, so these weights can be used in commercial and closed-source products. The one standing obligation is to keep the license text and the copyright notice, Kin-Yiu Wong and Hao-Tang Tsui, with any copy you redistribute. It places no condition on your own application code, and a model you train yourself on your own data is yours. The port follows the authors' MIT re-release of YOLOv7, not the GPL-3.0 WongKinYiu/yolov7 repository that carries the same model, so the permissive terms come from the source LibreYOLO actually derives from. ## Citation ```bibtex @inproceedings{wang2022yolov7, title={{YOLOv7}: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors}, author={Wang, Chien-Yao and Bochkovskiy, Alexey and Liao, Hong-Yuan Mark}, year={2023}, booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, } ``` Copied from https://github.com/MultimediaTechLab/YOLO#citations --- # YOLOv9 A single-stage convolutional detector: one pass scores a dense grid of boxes and NMS drops the duplicates. LibreYOLO carries three variants of it, one of which has no NMS step. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install YOLOv9 needs no extra beyond the base package. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreYOLO9s.pt save=True \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` **Without NMS** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # Same call, different checkpoint. The end-to-end head returns its own # top-scoring predictions, so no NMS runs and iou is ignored. model = LibreYOLO("LibreYOLO9E2Es.pt") result = model(SAMPLE_IMAGE, conf=0.25, max_det=300) print(len(result.boxes)) ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. On the base and stride-4 models, `conf` sets the confidence threshold and `iou` the NMS threshold. The end-to-end model runs no NMS and ignores `iou`, so `conf` and `max_det` are what shape its output. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Three variants share a backbone. All three detect only, and they take the same arguments. The base model predicts on three feature scales and clears duplicate boxes with NMS. The end-to-end model keeps that head and adds a one-to-one matching branch beside it. Inference reads the one-to-one branch alone and takes its top-scoring predictions, so no NMS runs. Choose it when the runtime you deploy to has no NMS operator. The stride-4 model surfaces one level further up the backbone, extends the neck down to it and predicts on four scales instead of three. The extra scale is for objects that cover few pixels; the one published checkpoint for it is trained on aerial imagery. Base detection checkpoints transfer into it: the backbone and neck load unchanged, the three pretrained head towers shift up one slot, and the stride-4 tower starts from random initialization. | Checkpoint | Input (px) | mAP 50-95 | Params (M) | | --- | --- | --- | --- | | `LibreYOLO9c` | 640 | 56.4 | 25.5 | | `LibreYOLO9m` | 640 | 55.3 | 20.12 | | `LibreYOLO9s` | 640 | 55.9 | 7.2 | | `LibreYOLO9t` | 640 | 54.0 | 2.02 | COCO val2017, 500 images. Published on https://www.visionanalysis.org/ Interactive chart: https://www.visionanalysis.org/embed/scatter?highlight=yolov9c%2Cyolov9m%2Cyolov9s%2Cyolov9t&title=YOLOv9%20on%20COCO&subtitle=Accuracy%20against%20latency%2C%20every%20benchmarked%20model ## Train **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") model.train(data="my-dataset.yaml", epochs=100, imgsz=640, batch=16) ``` **CLI** ```bash libreyolo train model=LibreYOLO9s.pt data=my-dataset.yaml \ epochs=100 imgsz=640 batch=16 ``` **Small objects** ```python from libreyolo import LibreYOLO9P2 # The stride-4 variant has no COCO checkpoint of its own, so name a # base detection one: its backbone and neck load unchanged and the # stride-4 head tower starts from random initialization. model = LibreYOLO9P2(None, size="s") model.train(data="my-dataset.yaml", epochs=100, pretrained="LibreYOLO9s.pt") ``` `pretrained` decides what the run starts from. Pass `True` to load the published checkpoint for the same model and size, or a name or path for anything else. Tensors whose shape does not match are skipped rather than refused, and the run logs how many loaded, so a checkpoint trained on a different class count is still a usable starting point. The stride-4 model has no published COCO checkpoint of its own, so `True` resolves there to a file that does not exist and the download fails. Name a base detection checkpoint instead. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreYOLO9s.pt data=my-dataset.yaml ``` **Against COCO** ```bash # The bundled COCO yaml carries an embedded download script, so it # needs explicit permission unless the dataset is already local. libreyolo val model=LibreYOLO9c.pt data=coco.yaml imgsz=640 \ allow_download_scripts=True ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | yes | yes | | yes | | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. A tick holds for all three variants: where they differ, the matrix carries the weakest of the three. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Running the graph in a bare runtime, with no LibreYOLO installed, is also supported, but then preprocessing and postprocessing are yours to write. For the base detection model, the postprocessing half of that can move into the graph. `nms=True` on an ONNX export puts suppression inside the model, and the first output becomes a fixed `(1, max_det, 6)` tensor whose rows are `x1, y1, x2, y2, score, class`, zero-padded past the detection count. That graph is batch 1 and carries no dynamic axes. The end-to-end and stride-4 models do not accept the flag. Each format installs a different extra and takes a few arguments of its own. Both are on that format's page. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") model.export(format="onnx", imgsz=640) ``` **CLI** ```bash libreyolo export model=LibreYOLO9s.pt format=onnx imgsz=640 ``` **With NMS in the graph** ```bash libreyolo export model=LibreYOLO9s.pt format=onnx nms=True \ conf=0.25 iou=0.45 max_det=300 ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreYOLO9s.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreYOLO9t.pt` | 640 | Detection | mit | | `LibreYOLO9s.pt` | 640 | Detection | mit | | `LibreYOLO9m.pt` | 640 | Detection | mit | | `LibreYOLO9c.pt` | 640 | Detection | mit | | `LibreYOLO9E2Et.pt` | 640 | Detection | mit | | `LibreYOLO9E2Es.pt` | 640 | Detection | mit | | `LibreYOLO9E2Em.pt` | 640 | Detection | mit | | `LibreYOLO9E2Ec.pt` | 640 | Detection | mit | | `LibreYOLO9P2s-visdrone.pt` | | Detection | cc-by-nc-sa-3.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: YOLOv9, MultimediaTechLab - Upstream license: MIT - Upstream source: https://github.com/MultimediaTechLab/YOLO - LibreYOLO code: MIT - Weights: MIT, republished at https://huggingface.co/LibreYOLO - Interpretation: MIT is a permissive license, so these weights can be used in commercial and closed-source products. The one standing obligation is to keep the license text and the copyright notice, Kin-Yiu Wong and Hao-Tang Tsui, with any copy you redistribute. It places no condition on your own application code, and a model you train yourself on your own data is yours. Two things are worth knowing beyond that. The port follows the authors' MIT re-release of YOLOv9, not the GPL-3.0 repository that carries the same model, so the permissive terms come from the source LibreYOLO actually derives from. And one checkpoint in this family is not MIT: the stride-4 model trained on VisDrone2019-DET inherits that dataset's CC BY-NC-SA 3.0 terms, which rule out commercial use and require share-alike on anything derived from it. One checkpoint here is not MIT. The stride-4 model trained on VisDrone2019-DET inherits that dataset's CC BY-NC-SA 3.0 terms: non-commercial use only, share-alike on anything derived from it, and outside the permissive license the rest of this family ships under. It predicts the VisDrone aerial classes rather than the COCO ones. The library prints all of this before it downloads the file. ## Citation ```bibtex @inproceedings{wang2024yolov9, title={{YOLOv9}: Learning What You Want to Learn Using Programmable Gradient Information}, author={Wang, Chien-Yao and Yeh, I-Hau and Liao, Hong-Yuan Mark}, year={2024}, booktitle={Proceedings of the European Conference on Computer Vision (ECCV)}, } ``` Copied from https://github.com/MultimediaTechLab/YOLO#citations --- # YOLOX YOLOX is an anchor-free, single-stage detector with a decoupled classification-regression head, trained with SimOTA label assignment. LibreYOLO supports it for detection. Tasks: Detection. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install YOLOX needs no extra beyond the base package. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLOXs.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreYOLOXs.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` The returned `Results` object is the one every family returns, so swapping in a different detector is a one line change. `conf` sets the confidence threshold and `iou` the NMS threshold applied across the three decoupled prediction scales. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Six sizes share the same CSP backbone and PAFPN neck. The two smallest, `n` and `t`, run at a smaller fixed input resolution than the other four; the benchmark table below carries the exact figure for each. | Checkpoint | Input (px) | mAP 50-95 | Params (M) | | --- | --- | --- | --- | | `LibreYOLOXnano` | 416 | 28.8 | 0.91 | | `LibreYOLOXtiny` | 416 | 35.1 | 5.06 | | `LibreYOLOXl` | 640 | 53.9 | 54.21 | | `LibreYOLOXm` | 640 | 50.9 | 25.33 | | `LibreYOLOXs` | 640 | 43.0 | 8.97 | | `LibreYOLOXx` | 640 | 56.3 | 99.07 | COCO val2017, 500 images. Published on https://www.visionanalysis.org/ Interactive chart: https://www.visionanalysis.org/embed/scatter?highlight=yolox-nano%2Cyolox-tiny%2Cyolox-l%2Cyolox-m%2Cyolox-s%2Cyolox-x&title=YOLOX%20on%20COCO&subtitle=Accuracy%20against%20latency%2C%20every%20benchmarked%20model ## Train **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLOXs.pt") model.train(data="my-dataset.yaml", epochs=300, imgsz=640, batch=16, lr0=0.01) ``` **CLI** ```bash libreyolo train model=LibreYOLOXs.pt data=my-dataset.yaml \ epochs=300 imgsz=640 batch=16 lr0=0.01 ``` Left alone, the trainer runs 300 epochs at `lr0=0.01` with SGD momentum 0.9, a 5-epoch warmup and mosaic and mixup augmentation switched off for the final 15 epochs. `train()` also accepts a `pretrained` argument, but the value is never read inside the method: training always continues from whatever weights the model was constructed with, so `pretrained=False` does not reinitialize the network. `imgsz` defaults to a fixed value in the base training config, not to the loaded checkpoint's native resolution. That affects the `n` and `t` checkpoints specifically: continuing to train either one without setting `imgsz` explicitly switches it up to the larger default rather than the smaller size it was published at. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a dictionary of `metrics/` keys covering precision, recall, mAP 50 and mAP 50-95, measured against any dataset in the format you trained on. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLOXs.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) ``` **CLI** ```bash libreyolo val model=LibreYOLOXs.pt data=my-dataset.yaml ``` **Against COCO** ```bash # The bundled COCO yaml carries an embedded download script, so it # needs explicit permission unless the dataset is already local. libreyolo val model=LibreYOLOXn.pt data=coco.yaml imgsz=416 \ allow_download_scripts=True ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Detection | yes | yes | yes | yes | yes | | | | yes | yes | yes | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Running the graph in a bare runtime, with no LibreYOLO installed, is also supported, but then preprocessing and postprocessing are yours to write. A CoreML export can bake NMS into the graph with `nms=True`; YOLOX and YOLOv9 are the only two families that flag currently accepts. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLOXs.pt") model.export(format="onnx", imgsz=640) model.export(format="tensorrt", imgsz=640, half=True) ``` **CLI** ```bash libreyolo export model=LibreYOLOXs.pt format=onnx imgsz=640 libreyolo export model=LibreYOLOXs.pt format=tensorrt imgsz=640 half=True ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreYOLOXs.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreYOLOXn.pt` | 416 | Detection | apache-2.0 | | `LibreYOLOXt.pt` | 416 | Detection | apache-2.0 | | `LibreYOLOXs.pt` | 640 | Detection | apache-2.0 | | `LibreYOLOXm.pt` | 640 | Detection | apache-2.0 | | `LibreYOLOXl.pt` | 640 | Detection | apache-2.0 | | `LibreYOLOXx.pt` | 640 | Detection | apache-2.0 | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: YOLOX, Megvii - Upstream license: Apache-2.0 - Upstream source: https://github.com/Megvii-BaseDetection/YOLOX - LibreYOLO code: Apache-2.0 - Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO - Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours. ## Citation ```bibtex @article{yolox2021, title={YOLOX: Exceeding YOLO Series in 2021}, author={Ge, Zheng and Liu, Songtao and Wang, Feng and Li, Zeming and Sun, Jian}, journal={arXiv preprint arXiv:2107.08430}, year={2021} } ``` Copied from https://github.com/Megvii-BaseDetection/YOLOX#cite-yolox --- # ZipDepth ZipDepth is a compact reparameterizable CNN distilled from Depth Anything V2 Large that predicts a dense relative inverse-depth map. LibreYOLO supports it for the depth task: predict and zero-shot validation, with no training path. Tasks: depth. Install: pip install libreyolo. Verified against LibreYOLO v1.5.0. ## Install ZipDepth needs no optional extra. Everything it imports is in the base install. ```bash pip install libreyolo ``` ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreZipDepthb-depth.pt") result = model(SAMPLE_IMAGE, save=True) depth = result.depth_map print(depth.min, depth.max, depth.mean) ``` **CLI** ```bash libreyolo predict model=LibreZipDepthb-depth.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True ``` **NPU/edge checkpoint** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # Same encoder, an unfold-free upsampling head for compilers that lack # gather/unfold support. Output is visually equivalent to the b checkpoint. model = LibreYOLO("LibreZipDepthbnpu-depth.pt") result = model(SAMPLE_IMAGE, save=True) ``` `result.depth_map` carries a dense relative inverse-depth map: higher values mean closer to the camera, and the values have no metric unit or cross-image scale. `save=True` writes a colormapped visualization of that map to disk; `Results.plot()` does not cover this family, since it is defined for surface normals and edges only. See [prediction](/docs/predict) for sources, streaming and result handling. ## Variants Two checkpoints, both the same encoder capacity, differing only in the trained upsampling head. `b` uses convex upsampling and runs on GPU or CPU. `bnpu` swaps in an unfold-free decoder for NPU and edge compilers that lack gather/unfold support; its output is documented as visually equivalent to `b`. Pick `bnpu` when the export target is a constrained runtime, `b` otherwise. Both checkpoints were distilled from Depth Anything V2 Large pseudo-labels, so this family is the compact, edge-oriented tier of LibreYOLO's depth task, alongside the larger Depth Anything V2 encoders. Training is not offered for this family. `LibreZipDepth.train()` raises `NotImplementedError` unconditionally: the upstream recipe distills pseudo-labels over a large image set that is not reproducible as a LibreYOLO training run. Train upstream at [fabiotosi92/ZipDepth](https://github.com/fabiotosi92/ZipDepth) and convert the result with `weights/convert_zipdepth_weights.py`. ## Validate `val()` runs the shared depth validator: it aligns each prediction to its ground truth with a per-image least-squares scale and shift, then reports the standard zero-shot relative-depth metrics, AbsRel, RMSE and the three delta thresholds. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreZipDepthb-depth.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/abs_rel"]) print(metrics["metrics/rmse"]) print(metrics["metrics/delta1"]) ``` **CLI** ```bash libreyolo val model=LibreZipDepthb-depth.pt data=my-dataset.yaml ``` ## Export | Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | depth | yes | yes | yes | yes | yes | | | | yes | | | yes | A "yes" means the export is supported. An empty cell means the exporter refuses that combination. Export follows a fixed-resolution dense contract: the source image is stretch-resized to the exported canvas, and the returned depth map is resized back to the original canvas afterward. An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.ncnn` file behaves like a checkpoint and returns the same `Results`, with `depth_map` in place of boxes. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreZipDepthb-depth.pt") model.export(format="onnx") model.export(format="ncnn") ``` **CLI** ```bash libreyolo export model=LibreZipDepthb-depth.pt format=onnx libreyolo export model=LibreZipDepthbnpu-depth.pt format=ncnn ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreZipDepthb-depth.onnx") result = model(SAMPLE_IMAGE) print(result.depth_map.data.shape) ``` ## Checkpoints Every published weight file for this family. | File | Input (px) | Task | Weights license | | --- | --- | --- | --- | | `LibreZipDepthb-depth.pt` | | depth | mit | | `LibreZipDepthbnpu-depth.pt` | | depth | mit | ## Licensing Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice. - Original work: ZipDepth, University of Bologna - Upstream license: MIT - Upstream source: https://github.com/fabiotosi92/ZipDepth - LibreYOLO code: MIT - Weights: MIT, republished at https://huggingface.co/LibreYOLO - Interpretation: MIT is a permissive license, so both checkpoints can be used in commercial and closed-source products with no redistribution restriction beyond keeping the license notice. It places no obligation on your own application code. The weights were trained by distilling pseudo-labels from Depth Anything V2 Large, whose Large checkpoint is itself CC-BY-NC-4.0; the ZipDepth authors publish the distilled student under MIT regardless, and LibreYOLO republishes that same MIT-licensed student. ## Citation ```bibtex @inproceedings{tosi2026zipdepth, title = {ZipDepth: Bringing Lightweight Zero-Shot Monocular Depth Anywhere, on Any Device}, author = {Tosi, Fabio and Bartolomei, Luca and Poggi, Matteo and Mattoccia, Stefano}, booktitle = {European Conference on Computer Vision (ECCV)}, year = {2026} } ``` Copied from https://github.com/fabiotosi92/ZipDepth#-citation --- # Ensembling detectors LibreEnsemble runs two or more detectors over the same decoded image and fuses their boxes into one Results object. Members keep their own weights, thresholds, devices and class lists. Verified against LibreYOLO v1.5.0. ## What an ensemble is `LibreEnsemble` takes two or more detectors, runs each on the same image, and fuses their boxes into a single `Results`. It is a prediction-time construct: there is nothing to train, and the members stay independent models that can be validated and exported on their own. Detection is the only task it supports. A member whose task is anything else raises `ValueError` at construction, naming the member index and its task. Both names are imported lazily, so they cost nothing until used: ```python from libreyolo import LibreEnsemble, ExternalDetector ``` ## Building one **Two detectors, fused** ```python from libreyolo import LibreEnsemble, SAMPLE_IMAGE # Members can be checkpoint paths or already-loaded models. ensemble = LibreEnsemble(["LibreYOLO9s.pt", "LibreRFDETRs.pt"]) result = ensemble(SAMPLE_IMAGE) for xyxy, conf, cls in zip( result.boxes.xyxy.tolist(), result.boxes.conf.tolist(), result.boxes.cls.tolist(), ): print(result.names[int(cls)], round(float(conf), 3), xyxy) ``` **Weights and a vote requirement** ```python from libreyolo import LibreEnsemble, SAMPLE_IMAGE ensemble = LibreEnsemble( ["LibreYOLO9s.pt", "LibreRFDETRs.pt"], weights=[1.0, 1.3], # by convention, proportional to validation mAP fusion="wbf", fusion_iou=0.55, min_votes=2, # keep only boxes both members found ) result = ensemble(SAMPLE_IMAGE) print(len(result.boxes), "agreed detections") ``` **Per-member thresholds** ```python from libreyolo import LibreEnsemble, SAMPLE_IMAGE ensemble = LibreEnsemble(["LibreYOLO9s.pt", "LibreRFDETRs.pt"]) # A scalar applies to every member; a list is read per member. result = ensemble(SAMPLE_IMAGE, conf=[0.3, 0.5], iou=0.5) print(len(result.boxes)) ``` ```python LibreEnsemble( members, *, weights=None, fusion="wbf", fusion_iou=0.55, min_votes=1, ) ``` `members` is a sequence of two or more. A `str` or `Path` entry is loaded through `LibreYOLO()`; anything else has to be callable and expose a `names` dict. Fewer than two raises `ValueError`, and passing a bare string raises `TypeError` rather than iterating its characters. `weights` defaults to `None`, which is uniform weighting. Supplied weights must be one per member and strictly positive, so a zero weight raises rather than silently dropping a member. The documented convention is to set them proportional to each member's validation mAP. `fusion_iou` defaults to `0.55` and is the IoU at which boxes from different members are clustered together. It is a different threshold from the per-call `iou`, which is each member's own NMS setting. `min_votes` defaults to `1`, meaning any single member can carry a box. Raising it keeps only clusters confirmed by that many distinct members. It must be a positive integer no larger than the member count, and it is capped per class to the number of members that actually know that class, so a class only one member was trained on is not silently erased. ## Fusion methods Three are accepted by name, and a callable is accepted too. | `fusion` | Behavior | |---|---| | `"wbf"` | Weighted boxes fusion, sequential and faithful to the paper [1]. The default | | `"wbf_seeded"` | One-pass weighted boxes fusion; class-aware NMS picks cluster seeds | | `"nms"` | Concatenate every member's boxes, then class-aware NMS | [1] Roman Solovyev, Weimin Wang, Tatiana Gabruseva, ["Weighted boxes fusion: Ensembling boxes from different object detection models"](https://arxiv.org/abs/1910.13302), arXiv:1910.13302. Weighted boxes fusion averages the coordinates of a cluster weighted by confidence, producing a box no single member proposed. The two weighted variants agree whenever clusters are unambiguous and can differ slightly on chains of overlapping clusters. `"nms"` picks a survivor instead of averaging, so survivors keep their original scores, and weights only influence which box wins. Because it selects rather than clusters, it cannot count votes: combining `fusion="nms"` with `min_votes` greater than `1` raises `ValueError`. Weighted boxes fusion rescales a cluster's score by the share of member weight that backed it. With two equally weighted members, a box only one of them found keeps half its score: `0.9` becomes `0.45`. A fused confidence can therefore fall below the `conf` each member was run at, so filter on the fused score rather than assuming the member threshold still holds. ## Members with different class lists Members do not have to share a class list. Their label spaces are unioned by name, and each member gets a lookup table remapping its own class ids into the union. `ensemble.names` is that union, and it is what the returned `Results` carries. Boxes only ever fuse within the same class name. A class that only one member knows passes through unfused, and it is not penalized for it: the score rescale uses a per-class denominator, so a solo-known class keeps its score. Partial overlap logs a warning naming the classes that are not shared by every member. That warning is the thing to read carefully, because a checkpoint whose class names are placeholders such as `class_0` builds a union that is disjoint from every other member, and no cross-member fusion happens at all. A member returning a class id outside its own `names` raises `RuntimeError`. ## Foreign detectors **Bringing in a detector LibreYOLO did not load** ```python from libreyolo import ExternalDetector, LibreEnsemble, SAMPLE_IMAGE def my_detector(pil_image): # Return (boxes, scores, labels): xyxy in original-image pixels. return ([[100.0, 100.0, 200.0, 300.0]], [0.9], [0]) external = ExternalDetector(my_detector, names={0: "person"}) ensemble = LibreEnsemble(["LibreYOLO9s.pt", external]) result = ensemble(SAMPLE_IMAGE) print(len(result.boxes)) ``` `ExternalDetector(fn, names)` wraps any callable that takes a PIL image and returns `(boxes, scores, labels)`, with boxes as xyxy in original-image pixels. It validates arity, box shape, length agreement and that every class id appears in `names`, and it applies the `conf` threshold itself. This is how a detector LibreYOLO did not load takes part in a fusion. ## Calling it **The same sources a single model takes** ```python from libreyolo import LibreEnsemble ensemble = LibreEnsemble(["LibreYOLO9s.pt", "LibreRFDETRs.pt"]) # Replace clip.mp4 with a video file on disk. for result in ensemble("clip.mp4", stream=True, vid_stride=2): print(result.frame_idx, len(result.boxes)) ``` The call signature mirrors a single model's, and it accepts the same sources: images, folders, lists, video, screen capture, webcams and network streams. Live sources require `stream=True` for the same reason they do elsewhere. | Argument | Default | Notes | |---|---|---| | `conf` | `0.25` | Per member; scalar broadcasts, or one per member | | `iou` | `0.45` | Each member's own NMS threshold, not the fusion threshold | | `imgsz` | `None` | A `list` is read per member; an `int` or tuple broadcasts | | `device` | `None` | Scalar or one per member, so members can sit on different devices | | `classes` | `None` | Filters the fused result, on union class ids | | `max_det` | `300` | Applies to the fused result | Because a `list` means per member for `imgsz`, `imgsz=[480, 640]` is 480 for the first member and 640 for the second, while `imgsz=(480, 640)` is one rectangular size for everyone. That distinction is easy to trip over. Members are called with a `max_det` of at least 300 regardless of what you ask for, so each runs generously and the ensemble trims once at the end. The image is decoded once and the same object is handed to every member. `batch` is accepted for parity and ignored; images are processed sequentially. ## What comes back An ordinary `Results`, the same type a single model returns, with `names` set to the union class space. Everything on [Working with results](/docs/predict/results) applies unchanged. The one difference is `result.speed`, which an ensemble does populate. Its keys are `member_0`, `member_1` and so on, plus `fusion`, in milliseconds. This is the one place in the library where `speed` is filled in. Rows carrying non-finite boxes or scores are dropped before fusion. When members sit on different devices, fusion runs on the device of the first member that returned anything. ## What an ensemble cannot do `val()` and `export()` both raise `NotImplementedError` and point you at the members: validate and export each one individually. There is no `train` method at all, so calling it raises `AttributeError`. Half precision is not handled at the ensemble level. `half=True` hits the same warned no-op path it does everywhere else; configure precision on each member. There is no command line interface for ensembling. It is a Python API. --- # Inference performance Five prediction-time controls change throughput or accuracy: CUDA graph replay, precision, batching, tiling and test-time augmentation. Each applies to a specific set of families, and two of them cost accuracy or latency rather than saving it. Verified against LibreYOLO v1.5.0. ## The controls and their defaults Every one of these is an argument to `predict`, and every default is off. | Argument | Default | Effect | |---|---|---| | `batch` | `1` | Images per forward pass, for folder and list sources | | `cuda_graph` | `False` | Replay the forward from a captured CUDA graph | | `tiling` | `False` | Split a large image into overlapping tiles | | `overlap_ratio` | `0.2` | Tile overlap when `tiling` is on | | `augment` | `False` | Run flipped views and merge them | | `half` | | Accepted, warned, and ignored | | `device` | `None` | Move the model before predicting | `imgsz` also affects cost, since it sets the resolution the model runs at, but it is an accuracy argument first and belongs with the model rather than here. ## Batching **Batched inference over a folder** ```python from pathlib import Path from PIL import Image from libreyolo import LibreYOLO, SAMPLE_IMAGE folder = Path("batch_demo") folder.mkdir(exist_ok=True) image = Image.open(SAMPLE_IMAGE) for index in range(8): image.save(folder / f"frame_{index}.jpg") model = LibreYOLO("LibreYOLO9s.pt") # One stacked forward per chunk of 4 on families that support it. results = model(str(folder), batch=4) print(len(results), "results") ``` **Streaming, so the list never materializes** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") for result in model("batch_demo", batch=4, stream=True): print(len(result.boxes)) ``` **CLI** ```bash libreyolo predict model=LibreYOLO9s.pt source=batch_demo batch=4 ``` `batch` applies to folder and list sources. With `batch=1`, images run one forward pass each. Above `1`, each chunk is preprocessed, stacked into a single tensor, run once, then sliced back so every family's existing single-image postprocess sees what it expects. The stacked path is taken only when all of these hold: - `batch` is greater than `1` - `tiling` is off - test-time augmentation is not active - the family sets `SUPPORTS_BATCHED_PREDICT` - the underlying network is not in training mode The last condition is not a technicality. A network in training mode would normalize the stacked chunk with cross-image batch statistics, letting images in the same chunk change each other's predictions, so those runs stay sequential. `SUPPORTS_BATCHED_PREDICT` defaults to true. These families opt out and run one image per forward regardless of `batch`: Depth Anything V2, Depth Anything 3, EoMT, Faster R-CNN, FCOS, HRNet, L2CS-Net, LibreMODUS, MiDaS, MoGe-2, PP-OCRv5, Real-ESRGAN, RetinaNet, SAM 3D Body, SwinIR, YOLOv1, ZipDepth, every open-vocabulary detector, and every vision language model. There is one more fallback. If preprocessing does not return uniform `(1, C, H, W)` tensors of matching shape, dtype and device across the chunk, the chunk runs sequentially rather than stacking, so correctness never depends on the images happening to be the same size. Combine `batch` with `stream=True` on a large folder to get batched forwards without holding every result in memory. ## CUDA graphs **Capture up front, then replay (needs CUDA)** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt", device="cuda") # Pay warmup and capture once, off the first request. model.capture_graph() result = model(SAMPLE_IMAGE, cuda_graph=True) print(len(result.boxes)) print(model.graph_info()) ``` **Capture only once a shape repeats (needs CUDA)** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt", device="cuda") # "auto" waits for a shape to be seen twice, so one-shot work # never pays for capture. for _ in range(3): model(SAMPLE_IMAGE, cuda_graph="auto") print(model.graph_info()) model.release_graphs() ``` A CUDA graph records a forward pass once and replays it as a single launch. Small detectors spend a large share of batch-1 time launching kernels, so collapsing those launches is a throughput win, and replay output is bit identical to eager execution. `cuda_graph` takes three values. `False` is the default and does nothing. `True` captures on first use for each input shape. `"auto"` waits until a shape repeats before capturing, so one-shot and shape-varying work never pays the capture cost. `capture_graph(imgsz=None, batch=1, dtype=None)` moves that cost off the first request. A graph is valid only for the exact shape it captured, so `batch` here has to match how `predict` is later called. `graph_info()` reports the captured graphs, replay counts, and any reason the run fell back to eager. `release_graphs()` frees them and their static buffers. Capture requires CUDA and a family that has opted in through `SUPPORTS_CUDA_GRAPH`, because it needs a forward with no host-visible work and that is verified per family. Asking for it on a family that has not opted in raises `NotImplementedError` rather than silently running eager. A graph records memory addresses, not values, so anything that relocates parameters drops it. Changing device through `predict(device=...)`, quantizing and dequantizing all invalidate captured graphs. The full per-family support matrix, the seam splits and the numerics contract are on [CUDA graphs](/docs/reference/cuda-graphs). ## Precision **Install the export extra** ```bash pip install "libreyolo[onnx]" ``` **Export and load back, at the default precision** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt") path = model.export(format="onnx") exported = LibreYOLO(path) result = exported(SAMPLE_IMAGE) print(len(result.boxes)) ``` **FP16 export (build and run this on a CUDA machine)** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt", device="cuda") path = model.export(format="onnx", half=True) exported = LibreYOLO(path) result = exported(SAMPLE_IMAGE) print(len(result.boxes)) ``` **FP16 in PyTorch, via a cast recipe (needs CUDA)** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt", device="cuda") # A cast recipe reads no calibration data. model.quantize(recipe="fp16", calib=None) result = model(SAMPLE_IMAGE) print(len(result.boxes)) ``` `half=True` at predict time does nothing. It is accepted for command line compatibility, raises a warning saying it is a no-op, and is discarded before it reaches any family. The CLI's `--half` flag prints the same warning for a `.pt` model. There are two real routes to lower precision. For an exported artifact, precision is chosen at export time with `export(format=..., half=True)`, and the resulting file loads back through `LibreYOLO()` unchanged. For PyTorch execution, `model.quantize(recipe="fp16")` casts the model to float16 and installs hooks that keep float32 at the model's inputs and outputs. `"bf16"` does the same with bfloat16. Neither cast reads calibration data, so `calib` is ignored for them. Quantization currently covers four families: YOLOv9, RF-DETR, BiRefNet and FeyNobg. A cast on a CPU device logs a warning that it will be slow, so these recipes are meant for a GPU. Both routes change numerics. Neither is a drop-in guarantee of the same detections, so validate before deploying. ## Tiled inference **Tiled inference on a large image** ```python from PIL import Image from libreyolo import LibreYOLO, SAMPLE_IMAGE # Tiling only engages when the image is larger than the input size. large = Image.open(SAMPLE_IMAGE).resize((2048, 1536)) large.save("large.jpg") model = LibreYOLO("LibreYOLO9s.pt") result = model("large.jpg", tiling=True, overlap_ratio=0.2) print(result.num_tiles, "tiles", len(result.boxes), "detections") ``` Tiling crops a large image into overlapping square tiles, predicts on each, and merges the results. It is the option for small objects in high-resolution images, where a whole-image resize shrinks the targets below what the model can resolve. Tile size is the model's input size, or `imgsz` when given, and it has to be square. `overlap_ratio` defaults to `0.2`. Tiles that overlap are reconciled with per-class non-maximum suppression at the `iou` threshold, and the merged list is then truncated to `max_det`. This means `iou` has an effect on tiled predictions even for families that run no NMS of their own. Tiling is skipped, not merely cheap, when the image already fits: if both dimensions are at or below the input size, one ordinary forward runs instead. It is also skipped for classification, semantic segmentation and the `embed` task, which fall back to a single pass because tiling has no meaning there. It raises for tasks whose payload cannot be stitched back together: instance segmentation masks, oriented boxes, points, depth, edges and normals. It cannot be combined with `augment`. The result carries `result.tiled` and `result.num_tiles`. With `save=True`, tiled runs write a directory under `runs/tiled_detections` holding every tile, the annotated image, a grid visualization, and a `metadata.json` recording the tile size, overlap and thresholds, with `result.tiles_path` and `result.grid_path` pointing at them. ## Test-time augmentation **Test-time augmentation** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt") plain = model(SAMPLE_IMAGE) flipped = model(SAMPLE_IMAGE, augment=True) print(len(plain.boxes), "->", len(flipped.boxes)) ``` `augment=True` runs the image more than once and merges the detections with per-class non-maximum suppression at the `iou` threshold. Like tiling, this makes `iou` load-bearing for families that otherwise ignore it. In practice this is horizontal flipping. The scale list `TTA_SCALES` defaults to a single scale of `1.0` and no shipped family overrides it, so every family runs two passes: the original image and its mirror. Families marked `TTA_FIXED_SIZE` resize to a fixed square, which makes multi-scale a no-op for them in any case. Semantic and panoptic segmentation take a different merge. Their flipped view is flipped back and the two softmax distributions are averaged before the argmax, rather than being merged as boxes. Test-time augmentation is not available for every task. It raises for oriented boxes, pose, points, depth, normals, edges, restoration, OCR and embedding models, and cannot be combined with tiling. These families disable it outright, so `augment=True` runs a single ordinary pass: BiRefNet, CenterNet, CLIP, DexiNed, FOMO, HRNet, L2CS-Net, LibreMODUS, NAFNet, PP-OCRv5, Real-ESRGAN, RetinaNet, SAM 3D Body, SigLIP2, SwinIR, TEED, every SAM variant, every open-vocabulary detector, and every vision language model. ## Measuring Nothing on this page carries a latency number, because a millisecond without its hardware, runtime, precision and batch size is not a fact. Measured figures across hardware and runtimes are published at [visionanalysis.org](https://www.visionanalysis.org), and `libreyolo profile` measures a specific model on the machine in front of you. --- # Working with results Every prediction returns a Results object per image. It has one named slot per kind of payload, all of them empty except the ones the model produces, plus the same slots on an exported artifact. Verified against LibreYOLO v1.5.0. ## One object, one slot per payload A prediction on one image returns one `Results`. It carries eighteen payload slots, and a model fills only the ones its task produces. Every other slot is `None`, so reading `result.masks` on a detector is `None` rather than an error. | Slot | Class | Shape | Produced by | |---|---|---|---| | `boxes` | `Boxes` | `(N, 4)` plus scores and classes | Detection, and any task that localizes first | | `masks` | `Masks` | `(N, H, W)` | Instance segmentation | | `keypoints` | `Keypoints` | `(N, K, 2)` or `(N, K, 3)` | Pose | | `probs` | `Probs` | `(C,)` | Classification | | `obb` | `OBB` | `(N, 7)` or `(N, 8)` | Oriented boxes | | `gaze` | `Gaze` | `(N, 2)` pitch and yaw in radians | Gaze estimation | | `points` | `Points` | `(N, 4)` as x, y, class, confidence | Point localization | | `semantic_mask` | `SemanticMask` | `(H, W)` class ids | Semantic segmentation | | `panoptic` | `PanopticSegmentation` | `(H, W)` segment ids plus `segments_info` | Panoptic segmentation | | `depth_map` | `DepthMap` | `(H, W)` floats | Depth estimation | | `normal_map` | `NormalMap` | `(H, W, 3)` unit vectors | Surface normals | | `edges` | `EdgeMap` | `(H, W)` floats in `[0, 1]` | Edge detection | | `restored` | `RestoredImage` | `(H, W, 3)` uint8 RGB | Restoration and super-resolution | | `matte` | `Matte` | `(H, W)` floats in `[0, 1]` | Alpha matting and background removal | | `ocr` | `OCRRegions` | `(N, 4, 2)` polygons plus transcripts | Text detection and recognition | | `embeddings` | `Embeddings` | `(N, D)` L2-normalized rows | The `embed` task | | `identities` | `Identities` | N names and scores | The `embed` task with a gallery | | `meshes` | `Meshes` | Body parameters and optional vertices | Body mesh recovery | Alongside them sit the fields every result has: `orig_shape` as `(height, width)`, `path` (the source path, or `None` for in-memory input), `names` mapping class id to class name, `frame_idx` for video and live frames, `track_id` when tracking, and `restore_scale`, the integer upscale factor of a restoration result. `result.normals` is an alias for `result.normal_map`. `result.speed` exists on every result but is populated only by [ensembles](/docs/predict/ensembling), where its keys are `member_0`, `member_1` and `fusion` in milliseconds. For a single model it stays an empty dict. ## Boxes **Boxes** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt") result = model(SAMPLE_IMAGE) print(result.orig_shape) # (height, width) of the source image print(result.path) # source path, None for in-memory input for xyxy, conf, cls in zip( result.boxes.xyxy.tolist(), result.boxes.conf.tolist(), result.boxes.cls.tolist(), ): print(result.names[int(cls)], round(float(conf), 3), xyxy) ``` **Normalized coordinates** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy[:1]) # pixels, x1 y1 x2 y2 print(result.boxes.xywh[:1]) # pixels, center x, center y, w, h print(result.boxes.xyxyn[:1]) # same box divided by width and height print(result.boxes.xywhn[:1]) ``` **NumPy and devices** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt") result = model(SAMPLE_IMAGE) # Each of these returns a new Results; the original is unchanged. as_numpy = result.numpy() on_cpu = result.cpu() print(type(as_numpy.boxes.xyxy).__name__) print(type(on_cpu.boxes.xyxy).__name__) ``` `Boxes` keeps coordinates and scores as separate arrays rather than one packed tensor. | Attribute | Contents | |---|---| | `xyxy` | `(N, 4)` absolute pixels, x1 y1 x2 y2 | | `xywh` | `(N, 4)` absolute pixels, center x, center y, width, height | | `xyxyn`, `xywhn` | The same divided by image width and height | | `conf` | `(N,)` confidence | | `cls` | `(N,)` class id, as a float | | `id` | `(N,)` track id, or `None` | | `is_track` | Whether `id` is set | | `data` | Everything concatenated: boxes, optional id, conf, cls | `cls` is a float array, so use it as `result.names[int(cls)]`. `xyxyn` and `xywhn` need `orig_shape`, which `Results` fills in for you. ## Dense payloads Payloads covering the whole image behave differently from per-instance ones, and it matters when slicing. `SemanticMask` holds `(H, W)` class ids on the original canvas, with `255` reserved as the ignore value that never counts as a class. `classes` lists the ids present and excludes it; `class_mask(id)` returns a boolean `(H, W)`. `PanopticSegmentation` holds `(H, W)` segment ids, with `0` as the void id, and a `segments_info` list of dicts carrying at least `id` and `category_id`. `segment_ids` lists the ids present, `segment_mask(id)` selects one. `DepthMap` holds `(H, W)` relative inverse depth: higher means closer, and the values are not metric meters. It exposes `min`, `max`, `mean` over finite values, and `normalized()` rescaling to `[0, 1]`. `NormalMap` holds `(H, W, 3)` unit vectors in the OpenCV camera frame, with `+x` right, `+y` down and `+z` into the scene, so a surface facing the camera is `(0, 0, -1)`. `assert_normalized()` checks every pixel is finite and unit length. `EdgeMap` holds `(H, W)` float32 in `[0, 1]`. The continuous map is kept rather than thresholded, so `binary(threshold=0.5)` is where you choose a cutoff. `Matte` holds `(H, W)` float32 in `[0, 1]`, where `1` is fully foreground. `array` returns it clipped as float32. `RestoredImage` holds `(H, W, 3)` uint8 RGB, with `array` for the raw ndarray and `save(path)` to write it. `Probs` holds one probability vector for the image. `top1` and `top5` are class indices, `top1conf` and `top5conf` the matching scores. `Embeddings` holds `(N, D)` rows that are already L2-normalized, so cosine similarity is a dot product. `similarity(other)` returns `(N, M)` against a gallery or `(N,)` against a single vector, and `verify(i, j, threshold=0.4)` compares two rows. `OCRRegions` holds `(N, 4, 2)` polygons in reading order, corners ordered top-left, top-right, bottom-right, bottom-left. Transcripts are in `texts`, recognition scores in `conf`, detection scores in `det_conf`. Because these are genuine rotated polygons they do not populate `boxes`; `ocr.xyxy` gives axis-aligned hulls when you need rectangles. ## Slicing and moving `result[i]` returns a new `Results` holding one instance. Per-instance payloads are sliced; whole-image payloads are carried through unchanged, so slicing a classification result cannot truncate its probability vector to a single class, and slicing a depth result cannot corrupt the `(H, W)` layout. `len(result)` counts instances: boxes, points, embeddings, OCR regions or meshes. Any dense whole-image payload counts as `1`. A result with nothing in it is `0`. `to()`, `cpu()`, `cuda()` and `numpy()` each return a new `Results` with every populated slot converted. They do not modify the original. `update()` is the one method that mutates in place, replacing named slots and returning the same object. ## JSON **summary and to_json** ```python import json from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt") result = model(SAMPLE_IMAGE) rows = result.summary() print(json.dumps(rows[:2], indent=2)) # Same content as a string, with the same keyword arguments. print(result.to_json(normalize=True, decimals=3)[:200]) ``` **CLI** ```bash libreyolo predict model=LibreYOLO9s.pt --json \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` `summary()` returns a list of plain dicts, and `to_json()` is that list passed through `json.dumps`. Both take the same three arguments: `normalize=False` switches coordinates to `[0, 1]`, `decimals=5` sets rounding, and `embeddings=False` controls whether embedding vectors are included. The row shape follows the payload. Detection rows carry `name`, `class`, `confidence` and a `box` dict, and pick up `segments` when masks are present, `obb` and `corners` for oriented boxes, `gaze` angles in both radians and degrees, `track_id` when tracking, and `mesh` parameters when meshes are present. Where there are no boxes, one payload decides the rows: OCR emits one row per region with its `text`, points one row per point, panoptic one row per segment with `pixel_count` and `pixel_fraction`, semantic one row per class present, classification the top five classes. Depth, normals, edges, restoration and matting each emit a single summary row describing the map rather than its pixels. Two payloads are deliberately abbreviated. An embedding vector is reported as `embedding_dim` only, because a 512-float row is about 2 KB per face; pass `embeddings=True` to include the values. Mesh vertices are never included at all, since that is tens of thousands of coordinates per person. Read `result.meshes.vertices` or call `result.meshes.save_obj(path)` for geometry. ## Drawing and saving **Annotated images** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt") # save=True draws the payload and writes it under runs/detect/predict*. result = model(SAMPLE_IMAGE, save=True) print(result.saved_path) ``` `predict(save=True)` is the path that annotates and writes. It picks the drawing routine from whichever slot is filled, so a semantic result is written as a colored mask, a depth result as a depth visualization, a panoptic result with its segments, a matte as a transparent-background RGBA PNG, and a detector as boxes with masks underneath them. The written path is attached to the result as `result.saved_path`. `Results.plot()` is narrower than its name suggests. It is defined for normal maps and edge maps only, and raises `NotImplementedError` for anything else. Use `save=True` for the other tasks. `Results.save(path)` is likewise narrow: it writes a matte result as a transparent-background RGBA PNG cutout and raises `NotImplementedError` otherwise. `Results.cutout()` returns that same RGBA array without writing it. Both need the source image, taken from `result.path` or passed as `image=`. Two payloads carry their own writers: `result.restored.save(path)` for a restored image, and `result.meshes.save_obj(path, index=0)` for a mesh. For where files land and how `output_path` and `output_file_format` behave, see [Prediction sources](/docs/predict/sources). ## Exported artifacts return the same object **Install the export extra** ```bash pip install "libreyolo[onnx]" ``` **The same Results from an exported artifact** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt") path = model.export(format="onnx") # returns the written path # LibreYOLO() dispatches on the file suffix. exported = LibreYOLO(path) result = exported(SAMPLE_IMAGE) print(type(result).__name__, len(result.boxes)) ``` `LibreYOLO()` dispatches on the file suffix, so an exported artifact loads through the same call as a `.pt` checkpoint and returns the same `Results`. `.onnx`, `.engine`, `.pte` and `.mnn` files are recognized by suffix, as are OpenVINO, Paddle and ncnn directories and a Triton model URL. Code that reads `result.boxes.xyxy` does not change when a model is swapped for its exported build. See [Export](/docs/export) for the full set of formats. Reaching for the runtime's own API instead means owning preprocessing, postprocessing and class names yourself. --- # Prediction sources The source argument is classified before anything is opened, so one call handles a JPEG, a folder, an MP4, a webcam index, an RTSP URL, a screen region, or a list of cameras. Verified against LibreYOLO v1.5.0. ## How a source is classified `classify_source` inspects the value before anything is opened or downloaded, in this order. The first rule that matches wins. | Source | Read as | |---|---| | `"screen"`, `"screen 1"`, `"screen 1 100 200 512 256"` | Screen capture | | A non-negative `int`, or a digit string with no file of that name | Webcam | | An `rtsp://`, `rtmp://`, `tcp://` or `udp://` URL | Network stream | | An `http(s)://` URL whose path ends in `.m3u8` | Network stream | | A YouTube page URL | Network stream | | A list or tuple whose entries are all live or video | Several live streams | | Any other list or tuple | Batch of images | | A path ending in `.streams` | Several live streams | | A path with a video extension | Video file | | An existing directory | Folder of images | | Anything else | Single image | A list that mixes live sources with images raises `TypeError`. A negative webcam index raises `ValueError`. The classifier never touches the network, so a mistyped URL surfaces when the capture opens, not when `predict` is called. ## Images **One image** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt") # A single image source returns one Results, not a list. result = model(SAMPLE_IMAGE) print(len(result.boxes), "detections") ``` **In-memory images** ```python import numpy as np from PIL import Image from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt") pil_image = Image.open(SAMPLE_IMAGE) array = np.asarray(pil_image) raw_bytes = open(SAMPLE_IMAGE, "rb").read() for source in (pil_image, array, raw_bytes): result = model(source) print(type(source).__name__, len(result.boxes)) ``` **A folder** ```python from pathlib import Path from PIL import Image from libreyolo import LibreYOLO, SAMPLE_IMAGE folder = Path("sample_folder") folder.mkdir(exist_ok=True) image = Image.open(SAMPLE_IMAGE) for index in range(3): image.save(folder / f"frame_{index}.jpg") model = LibreYOLO("LibreYOLO9s.pt") # A folder returns a list, one Results per image, sorted by path. results = model(str(folder)) print(len(results), "images") ``` **CLI** ```bash libreyolo predict model=LibreYOLO9s.pt \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` A single image source accepts seven types. | Type | Read as | |---|---| | `str` or `pathlib.Path` | Local file, `http(s)://`, `s3://` or `gs://` | | `PIL.Image.Image` | Converted to RGB | | `numpy.ndarray` | 2D grayscale, or 3D HWC or CHW; a 4D array uses its first image | | `torch.Tensor` | CHW or NCHW, read as RGB; a batched tensor uses its first image | | `bytes` | Encoded image data | | `io.BytesIO` | Encoded image data | Everything is converted to RGB before preprocessing. NumPy arrays are the one case where channel order is ambiguous, so `color_format` controls it: `"auto"` (the default) leaves the array as-is, `"bgr"` reverses the channels, which is what a frame read with OpenCV needs. Float arrays are rescaled by their own range: values at or below `1.0` are multiplied by 255, higher values are clipped into `[0, 255]`. An RGBA array drops its alpha channel. Remote paths need one package each, and none of them is installed by default: `requests` for `http(s)://`, `boto3` for `s3://`, and `gcsfs` for `gs://`. ## Folders A directory is scanned recursively and sorted, and every file with one of these suffixes becomes an image: `.jpg`, `.jpeg`, `.png`, `.gif`, `.webp`, `.bmp`, `.tiff`, `.tif`. Anything else in the folder is skipped. An empty folder returns an empty list rather than raising. Folders and lists are the two sources that accept `batch`, which runs one stacked forward pass per chunk on families that support it. See [Inference performance](/docs/predict/performance). ## Video files **A video file (supply your own clip)** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") # Replace clip.mp4 with a video file on disk. for result in model("clip.mp4", stream=True): print(result.frame_idx, len(result.boxes)) ``` **Every third frame, written to disk** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") for result in model("clip.mp4", stream=True, vid_stride=3, save=True): pass ``` A path counts as video when its suffix is one of `.asf`, `.avi`, `.gif`, `.m4v`, `.mkv`, `.mov`, `.mp4`, `.mpeg`, `.mpg`, `.ts`, `.wmv`, `.webm`. `.gif` appears in both lists. A `.gif` path passed directly to `predict` is opened as video, because the video check runs first; a `.gif` sitting inside a scanned folder is loaded as a still image. `vid_stride` processes every N-th frame and defaults to `1`. Without `stream=True` the whole video is decoded into a list, and anything above 500 frames after striding emits a warning suggesting `stream=True`. Each `Results` from a video carries `frame_idx`. ## Webcams, network streams and YouTube **Webcam (needs a camera attached)** ```python import itertools from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") # Webcam index 0. Live sources never end, so bound the loop. for result in itertools.islice(model(0, stream=True), 100): print(result.frame_idx, len(result.boxes)) ``` **RTSP (needs a reachable camera URL)** ```python import itertools from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") source = "rtsp://user:password@192.168.1.64:554/Streaming/Channels/101" for result in itertools.islice(model(source, stream=True), 100): print(result.frame_idx, len(result.boxes)) ``` Live sources are unbounded, so they require `stream=True`. Without it, `predict` raises `ValueError` rather than trying to collect an endless list. Frames are read on a background thread, one per capture. By default the queue holds only the newest frame, so a model slower than the camera skips frames instead of falling behind. `stream_buffer=True` keeps every captured frame, which preserves them at the cost of growing latency. A webcam index is an `int` or a digit string. On Windows the capture is opened through the DirectShow backend first and falls back to the default backend if that fails. YouTube page URLs are resolved to a direct media URL without downloading the video, which needs `yt-dlp`: ```bash pip install "libreyolo[stream]" ``` Stream labels are redacted before they are logged or used as filenames. A URL carrying credentials appears as `user:***@host`, and query strings are dropped from direct stream labels because signed URLs and bearer tokens live there. A YouTube video id is kept, since it is not a credential. ## Several cameras at once **A .streams file (supply your own cameras)** ```python import itertools from pathlib import Path from libreyolo import LibreYOLO Path("cameras.streams").write_text( "# one source per line, blank lines and comments are skipped\n" "rtsp://192.168.1.64:554/Streaming/Channels/101\n" "rtsp://192.168.1.65:554/Streaming/Channels/101\n", encoding="utf-8", ) model = LibreYOLO("LibreYOLO9s.pt") for result in itertools.islice(model("cameras.streams", stream=True), 100): print(result.frame_idx, len(result.boxes)) ``` **A list of cameras** ```python import itertools from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") cameras = [0, "rtsp://192.168.1.64:554/Streaming/Channels/101"] for result in itertools.islice(model(cameras, stream=True), 100): print(result.frame_idx, len(result.boxes)) ``` A `.streams` file is one source per line. Blank lines and lines starting with `#` are ignored. Every remaining line must itself be a webcam index, a network stream, a YouTube URL or a video file path; anything else raises `ValueError` naming the line number. An empty file raises rather than starting with no cameras. A list or tuple of live sources does the same thing without a file. Each capture gets its own thread, and frames from all of them are multiplexed into one generator. Every pass polls each active stream and yields whatever is ready, so a slow camera does not hold up a fast one, and frames from different cameras interleave. A stream that ends drops out of the rotation while the others continue. ## Screen capture **One screenshot (needs mss and a desktop session)** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") # Without stream=True this grabs a single frame. result = model("screen") print(len(result.boxes), "detections") ``` **A region of one monitor, continuously** ```python import itertools from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") # "screen " for result in itertools.islice(model("screen 1 100 200 512 256", stream=True), 50): print(len(result.boxes)) ``` A screen source is the word `screen` followed by zero, one, four or five integers. Any other count raises `ValueError`. | Form | Captures | |---|---| | `"screen"` | Every monitor, merged | | `"screen 1"` | Monitor 1 | | `"screen 100 200 512 256"` | A box on the merged desktop | | `"screen 1 100 200 512 256"` | A box on monitor 1 | Box coordinates are `left top width height`, relative to the top-left corner of the chosen monitor. A screen source reports its frame rate as 30 divided by `vid_stride`, which is the rate a saved video is written at. Capture needs the `mss` package: ```bash pip install mss ``` Without `stream=True`, a screen source grabs one frame and returns a single `Results`, which is the screenshot equivalent of predicting on an image file. With `stream=True` it captures until the loop is broken. ## What predict returns The shape of the return value depends on the source and on `stream`. | Source | `stream=False` | `stream=True` | |---|---|---| | Single image | One `Results` | Generator of one `Results` | | List of images | List of `Results` | Generator | | Folder | List of `Results` | Generator | | Video file | List of `Results` | Generator | | Screen | One `Results` | Generator, unbounded | | Webcam, network stream, `.streams` | `ValueError` | Generator, unbounded | A single image returns the `Results` object itself. Indexing it selects a detection, not an image, so `result[0]` on a single-image prediction is the first box rather than the first picture. For what those objects carry, see [Working with results](/docs/predict/results). ## Where save writes `save=True` writes annotated output next to a run directory rather than returning it. Images go to an auto-incrementing `runs/detect/predict`, `runs/detect/predict2` and so on, keeping the source filename. Every image in one process lands in the same directory, so two input folders holding the same filename overwrite each other. In-memory images have no filename to reuse and are numbered `image0`, `image1` and so on. Video and live sources are written as a single `.mp4` named after the source. `output_path` overrides the directory. A path with a suffix is treated as a file, a path without one as a directory. `output_file_format` selects the still-image encoding and accepts `jpg`, `png` or `webp`. After a save, the written path is also attached to the result as `result.saved_path`. --- # Thresholds and filtering Four arguments decide which predictions survive: conf, iou, max_det and classes. Only two of them apply to every family, because a set predictor decodes a fixed query set and never runs NMS. Verified against LibreYOLO v1.5.0. ## The four arguments | Argument | Default | Applies to | |---|---|---| | `conf` | `0.25` | Every family | | `iou` | `0.45` | Families that run non-maximum suppression | | `max_det` | `300` | Every family | | `classes` | `None` | Every family | **The four arguments** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt") result = model( SAMPLE_IMAGE, conf=0.25, # keep predictions at or above this score iou=0.45, # NMS overlap threshold, where NMS runs max_det=300, # cap per image classes=None, # or a list of class ids ) print(len(result.boxes)) ``` **Sweeping conf** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt") for conf in (0.1, 0.25, 0.5, 0.75): result = model(SAMPLE_IMAGE, conf=conf) print(conf, len(result.boxes)) ``` **CLI** ```bash libreyolo predict model=LibreYOLO9s.pt conf=0.4 iou=0.5 max_det=100 \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` Two of these are universal and two are not, which is the single most useful thing to know before tuning anything. Validation uses different defaults on purpose: `val()` runs at `conf=0.001` and `iou=0.6`, because average precision is computed over a full precision-recall curve and a 0.25 cutoff would truncate it. ## conf `conf` is the score below which a prediction is discarded. It applies to every family, including the ones that never run NMS, and it is the first control to reach for when there are too many or too few detections. The default of `0.25` suits looking at pictures. Feeding a downstream system usually wants it higher; measuring accuracy wants it far lower. ## iou `iou` is the overlap above which non-maximum suppression removes the lower-scoring of two boxes of the same class. It only means something if the family runs suppression at all. A set predictor decodes a fixed number of queries and takes the top scoring ones. Duplicates are suppressed inside the architecture during training, not by a postprocessing step, so there is no threshold to turn. These families accept `iou` for API parity and ignore it: CenterNet, DEIM, DETR, Deformable DETR, D-FINE, DINO-DETR, EdgeCrafter, Faster R-CNN, LW-DETR, Mask R-CNN, RF-DETR, RT-DETR, and the end-to-end YOLOv9 head. Variants built on those decoders inherit the behavior. **iou on a family that runs no NMS** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # RF-DETR decodes a fixed query set, so iou changes nothing here. model = LibreYOLO("LibreRFDETRs.pt") loose = model(SAMPLE_IMAGE, iou=0.9) tight = model(SAMPLE_IMAGE, iou=0.1) # Same count either way. conf and max_det are the controls that work. print(len(loose.boxes), len(tight.boxes)) ``` Most of them say so in their postprocessing docstrings, but no warning is raised at runtime, so a sweep over `iou` on RF-DETR produces a flat line rather than an error. Faster R-CNN and Mask R-CNN are a slightly different case: both already ran NMS inside the model, at a fixed upstream threshold that `iou` has no supported way to change. These families do use it: YOLOv1 through YOLOv4, YOLOv7, YOLOv9, YOLOX, YOLO-NAS, RTMDet, PicoDet, EfficientDet, FCOS, RetinaNet and SSD. Two prediction-time options make `iou` matter even for a set predictor, because both merge boxes after the model has finished: - `tiling=True` reconciles overlapping tiles with per-class NMS at `iou` - `augment=True` merges flipped views with per-class NMS at `iou` Both are covered in [Inference performance](/docs/predict/performance). Open-vocabulary detectors have their own rule. A family whose processor runs NMS declares its own default threshold and honors `iou`, which is the case for OMDet-Turbo. Families that suppress nothing, Grounding DINO, OWLv2 and OV-DEIM, emit a warning when `iou` is passed. That warning is the only one of its kind in the library. ## max_det `max_det` caps how many predictions come back for one image. It applies everywhere, but through different mechanisms: an NMS family truncates after suppression, a set predictor uses it as the size of its top-k selection. Some families clamp below whatever you ask for, because their upstream reference configuration does. SSD caps at 200, RTMDet instance segmentation at 100, and FCOS at its own per-image detection limit. Raising `max_det` past those has no effect. The one place `max_det` is applied centrally rather than per family is tiled inference, where the merged list is truncated after tiles are reconciled. ## Class filtering **Filtering to specific classes** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt") # Class ids index model.names. On COCO, 0 is person. result = model(SAMPLE_IMAGE, classes=[0]) print({result.names[int(c)] for c in result.boxes.cls.tolist()}) ``` **Finding the id for a name** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9s.pt") result = model(SAMPLE_IMAGE) wanted = {"person", "backpack"} ids = [i for i, name in result.names.items() if name in wanted] print(ids) filtered = model(SAMPLE_IMAGE, classes=ids) print(len(filtered.boxes)) ``` `classes` takes a list of class ids and keeps only predictions whose class is in it. Ids index `result.names`, and the surest way to get one is to read `names` off a result rather than assuming a dataset ordering. Filtering happens centrally, after each family's postprocessing, in the single funnel every prediction path goes through. That has two consequences worth knowing. It works on every family, including the ones with no NMS. And it also filters the payloads aligned with the boxes, so masks, keypoints and oriented boxes are cut down alongside them rather than left mismatched. On the command line, `classes` accepts a bare integer, a list, or a comma-separated string: ```bash libreyolo predict model=LibreYOLO9s.pt classes=0 source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg libreyolo predict model=LibreYOLO9s.pt classes="[0,2,5]" source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` Filtering is not free accuracy. A model still spends its budget predicting classes you then discard, and `max_det` is applied by the family before the filter, so an image crowded with unwanted classes can hit the cap before your class is reached. Lower `conf` or raise `max_det` if that happens. ## agnostic_nms `agnostic_nms` is accepted and does nothing. Passing it raises a warning saying it is a no-op for command line compatibility, and the argument is discarded. There is no class-agnostic suppression mode. Every NMS call in the library is class-aware, so two overlapping boxes of different classes both survive, at any `iou`. Where that is a problem, filter with `classes` first, or suppress across classes yourself on `result.boxes`. ## What predict rejects Two arguments raise instead of warning: `visualize` and `embed` both raise `NotImplementedError`. For embeddings, load the model with `task="embed"` and call `predict` or `embed` normally. Anything unrecognized raises `TypeError` naming the supported options, so a typo fails immediately rather than being silently ignored. These are accepted, warned about and discarded: `agnostic_nms`, `boxes`, `dnn`, `half`, `line_width`, `retina_masks`, `show_conf`, `show_labels` and `verbose`. --- # Quickstart The shortest path through LibreYOLO: predict on one image, train on a small dataset, then export the result. Every command here runs on CPU. Verified against LibreYOLO v1.5.0. ## Install ```bash pip install libreyolo ``` That is everything the predict and train sections below need. Export to ONNX adds one extra; see [install](/docs/install) for the full list. ## Predict **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # Downloads the checkpoint on first use, then caches it in weights/. model = LibreYOLO("LibreYOLO9t.pt") # A single image returns one Results object. result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(result.names[int(box.cls)], float(box.conf), box.xyxy.tolist()) ``` **CLI** ```bash libreyolo predict model=yolo9-t save=True \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` **Video and streams** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") # stream=True yields one Results per frame instead of building a list. # Replace the path with a webcam index, an RTSP URL or a folder. for result in model.predict("clip.mp4", stream=True, save=True): print(len(result.boxes)) ``` `LibreYOLO()` is a factory. It reads the file, works out which family the weights belong to, and returns that family's model, so swapping in a different detector is a one-line change. Passing `LibreYOLO9t.pt` with no directory looks for `weights/LibreYOLO9t.pt` relative to the working directory and downloads it there when it is missing. See [checkpoints and weights](/docs/weights) for the download rules and how to work offline. `save=True` writes an annotated copy under `runs/detect/`, into a `predict` directory that increments per run. The returned `Results` carries `boxes`, and `names` maps a class index to its label. A single image path returns one `Results`; a directory, a list of images or `stream=True` returns a list or a generator of them. ## Train **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") # coco8 is an 8-image dataset bundled with the library. It downloads # from a URL on first use, so no script has to be executed. results = model.train( data="coco8.yaml", epochs=1, imgsz=640, batch=4, device="cpu", ) print(results["save_dir"]) print(results["best_checkpoint"]) ``` **CLI** ```bash libreyolo train model=yolo9-t data=coco8.yaml \ epochs=1 imgsz=640 batch=4 device=cpu ``` **Validate** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") # val() returns a plain dict, not an object. metrics = model.val(data="coco8.yaml", device="cpu") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) print(metrics["metrics/precision"], metrics["metrics/recall"]) ``` `data` is a dataset YAML. `coco8.yaml` ships with the library, which is why the snippet runs as pasted; a name that is not bundled is read as a path. Datasets resolve under `~/datasets`, or under `LIBREYOLO_DATASETS_DIR` when that variable is set. A run writes to `project/name`, defaulting to a directory below `runs/train`, with `weights/best.pt` and `weights/last.pt` inside it. `train()` returns a dictionary that includes `save_dir`, `best_checkpoint`, `last_checkpoint`, per-epoch losses and per-epoch validation metrics. The trained checkpoint loads through `LibreYOLO()` exactly like the pretrained one. Not every family is trainable. Where a family ships inference only, `train()` raises `NotImplementedError` and says so. [Core concepts](/docs/concepts) explains which support tier means what. ## Export **TorchScript** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9t.pt") # export() returns the path it wrote. path = model.export(format="torchscript") print(path) # The factory routes on file suffix, so the artifact loads back like a # checkpoint and returns the same Results object. exported = LibreYOLO(path) result = exported(SAMPLE_IMAGE) print(len(result.boxes)) ``` **ONNX** ```bash pip install "libreyolo[onnx]" libreyolo export model=yolo9-t format=onnx imgsz=640 ``` TorchScript needs nothing beyond the base install. The other targets each have their own extra, and coverage is per family and per task rather than uniform: see [export and deploy](/docs/export). Arguments accepted by every format include `imgsz` (an int, or a height and width pair), `batch` (default 1), `half`, `int8` with a `data` YAML for calibration, `dynamic` (default True), `simplify` (default True), `opset`, `device` and `output_path`. When `output_path` is omitted the file is written under `weights/` with a name derived from the checkpoint. ## Where to go next - [Core concepts](/docs/concepts) for tasks, families, sizes and checkpoint names. - [Checkpoints and weights](/docs/weights) for auto-download, offline use and loading safety. - [Import existing weights](/docs/migrate) if you already have a checkpoint from an upstream project. - [All models](/docs/models) for the family that fits your problem. - [Train](/docs/train), [Predict](/docs/predict) and [Export](/docs/export) for the full workflows. --- # Augmentation matrix Setting an augmentation knob does not guarantee it reaches the pipeline. This page records how each trainable family treats each knob on TrainConfig, using the declarative table the library ships as its single source of truth. Verified against LibreYOLO v1.5.0. ## The knobs These are `TrainConfig` field names, not CLI spellings. The CLI maps its own aliases onto them, so `--mosaic` sets `mosaic_prob`. | Knob | Meaning | |---|---| | `mosaic_prob` | Probability of building a 4-image mosaic sample | | `mixup_prob` | Probability of blending in a second sample | | `hsv_prob` | Probability of HSV color jitter | | `flip_prob` | Horizontal-flip probability | | `degrees` | Random-rotation range for the affine warp, in degrees | | `translate` | Random-translation fraction for the affine warp | | `mosaic_scale` | Random-scale range for the affine warp | | `mixup_scale` | Jitter-scale range applied to the MixUp partner image | | `shear` | Random-shear range for the affine warp, in degrees | | `perspective` | Projective warp magnitude for the affine warp | | `flipud` | Vertical-flip probability | | `no_aug_epochs` | Final epochs trained with strong augmentation disabled | | `auto_augment` | Classification AutoAugment policy: randaugment, autoaugment or augmix | | `erasing` | Classification RandomErasing probability | | `mixup` | Classification batch-MixUp probability, with soft labels | | `cutmix` | Classification batch-CutMix probability, with soft labels | The last four are the classification pack. Detection families ignore them. `mixup` is an API-only knob: the CLI `--mixup` is the alias for the detection `mixup_prob`. **Ask the spec directly** ```python from libreyolo.data.augment.spec import ( AUG_KNOBS, aug_support, ignored_aug_params, uses_mosaic_gating, ) print(sorted(AUG_KNOBS)) table = aug_support("yolo9") print(table["mixup_prob"].status, table["mixup_prob"].note) print(sorted(ignored_aug_params("dfine"))) print(uses_mosaic_gating("yolo9"), uses_mosaic_gating("yolonas")) ``` ## The three statuses | Status | Meaning | |---|---| | `used` | The knob reaches the family's train pipeline and changes samples | | `gated_by_mosaic` | The knob applies only to samples that took the mosaic branch, so with `mosaic_prob == 0` it never fires | | `ignored` | The knob never reaches the pipeline; setting it does nothing | `ignored` is the one worth checking before a run, because nothing fails. The CLI warns when an explicitly set training parameter is one the selected family ignores, and the trainer warns when `mixup_prob > 0` cannot fire because the family gates MixUp on mosaic and `mosaic_prob` is zero. ## Pipeline archetypes Every covered family follows one of six pipelines, with a handful of per-family deviations listed below. | Knob | YOLOX-style | YOLO-NAS | DETR-style | Classification | Semantic | Restore | |---|---|---|---|---|---|---| | `mosaic_prob` | used | ignored | ignored | ignored | ignored | ignored | | `mixup_prob` | gated | used | ignored | ignored | ignored | ignored | | `hsv_prob` | used | used | ignored | ignored | ignored | ignored | | `flip_prob` | used | used | used | ignored | ignored | ignored | | `degrees` | gated | used | ignored | ignored | ignored | ignored | | `translate` | gated | used | ignored | ignored | ignored | ignored | | `mosaic_scale` | gated | used | ignored | ignored | ignored | ignored | | `mixup_scale` | gated | used | ignored | ignored | ignored | ignored | | `shear` | gated | used | ignored | ignored | ignored | ignored | | `perspective` | gated | used | ignored | ignored | ignored | ignored | | `flipud` | used | used | ignored | ignored | ignored | ignored | | `no_aug_epochs` | used | used | used | used | used | used | | `auto_augment` | ignored | ignored | ignored | used | ignored | ignored | | `erasing` | ignored | ignored | ignored | used | ignored | ignored | | `mixup` | ignored | ignored | ignored | used | ignored | ignored | | `cutmix` | ignored | ignored | ignored | used | ignored | ignored | In the YOLOX-style pipeline the per-sample preprocessing applies HSV jitter and flips, while the affine warp and MixUp run only inside the mosaic branch. YOLO-NAS instead runs a per-sample affine that is always on, ignores mosaic, and applies MixUp independently, reusing `mosaic_scale` as the affine scale range. The DETR-style pipeline is a pass-through transform with no mosaic. Its photometric distortion, zoom-out and IoU-crop are recipe constants rather than configurable knobs, which is why `hsv_prob` and the geometry knobs never reach it. The classification pipeline uses an ImageFolder transform whose horizontal flip is a fixed 0.5 rather than `flip_prob`. Semantic scale jitter and HSV come from family class attributes rather than config knobs, and restoration flips are coupled input-and-target operations with a fixed 0.5 probability. `no_aug_epochs` is honored everywhere, though what it turns off differs: mosaic and MixUp for YOLOX-style, the affine and MixUp for YOLO-NAS, the strong photometric and crop augmentations plus the learning-rate tail for DETR-style, and the scheduler tail for the rest. ## Families by archetype | Archetype | Families | |---|---| | YOLOX-style | `yolox`, `yolo7`, `yolo9`, `yolo9_e2e`, `yolo9_p2`, `rtmdet`, `picodet`, `rtdetr`, `rtdetrv2`, `fomo` | | YOLO-NAS | `yolonas` | | DETR-style | `dfine`, `domedetr`, `deim`, `deimv2`, `rtdetrv4`, `rfdetr`, `ec`, `dinov2` | | Classification | `resnet`, `convnext`, `mobilenetv4`, `efficientnetv2` | | Semantic | `segformer` | | Restore | `nafnet` | Twenty-five families are covered. A family outside this list returns an empty ignored set, so no warning is emitted for it. ## Deviations | Family | Difference from its archetype | |---|---| | `rtmdet` | `flipud` ignored: its transform has no vertical flip | | `picodet` | `flipud` ignored | | `rtdetr` | `flipud` ignored | | `rtdetrv2` | `flipud` ignored | | `fomo` | `perspective` and `flipud` ignored | | `ec` | `hsv_prob`, `degrees` and `translate` used, for `task="pose"` only; detect and segment use fixed photometric recipes | | `dinov2` | The classification pack is used, for `task="classify"` only | `ec` and `dinov2` are multi-task families, so a knob is marked ignored only when every one of the family's trainable tasks ignores it. That keeps the CLI warning from ever being wrong for one task while right for another. Dome-DETR inherits D-FINE's transforms unchanged. The one thing it cannot take is multi-scale training, which its config disables rather than the augmentation spec. ## Family-specific knobs Some families carry augmentation knobs on their own `TrainConfig` subclass rather than on the base. The CLI does not expose these; set them through the Python API. | Family | Knob | Meaning | |---|---|---| | `yolo9`, `yolo9_e2e`, `yolo9_p2` | `copy_paste` | Copy-paste instance augmentation probability, `task="segment"` only | | `yolo9`, `yolo9_e2e`, `yolo9_p2` | `copy_paste_mode` | Copy-paste source: `flip` mirrors the same sample, `mixup` uses a second sample | | `yolo9`, `yolo9_e2e`, `yolo9_p2` | `rot90` | Random 90-degree rotation probability | | `rfdetr` | `copy_paste` | Copy-paste probability for `task="segment"`, `flip` mode only | | `rfdetr` | `copy_paste_mode` | Copy-paste source mode for `task="segment"` | | `rfdetr` | `crop_resize_prob` | Random crop-resize probability in the native pipeline | | `dfine` | `crop_resize_prob` | Random crop-resize probability, `task="segment"` | | `ec` | `crop_resize_prob` | Random crop-resize probability, `task="segment"` | | `ec`, `yolonas` | `brightness_contrast_prob` | Brightness and contrast jitter probability, `task="pose"` | | `ec`, `yolonas` | `affine_prob` | Keypoint-aware affine probability, `task="pose"` | `rot90` applies to detect and OBB on `yolo9`. ## Querying the spec | Helper | Returns | |---|---| | `aug_support(family)` | The knob-to-`Support` table, or `None` for an unknown family | | `ignored_aug_params(family)` | The set of knob names the family ignores; empty for an unknown family | | `uses_mosaic_gating(family)` | Whether the family's MixUp only fires on mosaic samples | | `display_name(family)` | The human-facing family name used in warnings | | `mixup_gating_warning(family, mosaic_prob, mixup_prob)` | The warning text when MixUp can never fire, else `None` | A `Support` is a named tuple of `status` and `note`, where the note explains why a knob is ignored or gated for that family. ## The mosaic gate For a YOLOX-style family, `mixup_prob=0.5` with `mosaic_prob=0` disables MixUp entirely, because MixUp applies only to mosaic samples. That combination is easy to reach when turning mosaic off late in training. The trainer logs a warning naming the family, and `mixup_gating_warning` is the pure function behind it. --- # Checkpoint schema A LibreYOLO .pt file is a flat dictionary saved with torch.save. The model key holds the state dict; the other top-level keys are metadata that identifies the checkpoint without filename parsing or state-dict sniffing. Verified against LibreYOLO v1.5.0. ## Schema v1.0 Every official LibreYOLO `.pt` checkpoint contains: ```python { "model": state_dict, "schema_version": "1.0", "libreyolo_version": "0.x.y", "model_family": "yolo9", "size": "t", "task": "detect", "nc": 80, "names": {0: "cat", 1: "dog"}, "imgsz": 640, } ``` | Key | Type | Meaning | |---|---|---| | `model` | state dict | The model weights | | `schema_version` | str | Metadata contract version; v1.0 uses the string `"1.0"` | | `libreyolo_version` | str | The version that produced the checkpoint | | `model_family` | str | A registered family, such as `yolo9`, `rfdetr`, `dfine`, `ec` | | `size` | str | Variant within the family, such as `t`, `s`, `r18`, `atto` | | `task` | str | Canonical task name | | `nc` | int | Positive class count | | `names` | dict | `dict[int, str]` with keys in `0..nc-1` | | `imgsz` | int | Positive square input resolution, or the legacy scalar for a rectangular contract | `task` is one of `detect`, `segment`, `semantic`, `panoptic`, `pose`, `classify`, `gaze`, `obb`, `point`, `depth`, `edge`, `normal`, `restore`, `matte`, `ocr`, `embed` or `mesh`. Official checkpoints write every `names` key. Readers may pad missing keys with `class_i` labels for legacy sparse mappings, but out-of-range keys are invalid. Rectangular checkpoints keep a scalar `imgsz` for legacy readers, set to `max(imgsz_h, imgsz_w)`, and additionally write `imgsz_h` and `imgsz_w` with the real dimensions. A reader that understands the rectangular fields must prefer them over the scalar. Families with a fixed rectangular contract, such as HRNet pose, reject incompatible runtime sizes. The schema is deliberately flat, and `model` is deliberately a state dict. **Read the metadata off a checkpoint** ```python from libreyolo import LibreYOLO from libreyolo.utils.serialization import unwrap_libreyolo_checkpoint import torch # Download a checkpoint, then re-save it so a local path exists. LibreYOLO("LibreYOLO9t.pt").save("roundtrip.pt") loaded = torch.load("roundtrip.pt", map_location="cpu", weights_only=False) state_dict, metadata = unwrap_libreyolo_checkpoint(loaded) print(metadata["schema_version"], metadata["model_family"]) print(metadata["size"], metadata["task"], metadata["nc"], metadata["imgsz"]) print(len(state_dict), "tensors") ``` ## Pose additions Pose is usually single-class, `nc: 1` with `person`, but the YOLO-NAS pose head also supports multi-class pose with one shared keypoint skeleton, in which case `nc` and `names` describe the classes as in detection. Runtime pose exports emit `scores` with shape `[batch, anchors, nc]`. | Key | Meaning | |---|---| | `num_keypoints` | Positive keypoint count used by the pose head | | `keypoint_dim` | `2` for `x,y` labels or `3` for `x,y,visibility` labels; model outputs always expose `x,y,visibility` | | `oks_sigmas` | Optional per-keypoint OKS sigmas; the task default for `num_keypoints` is used when absent | | `num_keypoints_per_class` | Optional per-class keypoint counts for GroupPose-style heads whose keypoint tensor is padded by class; `0` for classes without keypoints | ## Mesh additions Mesh checkpoints use `task: "mesh"`, `nc: 1` and `names: {0: "person"}`. Parameter layouts differ between body models, so the dimensions are recorded rather than assumed. | Key | Meaning | |---|---| | `body_model` | The parameterization, such as `mhr`; required, and used to interpret every field below | | `num_betas` | Identity and shape coefficient count; 45 for MHR | | `num_body_pose` | Width of the body-pose parameter block; 130 for MHR. A flat vector, not one triplet per joint, because rig joints carry different degrees of freedom | | `num_vertices` | Vertex count the decoder emits; 18439 for MHR | | `num_joints` | Joint count the decoder emits; 127 for MHR | | `rotation_format` | How rotations are encoded, such as `euler_zyx` for MHR or `axis_angle`. Never inferred from tensor shape, since a 3-vector is ambiguous | ## Dense-task placeholders Several tasks predict dense maps rather than classes, so the class-like slots exist only for schema compatibility. | Task | `nc` | `names` | |---|---|---| | `depth` | 1 | `{0: "depth"}` | | `edge` | 1 | `{0: "edge"}` | | `restore` | 1 | `{0: "image"}` | | `ocr` | 1 | `{0: "text"}` | Edge predictions are dense float32 probability maps in `[0, 1]`. Restore checkpoints may add `degradation`, a short corruption label such as `deblur`, `denoise` or `super-resolution`; `dataset`, a provenance label such as `GoPro` or `SIDD`; and `scale`, a positive integer output-to-input upscale factor, for example `4` for a x4 super-resolution model. Absent or `1` means the restored image keeps the input resolution. The runtime also derives the scale from the family and size, so `scale` is provenance metadata rather than a load-time requirement. ## OCR additions The `ppocr` family ships one composite checkpoint per tier whose `model` state dict holds two submodels under the `det.*` and `rec.*` key namespaces. | Key | Meaning | |---|---| | `charset` | The full CTC alphabet in output-index order: index 0 is the CTC blank, then the recognition dictionary, then the space character. Loaders must read it from the checkpoint, never from a side file | | `pipeline` | Pipeline defaults baked at conversion time: `det_limit_side_len`, `det_db_thresh`, `det_db_box_thresh`, `det_db_unclip_ratio`, `rec_image_shape`. Runtime arguments may override them per call | | `components` | Reserved for optional pipeline stages such as document orientation, unwarping and textline rotation. Empty in v1 | ## Export runtime metadata Exported artifacts use the same rectangular dual-write convention: `imgsz_h` and `imgsz_w` are written next to the legacy scalar `imgsz`, and a reader that does not understand the rectangular fields must not silently treat the scalar as a square contract. Rectangular runtime support is family-scoped and format-scoped. YOLO9-family, HRNet, NAFNet and Real-ESRGAN exports may use non-square `imgsz_h` and `imgsz_w` in supported formats; families or formats without explicit rectangular support reject the metadata rather than preprocessing those artifacts as square. HRNet exports are fixed, batch-one, FP32 person-crop heads, where W32 accepts 256x192 and W48 accepts 384x288, and the person detector is not embedded in the graph. Embedded-NMS exports may add these flat keys: | Key | Meaning | |---|---| | `nms` | String boolean; `"true"` means the graph includes an embedded post-processing output | | `nms_conf` | Confidence threshold baked into the embedded output | | `nms_iou` | IoU threshold baked into the embedded output | | `max_det` | Maximum post-NMS detection rows the embedded output emits | | `nms_raw_output` | String boolean; `"true"` means the graph also exposes an auxiliary raw detector output | For ONNX YOLO9 detection exports with `nms=true`, output `0` (named `output`) is the standalone post-NMS tensor at the export-time thresholds. When `nms_raw_output=true`, output `1` (named `raw`) is reserved for LibreYOLO backends so they can apply native original-canvas clipping and runtime `predict(conf=..., iou=..., max_det=...)` semantics. Third-party consumers should use the first output. Pose exports may add `num_keypoints`; `keypoint_dim`, where GroupPose-style raw exports may use larger values such as `8` when the tensor includes precision or class-logit fields; `num_keypoints_per_class` as a JSON-encoded list, where zero-keypoint class slots must be preserved because they define the schema; and `pose_input`, where `"person_crop"` means the graph consumes one already-extracted crop and contains no detector. HRNet runtime exports require that value. Classification exports may add `crop_pct`, a float center-crop ratio whose pre-crop resize target is `round(imgsz / crop_pct)` and which defaults to `0.875` when absent, and `interpolation`, `"bilinear"` or `"bicubic"`, defaulting to `"bilinear"`. ExecuTorch exports write the flat metadata to a required `.pte.json` sidecar. The v1 contract is CPU, FP32, batch 1 and a fixed input canvas, and it additionally requires `executorch_version`, `executorch_delegate` equal to `"xnnpack"`, and a positive `executorch_delegate_partitions`. The loader rejects a sidecar that claims another delegate, dynamic shapes, or non-FP32 precision. MNN exports write the flat metadata to a required `.mnn.json` sidecar. The v1 contract is CPU, FP32, detection-only and a fixed NCHW input shape, and it additionally requires `mnn_version`, `mnn_backend` equal to `"cpu"`, ordered non-empty `mnn_input_names` and `mnn_output_names`, `mnn_input_shape` as four positive integers in `[batch, channels, height, width]` order, and `mnn_batch` equal to `mnn_input_shape[0]`. The loader rejects dynamic, non-FP32, non-detection, unsupported-family or inconsistent shape metadata. A `.pte` and a `.mnn` are backend-specific artifacts, not PyTorch checkpoints. ## Quantized checkpoints A quantized model adds one optional flat key, `quant`, holding a manifest dict with `schema`, `recipe`, `keep_high_precision`, `execution`, calibration provenance, `module_count` and `state`. FP8 manifests may also carry `fp8_tensorwise_weights`, the exact list of `QuantLinear` module names whose weight scale is tensorwise rather than per-output-channel. A loader that sees `quant` rebuilds the quantized module structure and scaling policy before `load_state_dict`. `state` distinguishes the two artifact forms. `"prepared"`, the default, holds FP32 master weights plus `_q_*` scale buffers and is trainable. A reader without quantization support may ignore the `quant` key and load the masters as a float model. `"finalized"` is the deployment form written by `export(format="pt")`. Masters are stripped and each quantized module instead carries packed weights: | Recipe | Packed tensors | Dequantization | |---|---|---| | int8 | `weight_packed` int8 at the original weight shape, `_q_w_scale` FP32 per channel | `weight_packed * scale` | | fp8 | `weight_packed` float8_e4m3fn at the original shape, `_q_w_scale` FP32 one entry per output channel | `weight_packed * scale` | | w4a16, w4a8 | `weight_packed` uint8, two 4-bit codes per byte, low nibble first, code `q + 8`; `_q_w_gscale` FP32 `[out, ngroups]`, group 128 along in_features | Group-wise scale | | int2 | Four 2-bit codes per byte, code `q + 2`, group 64 | Group-wise scale | | nvfp4 | `weight_packed` uint8 `[out, ceil(in/16)*8]`, code `sign<<3 \| E2M1 level`; `weight_block_scale` float8_e4m3fn `[out, ceil(in/16)]`; `_q_w_amax` FP32 per tensor | `block_scale * amax / (448 * 6)` | | mxfp4 | As nvfp4 but 32-element blocks, plus `weight_block_exp` int8 `[out, ceil(in/32)]` | `2 ** exponent` | Activation range buffers `_q_act_lo`, `_q_act_hi` and `_q_calibrated` are retained for int8. The manifest records `remainder`, `"fp16"` or `"fp32"`, for the non-quantized tensors. Unpacking reproduces the simulation bit for bit, so finalized inference matches prepared inference exactly on the finalizing device. This layout is the stable contract for external exporters and runtimes. ## Training checkpoints Trainer checkpoints use the same required metadata core and may add flat training and resume fields: ```python { "model": state_dict, "epoch": 42, "optimizer": optimizer_state_dict, "config": {}, "loss": 1.23, "best_metric_key": "metrics/mAP50-95", "best_metric_value": 0.51, "best_epoch": 39, "is_ema_weights": True, "train_model": raw_state_dict, "ema": ema_state_dict, "ema_updates": 12345, } ``` `is_ema_weights` declares whether the top-level `model` is EMA-smoothed. When EMA is enabled, `train_model`, `ema` and `ema_updates` preserve resume state. Published inference weights should be lean and should not include optimizer, epoch, config, loss or EMA resume state unless they are intentionally distributed as training checkpoints. For release compatibility, readers accept the legacy best-metric aliases `best_mAP50_95`, `best_mAP50`, `best_metric` and `best_metric_name`. ## External snapshots The schema governs LibreYOLO-authored `.pt` files. It does not rename or wrap multi-file upstream snapshots used by the separate model tiers. LibreMODUS size `14b-a7b` is an explicit exception: the alias resolves through `LibreVLM(...)` to a directory of pinned upstream files, and LibreYOLO neither adds v1.0 metadata to it nor republishes it as a `.pt`. ## Legacy and foreign weights New writers validate strictly and must emit v1.0 metadata. When metadata is missing or incomplete, legacy LibreYOLO-looking checkpoints load through the compatibility path with a warning and conversion instructions, and foreign upstream checkpoints route to auto-conversion. See [upstream checkpoints](/docs/reference/upstream-checkpoints). ## Helpers The schema helpers live in `libreyolo.utils.serialization`: ```python wrap_libreyolo_checkpoint( state_dict, *, model_family, size, task, nc, names=None, imgsz=None, libreyolo_version=None, schema_version="1.0", **extra_metadata, ) -> dict validate_checkpoint_metadata(checkpoint, *, strict=False) -> list[str] unwrap_libreyolo_checkpoint(loaded, *, strict=False) -> tuple[dict, dict] ``` `validate_checkpoint_metadata` is non-mutating and returns the list of errors; with `strict=True` it raises `CheckpointMetadataError` instead. `model.save(path)` is the supported way to write a conforming checkpoint. --- # CUDA graphs A CUDA graph records one execution of a fixed sequence of kernels and replays it as a single launch. LibreYOLO captures inference on 39 verified families and training on 24, always per family, always after a bitwise parity check, and never as a silent fallback. Verified against LibreYOLO v1.5.0. ## What is captured A graph records a fixed sequence of kernels and the memory addresses they read and write. It does not record values, shapes or control flow. Replay is a single launch instead of hundreds, which is why the gain is largest on small networks at small batch sizes, where a step is dominated by launch overhead rather than by arithmetic. The two entry points capture different amounts of work. | | Inside the graph | Eager | |---|---|---| | Inference | The network forward, `model._forward(x)` | Preprocessing, NMS, all postprocessing | | Training | The network forward and backward | Loss, optimizer step, gradient clipping, EMA, LR schedule | Neither NMS nor detection loss is a candidate. Both select with boolean masks, run Hungarian matching or an assigner, and branch on the result, which is exactly what a graph cannot record. Keeping them out is what makes capture safe rather than a limitation to work around. **Predict** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9t.pt") # True captures on first use per input shape. # "auto" waits until a shape repeats before paying the capture cost. result = model(SAMPLE_IMAGE, cuda_graph=True) ``` **Train** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") model.train(data="my-dataset.yaml", epochs=100, cuda_graph=True) ``` **Train from the CLI** ```bash libreyolo train model=LibreYOLO9t.pt data=my-dataset.yaml \ epochs=100 --cuda-graph ``` `cuda_graph` accepts three values at predict time. `False` is the default. `True` captures the first time each input shape is seen. `"auto"` waits for a shape to repeat, so one-shot and shape-varying work never pays for a capture it will not reuse. `capture_graph(imgsz=None, batch=1, dtype=None)` moves the cost off the first request, `graph_info()` reports captured graphs and replay counts, and `release_graphs()` frees them. At train time the flag is a plain boolean, `--cuda-graph` on the CLI. See [prediction performance](/docs/predict/performance) and [training performance](/docs/train/performance) for the surrounding controls. ## Inference support Support is per family, declared through the `SUPPORTS_CUDA_GRAPH` class variable, and a family is only flagged after it captures and replays bit-identically against two probe inputs drawn from different distributions. That shared parity matrix covers 39 families across nine tasks. | Task | Families | |---|---| | detect | yolo1, yolo2, yolo3, yolo4, yolo9, yolo9_p2, yolo9_e2e, yolox, yolo7, yolonas, picodet, rtmdet, dfine, deim, deimv2, rtdetr, rtdetrv2, rtdetrv4, rfdetr, ec | | segment | dfine, rtmdet, rfdetr, ec | | pose | ec, yolonas, rfdetr | | point | fomo | | classify | resnet, convnext, mobilenetv4, efficientnetv2, clip, dinov2, siglip2 | | semantic | eomt, dinov2, segformer, pidnet, lingbotvision | | depth | depth_anything, depth_anything3, zipdepth | | restore | nafnet, realesrgan, swinir | | matte | birefnet | Several families appear under more than one task, so the matrix runs more rows than it has distinct families. Three more families capture through family-specific code paths with their own dedicated tests rather than through the shared matrix, and are not part of the 39: PP-OCR, SAM and SenseNova. The verification is bitwise, not approximate. An earlier version of the protocol judged parity by relative magnitude and wrongly demoted three healthy families, YOLOX, EfficientNetV2 and YOLOv7, whose eager-to-graph difference measures around 1e-7 while still being bit-identical on the probe that matters. ## Training support Training capture went from two families to 24 in this release, across five tasks. | Task | Families | |---|---| | detect | yolo9, yolo9_p2, yolo9_e2e, yolox, yolo7, yolonas, picodet, rtmdet, rfdetr, dfine, deim, deimv2, rtdetr, rtdetrv2, rtdetrv4, ec | | classify | resnet, convnext, mobilenetv4, efficientnetv2 | | semantic | segformer, lingbotvision | | point | fomo | | restore | nafnet | Everything else trains eager: other tasks on those same families, families not listed, distributed runs and distillation runs. Capture is also skipped while a shape is still new, since the training path waits for an input shape to repeat three times before capturing, which means `multi_scale=True` may never capture at all. ## Two different answers for an unsupported family The inference path raises. `predict(cuda_graph=True)` on a family that has not opted in raises `NotImplementedError` naming the family, rather than running eager and letting you believe you got a speedup you did not get. The reason is that a bad capture does not fail loudly: replay of a forward that does something uncapturable returns wrong numbers silently, so support has to be an explicit per-family assertion rather than an attempt with a fallback. The training path logs. `train(cuda_graph=True)` is always safe to pass, and a family, task or configuration that cannot be captured writes one line and trains eager, unchanged. A capture that fails partway through a run also drops the rest of the run to eager rather than aborting it. The asymmetry is deliberate: prediction is a call you can fix at the call site, while a training run should not die at hour six over an optional optimization. ## Seam splitting Some families cannot be captured whole because one stage genuinely does something a graph cannot record. Rather than dropping the family, capture is split at a verified seam: the capturable part replays, the rest runs eagerly, and the combined output is the same as running everything eagerly. | Family | Captured | Eager, and why | |---|---|---| | Depth Anything 3 | The network | The sky step, which is host-visible work after the forward | | BiRefNet | The encoder, `forward_enc` | The decoder, whose `deform_conv2d` replays to a different result under capture | | PP-OCR | The detection stage, `forward_det` | Recognition, because crop widths vary per line | | SAM | The image encoder | The prompt path, which runs many times per encode | | SenseNova | The vision tower | Autoregressive generation, with a KV cache that grows every step | | Encoder-decoder detectors | Backbone and encoder | Decoder and Hungarian criterion | The BiRefNet split is worth reading twice: `deform_conv2d` misbehaving under capture reproduces on a bare call outside any model. Replacing it with a pure-PyTorch equivalent was rejected because that would have shifted eager predictions too, and eager numbers are the contract. The encoder-decoder case covers D-FINE, DEIM, DEIMv2, RT-DETR, RT-DETRv2, RT-DETRv4 and EC. Their decoder builds contrastive-denoising queries from the ground truth, and the number of those queries comes from the largest ground-truth count in the batch, so the decoder's token count changes from batch to batch. That is the one thing a graph cannot tolerate. Backbone plus encoder is roughly a fifth to a quarter of a step for these families, which is why they sit at the bottom of the speedup table. PP-OCR captures one graph per detection input shape, bounded by the runner's cache cap, and returns the eager result when no capture scope is active. ## Numerics Most families are bit-identical, and where they are not, the reason is named rather than waved at. At step zero of training the loss is bit-identical for all 24 families and no BatchNorm buffer differs; the gradient comparison is what separates the categories. | Class | Families | Meaning | |---|---|---| | Exact | Most of the 24 | Every gradient bit-identical | | 1 ULP | fomo, lingbotvision | The last bit of float32, about 1e-7 relative, from a different summation order | | Eager noise | The DETR lineage | Graphed differs from eager no more than two eager runs differ from each other | | Float rounding | rtmdet | 137 of 139 gradients bit-identical, two differ by about 3e-4 | | Own RNG stream | segformer | Stochastic depth sits inside the captured region | The eager-noise class is the important one to read correctly. For those families, two seeded eager runs already disagree, so bit-identical is not a bar the graphed run failed; it is a bar nothing clears. That holds more widely at `amp=False`, where a measured 3.2e-7 relative nondeterminism in an fp32 weight gradient compounds: two seeded eager YOLOv9-t runs diverge by 36 percent over 20 steps, and turning TF32 off does not fix it. ## Pin memory Capture runs with `capture_error_mode="thread_local"`. Under PyTorch's default `"global"` mode, a DataLoader pin-memory thread staging the next batch calls `cudaHostAlloc`, which both invalidates the in-flight capture and gets poisoned by it, so the run dies on its next batch fetch with an error raised from inside the pin-memory thread. That pairing was observed twice on a real training campaign before it was diagnosed. Thread-local mode restricts only the capturing thread. The pin thread never touches the capturing stream, so nothing it does belongs in the graph in the first place. Training goes further and temporarily substitutes a `torch.cuda.CUDAGraph` subclass that forces the mode, because `make_graphed_callables` exposes no argument for it, under a lock so two concurrent captures cannot leave the substitution installed. ## What it is worth Measured on an RTX 5070 Ti under AMP, one process per arm, replaying one real batch so the dataloader is out of the loop, fastest of 24 steps after warm-up. Detection at 640 px, classification at 224 px. | Family | Batch | Speedup | |---|---:|---:| | FOMO s | 16 | 3.63x | | MobileNetV4 s | 16 | 2.74x | | EfficientNetV2 b0 | 16 | 2.44x | | YOLOv9-t | 8 | 1.99x | | YOLOv9 e2e | 8 | 1.76x | | YOLOv9 p2 | 8 | 1.49x | | Everything else | varies | 1.04x to 1.26x | A whole run gains less, because a graph cannot speed up the dataloader or validation. A 20-epoch YOLOv9-t fine-tune on 406 images went from 428.4 s to 367.7 s, a 1.16x end-to-end gain, with an identical mAP50-95 of 0.6394 in both arms and identical per-epoch losses. The ceiling is set by how much of a step is network. On the same hardware at 640 px and batch 8, that is 84 percent for YOLOv9-t but only 26 percent for RTMDet-t, which spends most of a step in its label assigner. Launch overhead is highest on Windows, so Linux gains land at roughly a third to half of this table, and a dataloader-bound run sees no wall-clock change at all. Peak memory moves between 5 percent lower and 19 percent higher. ## Caveats A graph records addresses, not values, so anything that relocates parameters drops it. Changing device through `predict(device=...)`, quantizing and dequantizing all invalidate captured graphs. Batch size matters more than family: RT-DETR-r18 gains 1.19x at batch 2 and 1.04x at batch 8, because a large batch is compute-bound and has less launch overhead to remove. The inference parity suite ran without the optional `kernels` package installed, so capture safety with compiled Hub kernels active is not covered by it. Set `LIBREYOLO_HUB_KERNELS=0` to take them out of the picture while isolating a capture problem. See [kernels](/docs/reference/kernels). --- # Dataset formats This page mirrors the dataset-file contract in the library's own docs/dataset_schema.md. It covers the YAML keys and on-disk layout each canonical task expects. Verified against LibreYOLO v1.5.0. ## Common YAML Applies to `detect`, `segment`, `pose` and `obb`. | Key | Required | Meaning | |---|---|---| | `path` | | Dataset root | | `train` | For training | Training images | | `val` | For validation | Validation images | | `test` | | Test images | | `names` | Yes | Class list, or an integer-keyed mapping | | `nc` | | Class count; must match `names` when present | | `download` | | Download instructions; Python scripts need explicit opt-in | | `annotations` | | Split to native COCO JSON file, for detect, segment and obb | `train`, `val` and `test` may be image directories, image-list `.txt` files, or lists of those. Label paths follow one substitution: ```text images/.../image.jpg -> labels/.../image.txt ``` For a native COCO JSON dataset, `annotations` maps a split to its JSON file and the split path gives the image root: ```yaml path: dataset train: images/train val: images/val annotations: train: annotations/train.json val: annotations/val.json ``` When `names` is present, native COCO JSON category names must match the YAML class names, and those names define the model label IDs. Without `names`, COCO category IDs are sorted and mapped densely to `0..N-1`. A dataset YAML does not carry a `task` key. Explicit model and task selection wins. Rules common to every text label file: - one `.txt` label file per image; - a missing or empty label file means no objects; - `class_id` is an integer in `0..nc-1`; - coordinates are finite normalized floats in `[0, 1]`; - coordinates are relative to the original image width and height; - rows carry no confidence and no track ID. **Parse one detection label row** ```python from libreyolo.data import parse_yolo_label_line # class_id cx cy w h, normalized to [0, 1] row = parse_yolo_label_line("0 0.5 0.5 0.25 0.5", 640, 480, num_classes=80) # (class_id, x1, y1, x2, y2, area) in pixels print(row) ``` ## detect Exactly five fields per row: ```text ``` `cx cy w h` is a normalized axis-aligned box, and `w` and `h` must be positive. ## segment A polygon row: ```text ... ``` `N` is at least 3, the coordinate count after `class_id` must be even, and the polygon must be non-degenerate. A five-field detection row is also accepted and represents a rectangular segment. ## pose YAML adds `kpt_shape`, which is required and is `[K, 2]` or `[K, 3]`, and the optional `flip_idx`, an integer permutation of `0..K-1`. ```text [] ... [] ``` The field count is exactly `5 + K * D`, where `D` is the second `kpt_shape` value. Keypoint coordinates are normalized. Visibility `v`, when present, is `0`, `1` or `2`. ## obb Exactly nine fields: ```text ``` The four points are normalized image coordinates in `[0, 1]` and form a non-degenerate oriented rectangle. No angle is stored in the label file. The canonical parser is strict by default and rejects out-of-range coordinates. Dataset and validation ingestion may clip coordinates to `[0, 1]` for otherwise valid crop-boundary labels, then still reject degenerate boxes. Parsing is task-aware: nine fields mean `obb` only in `obb` mode, while in `segment` mode they may be a four-point polygon. Internally, normalized corners are converted to canonical `xywhr`, with the angle in radians representing rotation of the width side around the box center. Public results expose OBB detections as `xywhr, conf, cls` rows. Native COCO JSON OBB loading accepts annotations in this priority order: `obb` as eight pixel-space corners; `obb` as `[cx, cy, w, h, angle]` with the angle in radians; a COCO `segmentation` polygon or RLE, refit to a minimum-area rectangle; and a COCO `bbox`, read as axis-aligned and canonicalized. Mosaic and mixup are disabled for OBB training until corner-aware OBB augmentation exists. The canonical row parser is `libreyolo.data.parse_yolo_obb_label_line`. ## semantic Each image pairs with a dense single-channel mask in a lossless format, typically PNG, instead of a `.txt` file: ```text images/.../image.jpg -> /.../image.png ``` The mask is single channel, and palette-mode PNGs are read as palette indices. Each pixel value is a class ID in `0..nc-1`, pixel value `255` means ignore and is excluded from loss and metrics, and the mask resolution must equal the image resolution. Two optional YAML keys sit on top of the common contract. `masks_dir` is the mask directory name substituted for `images` in each image path, defaulting to `masks`. `label_mapping` is a `{source_id: train_id}` remap applied to mask pixel values at load time, where unmapped source values become ignore and train IDs must fall in `0..nc-1`. When `masks_dir` is omitted, masks are rasterized at load time from `segment` polygon labels resolved through the `images` to `labels` convention, and a `background` class is appended after the object classes, so `nc` grows by one. Canonical loader: `libreyolo.data.SemanticDataset`. ## panoptic LibreYOLO adopts the COCO-panoptic format verbatim (Kirillov et al., CVPR 2019). There is no LibreYOLO-specific panoptic format. One RGB PNG per image, at the image resolution, encodes each pixel's segment ID in its color: ```text segment_id = R + 256 * G + 256 * 256 * B ``` Every pixel belongs to exactly one segment and segments never overlap. Segment ID `0`, RGB black, is void: unlabeled pixels excluded from the metric. ```json { "images": [{"id": 139, "file_name": "000000000139.jpg"}], "annotations": [{"image_id": 139, "file_name": "000000000139.png", "segments_info": [ {"id": 3226956, "category_id": 1, "area": 2840, "bbox": [413, 158, 53, 138], "iscrowd": 0}]}], "categories": [{"id": 1, "name": "person", "isthing": 1, "supercategory": "person"}] } ``` `annotations[].file_name` names the segment-ID PNG inside `panoptic_dir`, and `segments_info[].id` matches a value in that PNG. `iscrowd` marks group regions: they are never false negatives, and a prediction mostly covering one is not a false positive. Thing-versus-stuff is a per-category property. `isthing` lives on `categories`, never on `segments_info`. COCO-panoptic `category_id` values are the dataset's raw IDs and are typically non-contiguous. Models predict contiguous `0..nc-1`, so raw IDs are remapped through the YAML `names` by category name, the same rule the native COCO JSON detect loader follows. A JSON category absent from `names` is an error rather than a silent drop, because it would otherwise score as a permanent false negative. ```yaml path: coco val: images/val2017 annotations: val: annotations/panoptic_val2017.json panoptic_dir: val: annotations/panoptic_val2017 names: {0: person, 1: bicycle, 132: rug-merged} ``` `annotations` and `panoptic_dir` accept either a single path or a per-split mapping. Validation reports Panoptic Quality, computed at the ground-truth resolution and averaged over the categories that appear, then split into `PQ_things` and `PQ_stuff`. Matching is unique: a predicted and a ground-truth segment of the same category match when IoU is above 0.5. Canonical loader: `libreyolo.data.PanopticDataset`. ## depth Each image pairs with a dense single-channel depth map: ```text images/.../image.jpg -> /.../image.png ``` The map is a single-channel PNG or TIF, or a `.npy` file, at the image resolution. Values are plain depth in a dataset-consistent unit. Zero, negative, NaN and infinite values mark invalid pixels and are excluded from loss and metrics. | Key | Default | Meaning | |---|---|---| | `depths_dir` | `depths` | Depth directory substituted for `images` | | `depth_stem_suffix` | | Suffix appended to the image stem; when omitted both the same stem and a `_depth` suffix are tried | | `depth_mask_suffix` | `_mask` | Suffix for a validity mask; mask values at or below zero, NaN and infinite invalidate the depth pixel | | `depth_scale` | `256.0` | Divisor for integer-typed depth maps, the common 16-bit PNG convention | Float `.npy` maps are used as-is and do not apply `depth_scale`. Canonical loader: `libreyolo.data.DepthDataset`. ## edge Each RGB image pairs with a same-stem single-channel lossless map and an optional validity mask: ```text images/val/scene.jpg -> edges/val/scene.png -> masks/val/scene.png ``` The map is single-channel PNG or TIF, not an RGB visualization, at the image resolution. Integer maps are divided by their dtype maximum; float maps must already be finite and in `[0, 1]`. `0` means non-edge and `1` means edge. Optional mask pixels are valid when nonzero. Resizing uses nearest-neighbor interpolation for targets and masks, and padded pixels are invalid and do not contribute to validation. | Key | Default | Meaning | |---|---|---| | `edges_dir` | `edges` | Edge-map directory substituted for `images` | | `edge_stem_suffix` | | Suffix appended to image stems | | `edge_extension` | `.png` | Lossless target extension | | `edge_invert` | | Set true when source maps store black edges over white | | `masks_dir` | `masks` | Optional validity-mask directory | ```yaml path: edge-dataset train: images/train val: images/val edges_dir: edges masks_dir: masks nc: 1 names: {0: edge} ``` Validation thins continuous predictions with four-direction gradient non-maximum suppression and reports ODS and OIS F-measures over a configurable threshold sweep. Predicted and ground-truth pixels are matched one-to-one within `edge_max_dist * image_diagonal`, with a default normalized tolerance of `0.0075`. Canonical loader: `libreyolo.data.EdgeDataset`. The loader is format-only: it does not download or redistribute benchmark data. ## normal Each image pairs with a same-stem three-channel 16-bit PNG, plus an optional same-stem validity mask: ```text images/val/room.jpg -> normals/val/room.png -> masks/val/room.png ``` The PNG is exactly three-channel `uint16` with channels stored as RGB, at the image resolution. Decode with `n = png / 65535 * 2 - 1`, then renormalize each vector. Decoded vectors use the OpenCV camera frame, `+x` right, `+y` down, `+z` into the scene, and face the camera. The optional mask is a single-channel PNG where nonzero means valid; without a mask, every finite, nonzero decoded vector is valid. Invalid and padded target pixels are represented internally by `(0, 0, 0)`. Resizing interpolates the three components bilinearly and then renormalizes, validity masks use nearest-neighbor interpolation, and a horizontal flip also negates the x component. | Key | Default | Meaning | |---|---|---| | `normals_dir` | `normals` | Normal-map directory substituted for `images` | | `masks_dir` | `masks` | Optional validity-mask directory | Validation reports mean and median angular error in degrees and the percentage of valid pixels within 11.25, 22.5 and 30 degrees. Canonical loader: `libreyolo.data.NormalDataset`. ## restore Each degraded input image pairs with a clean RGB target: ```text inputs/.../image.jpg -> targets/.../image.jpg ``` Input and target are RGB-compatible image files and their resolutions must match exactly. Validation keeps native resolution and pads only enough to stack a batch, and metrics are computed on the original image canvas. Training applies a coupled crop and horizontal flip to the input and target pair. | Key | Default | Meaning | |---|---|---| | `input_dir` | `inputs` | Degraded-input directory used in split paths | | `target_dir` | `targets` | Clean-target directory substituted for `input_dir` | | `target_stem_suffix` | | Suffix appended to the input stem before target lookup | | `target_stem_suffixes` | | List form of `target_stem_suffix` | | `degradation` | | Metadata label such as `deblur` or `denoise` | | `dataset` | | Dataset or provenance label | The class-like YAML fields are schema placeholders: use `nc: 1` and `names: {0: image}`. Restore models expose `Results.restored`, not detections. Canonical loader: `libreyolo.data.RestoreDataset`. ## matte Each RGB image pairs with a single-channel ground-truth matte sharing the same stem, where 0 is background and 255 is foreground: ```text images/subject.jpg -> mattes/subject.png ``` Two layouts are accepted. A directory root containing `images/` and a matte directory, auto-detected among `mattes/`, `matte/`, `gt/`, `masks/`, `mask/` and `alpha/`, passed as `data=`. Or a YAML with `path` plus per-split `val_images` and `val_mattes`, and optionally `train_images` and `train_mattes`, each relative to `path` or absolute. The matte is grayscale and read as opacity in `[0, 1]`, and it is resized to the prediction canvas with bilinear interpolation when the shapes differ. Metrics are MAE and S-measure (Fan et al., ICCV 2017) on the original image canvas, with S-measure as the best-checkpoint fitness. The class-like YAML fields are schema placeholders: use `nc: 1` and `names: {0: matte}`. Matte models expose `Results.matte`. Validation is inference-only in this version. Canonical pair resolver: `libreyolo.data.matte_dataset.resolve_matte_pairs`. ## ocr Labels are one JSONL file per split, one JSON object per image: ```text images/val/receipt.jpg -> labels/val.jsonl ``` ```json {"image": "receipt.jpg", "regions": [{"polygon": [[10, 12], [118, 14], [117, 40], [9, 38]], "text": "TOTAL 12.50"}]} ``` `polygon` is a four-point quad in absolute pixel coordinates, ordered top-left, top-right, bottom-right, bottom-left. Regions with unreadable text use `"text": "###"`, the ICDAR do-not-care convention: they are excluded from recognition scoring, and predictions overlapping them are ignored rather than penalized in detection matching. Metrics are detection hmean with one-to-one polygon matching above IoU 0.5, end-to-end F1 requiring both IoU above 0.5 and an exact transcript after NFKC normalization and whitespace removal, case-sensitive, and 1-NED on matched pairs. Best-checkpoint fitness is end-to-end F1. Two layouts are accepted: a directory root containing `images//` and `labels/.jsonl`, passed as `data=`, or a YAML with `path` plus optional `images` and `labels` directory names. The class-like YAML fields are schema placeholders: use `nc: 1` and `names: {0: text}`. OCR models expose `Results.ocr`. Validation is inference-only in this version. Canonical sample resolver: `libreyolo.data.ocr_dataset.resolve_ocr_samples`. ## classify An ImageFolder-style directory tree, not label files: ```text dataset_root/ train/ class_a/*.jpg class_b/*.jpg val/ class_a/*.jpg class_b/*.jpg ``` `train/` is required for training and defines the class-to-index mapping by sorted folder name. `val/` is required for validation. `test/` may be present but the default train and val commands do not use it. Non-training splits must contain the same class folder names as the expected train or checkpoint class set. Supported image extensions are defined in `libreyolo.data.classify_dataset.IMAGE_EXTENSIONS`. ## gaze and point No training or validation dataset-file contract is implemented for `gaze`. `point` is a model-output task rather than a dataset-label schema. Point families may adapt existing labels internally, for example by deriving object centers from box rows, but a point-only text label format is not defined. --- # Ensemble API LibreEnsemble runs several detectors on the same image and fuses their detections into one Results. Fusion happens after each member's own postprocessing, so members keep their own input size, normalization and suppression. Verified against LibreYOLO v1.5.0. ## LibreEnsemble ```python LibreEnsemble( members, *, weights=None, fusion="wbf", fusion_iou=0.55, min_votes=1, ) ``` | Argument | Default | Meaning | |---|---|---| | `members` | | Two or more detectors | | `weights` | `None` | Per-member trust factors; all `1.0` when omitted | | `fusion` | `"wbf"` | `"wbf"`, `"wbf_seeded"`, `"nms"`, or a callable | | `fusion_iou` | `0.55` | IoU threshold for fusion clustering | | `min_votes` | `1` | Keep only boxes confirmed by at least this many members | A member is a weights path resolved through the `LibreYOLO()` factory, an already-constructed model, an exported backend, or an `ExternalDetector`. Every member must be a detect-task model. **Two members, default fusion** ```python from libreyolo import LibreEnsemble, SAMPLE_IMAGE ens = LibreEnsemble(["LibreYOLO9t.pt", "LibreYOLO9s.pt"]) # A single image source returns one Results, not a list. result = ens(SAMPLE_IMAGE, conf=0.25) print(result.boxes.xyxy) print(result.speed) ``` **Consensus and per-member thresholds** ```python from libreyolo import LibreEnsemble, SAMPLE_IMAGE ens = LibreEnsemble( ["LibreYOLO9t.pt", "LibreYOLO9s.pt"], weights=[1.0, 2.0], fusion="wbf", fusion_iou=0.55, min_votes=2, ) result = ens(SAMPLE_IMAGE, conf=[0.25, 0.4]) print(len(result)) ``` Construction rejects fewer than two members, a `weights` list of the wrong length, a non-positive weight, a `min_votes` that is not a positive integer, and a `min_votes` larger than the member count. `fusion="nms"` with `min_votes > 1` also raises, because NMS discards cluster membership and cannot count votes. `weights` scales the trust placed in each member. Higher weight pulls fused coordinates and scores toward that member. The convention is to make them proportional to validation mAP. ## Class spaces Members with identical `names` pass straight through. Otherwise the class spaces are unioned by name, member class IDs are remapped through lookup tables, and the fused `Results.names` is the union. Fusion merges boxes only within the same unified class, so a class only one member knows passes through unfused. A mismatch logs a warning at construction. `min_votes` is capped per class by how many members' label spaces contain that class, so consensus stays meaningful on partially shared vocabularies. ## Calling the ensemble ```python ens( source=None, *, conf=0.25, iou=0.45, imgsz=None, device=None, classes=None, max_det=300, augment=False, save=False, output_path=None, color_format="auto", batch=1, stream=False, stream_buffer=False, vid_stride=1, show=False, **kwargs, ) ``` `predict` is an alias for `__call__`. The return is the usual `Results`, whose `speed` breaks the cost down per member and adds a `fusion` entry. A single image source returns one of them, a list or directory returns a list, and `stream=True` returns a generator. `conf`, `iou` and `device` broadcast to every member and also accept one value per member, so `conf=[0.25, 0.4]` gives member 0 a threshold of 0.25 and member 1 a threshold of 0.4. `imgsz` broadcasts when it is an int or a tuple and is per-member only when it is a list, so `imgsz=(480, 640)` is one rectangular size for everyone while `imgsz=[480, 640]` is 480 for member 0 and 640 for member 1. Each entry must be valid for that member's family. `augment` broadcasts to members that support test-time augmentation, and exported backends ignore it. `classes` takes union class IDs and `max_det` applies to the fused result, so members run generously and the ensemble trims once. `batch` is accepted for API parity; images are processed sequentially. `val()` and `export()` raise `NotImplementedError`. Validate and export the members individually. ## ExternalDetector ```python ExternalDetector(fn: Callable, names: dict[int, str]) ``` Adapts any detection callable into a member. `fn` takes a PIL image and returns `(boxes, scores, labels)`, where boxes are xyxy in original-image pixels and labels are class IDs valid in `names`. Tensors, arrays and nested lists all work. LibreYOLO imports nothing from the external code. The adapter validates the return: it must be a 3-tuple, boxes must have shape `(N, 4)`, the three arrays must be the same length, and every class ID must appear in `names`. Detections at or below `conf` are dropped before fusion. ## Fusion operations The fusion primitives are standalone torch ops in `libreyolo.ops`. They are model-free and importable on their own, which is why they are exported separately from the ensemble. **Fusion op, no model involved** ```python import torch from libreyolo.ops import weighted_boxes_fusion boxes = torch.tensor([[10.0, 10.0, 50.0, 50.0], [12.0, 11.0, 51.0, 49.0]]) scores = torch.tensor([0.9, 0.8]) labels = torch.tensor([0, 0]) model_ids = torch.tensor([0, 1]) fused = weighted_boxes_fusion( boxes, scores, labels, model_ids, num_models=2, iou_thr=0.55 ) print(fused) ``` All three take the same positional arguments, `boxes, scores, labels, model_ids`, and return `(boxes, scores, labels)`. | Op | Registry key | Behavior | |---|---|---| | `weighted_boxes_fusion` | `wbf` | Sequential, paper-faithful weighted boxes fusion | | `wbf_seeded` | `wbf_seeded` | Parallel one-pass variant of the same reduction | | `nms_fusion` | `nms` | Concatenate everything and apply class-aware NMS | `FUSIONS` maps the three registry keys to the callables, and `LibreEnsemble` looks up `fusion=` there. ```python weighted_boxes_fusion( boxes, scores, labels, model_ids, *, weights=None, num_models=None, iou_thr=0.55, skip_box_thr=0.0, conf_type="avg", min_votes=1, models_per_label=None, label_weights=None, ) ``` `wbf_seeded` takes the identical signature. `nms_fusion` takes the same arguments except `conf_type`, and raises `ValueError` when `min_votes > 1`. In `weighted_boxes_fusion`, detections are visited in order of decreasing weight-scaled confidence. Each one either joins the existing cluster whose running fused box it overlaps best, at IoU above `iou_thr` and with the same label, or starts a new cluster. A cluster's fused box is the confidence-weighted average of its members' coordinates, and its score is the weighted mean or maximum of their confidences, rescaled so that boxes confirmed by fewer models score lower. `wbf_seeded` picks cluster seeds with class-aware NMS at `iou_thr`, assigns every detection to its best-IoU seed of the same label, then reduces each cluster the same way. Cluster shapes never shift mid-pass, so the whole op is fixed-shape tensor math. The two variants agree whenever clusters are unambiguous and can differ slightly on overlapping cluster chains. `nms_fusion` keeps the highest-confidence box of each overlapping group, unchanged. Per-model `weights` scale confidences for the suppression ranking only, and surviving boxes keep their original scores. ## Custom fusion `fusion=` also accepts a callable with the same signature as the ops above. Its name is recorded on `ens.fusion`, or `"custom"` when it has none. The return is validated: it must be a `(boxes, scores, labels)` triple with consistent shapes. --- # Full export matrix Export support is a lookup on the triple (family, task, format). This page describes the shape of that matrix, the rules that fill the cells no explicit entry covers, and how to query it for a combination you care about. Verified against LibreYOLO v1.5.0. ## Shape of the matrix The matrix is keyed by `(family, task, format)`. Family keys are the canonical names from the model registry, task keys come from `libreyolo.tasks.TASKS`, and there are twelve formats: `onnx`, `torchscript`, `executorch`, `tensorrt`, `openvino`, `paddle`, `mnn`, `rknn`, `ncnn`, `tflite`, `coreml`, `coreai`. `model.export(format=...)` additionally accepts two aliases: `engine` for `tensorrt`, and `litert` for `tflite`, which is the current name for TensorFlow Lite. The format and the `.tflite` suffix are unchanged. **Query the matrix, no model needed** ```python from libreyolo.export.support import ( EXPORT_FORMATS, get_support, validated_alternatives, ) print(EXPORT_FORMATS) entry = get_support("yolo9", "detect", "onnx") print(entry.tier, entry.since) print(entry.constraint) print(validated_alternatives("yolo9", "detect")) ``` **CLI** ```bash libreyolo formats --family yolo9 --task detect libreyolo formats --family yolo9 --task detect --json ``` Because a cell is a function of three keys, the full grid is large and changes every release. It is generated rather than written by hand, and lives in `docs/export_support.md` in the library repository. Query the matrix from Python or the CLI rather than reading a copy. ## The three tiers | Tier | Meaning | |---|---| | `validated` | Numeric parity is covered in CI or a documented nightly run | | `available` | Conversion is implemented, but numeric runtime parity evidence has not been recorded | | `blocked` | Preflight raises `NotImplementedError` with a reason before tracing | Validated and available combinations both proceed without an acknowledgement or a blanket warning. Their recorded evidence and constraints stay visible in the generated documentation. A blocked combination fails before dependency checks, calibration loading, tracing or artifact creation. Adding a validated entry requires a parity test and a `since` field. A `SupportEntry` carries four fields: `tier`, a `reason` string, the `since` release, and a `constraint` string. The constraint is the part that matters at integration time: a check mark applies only under the conditions it names, which are typically a fixed input canvas, batch 1, FP32, and a named runtime version. ## How a cell is decided `get_support(family, task, fmt)` resolves in this order. The first rule that matches wins. 1. An unknown task, or a format outside the twelve, returns `blocked`. 2. An explicit `(family, task, format)` entry returns as recorded. 3. A family-wide block returns `blocked` with that family's reason. 4. A task-wide block returns `blocked` with that task's reason. 5. For `ncnn`, a family on the NCNN block list returns `blocked`. 6. `mnn` returns `blocked`: no runtime contract for this family and task. 7. `rknn` returns `blocked`. RKNN in this version is limited to the exact simulator-tested detection variants: YOLO9-t, YOLO9-E2E-t, YOLO-NAS-s and PicoDet-s on RK3588. 8. `tensorrt` and `openvino` return `available`: the converter path exists but runtime parity has not been recorded for that family and task. 9. `tflite`, `paddle`, `coreai` and `coreml` return `blocked`, each with its own reason. 10. Everything else returns `available`: conversion is implemented, numeric runtime parity is not recorded. The asymmetry in steps 8 through 10 is deliberate. TensorRT and OpenVINO convert generically from ONNX, so an unlisted combination is worth attempting. TFLite, Paddle, Core AI and CoreML each need a per-family path, so an unlisted combination is a rejection rather than an invitation. ## Blocked tasks These tasks are blocked for any family with no explicit entry. | Task | Reason | |---|---| | `ocr` | Two networks with dynamic per-region cropping do not fit the single-graph export contract | | `point` | The family is not wired to the shared point heatmap and backend peak-decoding contract | | `semantic` | The family is not wired to the shared dense-logits and backend argmax contract | | `mesh` | Body-mesh graph outputs, metadata and runtime contract are not defined | | `normal` | The family is not wired to the fixed-canvas dense unit-normal and backend renormalization contract | | `panoptic` | Panoptic export has no backend runtime contract | | `gaze` | The family is not wired to the shared two-head logits and backend expectation-decoding contract | An explicit entry overrides these, which is how, for example, a wired semantic family still exports. ## Blocked families | Family | Blocked for | |---|---| | `depth_anything3` | Every format; its depth graph is not in the exported-runtime contract | | `domedetr` | Every format. PAQI sets the query count per image, so a traced graph is valid only for the image it was traced on. Use D-FINE for an exportable DETR | | `eomt` | Instance and panoptic export, which have no runtime parsing | | `l2cs` | Anything outside ONNX, TorchScript, ExecuTorch, TensorRT and OpenVINO | | `hrnet` | Anything outside ONNX, TorchScript, OpenVINO and TensorRT | | `sam`, `sam2`, `sam3`, `edgetam`, `mobilesam` | Every format; promptable model export is out of scope for the v1 runtime contract | | `grounding_dino`, `owlv2`, `omdet_turbo`, `ov_deim` | Every format; open-vocabulary runtime export is out of scope for v1 | | `florence2`, `kosmos2`, `lfm2vl`, `internvl3`, `qwen3vl`, `smolvlm2`, `locateanything` | Every format; generative VLM export is out of scope for v1 | PicoSAM3 is the exception in the promptable tier: it exports its raw 96 pixel ROI network to ONNX. ## Blocked for NCNN DETR-style decoders need sampling operations NCNN does not implement, so these families are blocked for `ncnn` unless an explicit entry says otherwise: Deformable DETR, DETR, DINO-DETR, D-FINE, LW-DETR, DEIM, DEIMv2, RT-DETR, RT-DETRv2, RT-DETRv4, RF-DETR and EC. The rejection names ONNX, OpenVINO, TorchScript and TensorRT as the alternatives. ## Parity thresholds A validated cell means the exported artifact reproduced the native model within these bounds: | Task group | Threshold | |---|---| | Detection and OBB | Matched box IoU above 0.95, score MAE below 0.01 | | Segmentation and panoptic | Mask IoU above 0.95 | | Pose | Keypoint L2 below 2 pixels at native resolution | | Classification | Logits cosine above 0.999 and equal top-1 class | | Depth and restoration | PSNR above 40 dB against native output | | Surface normals | Mean angular error below 0.1 degree | | Point | Peak locations equal within one output cell | DETR query rows are an unordered set, so DETR-family parity aligns query rows as a set rather than positionally. ## Exporting **Export, and read a rejection** ```python from libreyolo import LibreYOLO from libreyolo.export.support import get_support model = LibreYOLO("LibreYOLO9t.pt") print(model.export(format="onnx")) # Check before calling: a blocked combination raises in preflight # and the message carries this reason. blocked = get_support("domedetr", "detect", "onnx") print(blocked.tier) print(blocked.reason) ``` A blocked combination raises `NotImplementedError` in preflight and the message carries the recorded reason. `validated_alternatives(family, task)` returns the formats that are validated for that pair, which is the useful thing to print next to a rejection. The arguments every exporter shares are listed on the [model API page](/docs/reference/model-api). Format-specific arguments live on the individual format pages. ## Reading a constraint A validated cell is a claim about one measured configuration, not about the format in general. A constraint string such as `FP32, batch 1, fixed 520x520 input` means parity was recorded at that shape and precision. Exporting at a different resolution or batch size still produces an artifact; it just is not the configuration the number came from. --- # Kernels Every accelerated operation in LibreYOLO has a portable default and, sometimes, a faster variant registered on top of it. Selection happens at runtime by predicate, a missing optional dependency is a fallback rather than an error, and an exported graph always takes the portable path. Verified against LibreYOLO v1.5.0. ## The registry `libreyolo/kernels/` is a small runtime registry of pluggable implementations. An op slot is a name such as `fake_quant_fp8` or `ms_deform_attn`. Callers ask the registry for a slot and get back whichever registered implementation passes its predicate first, newest registration winning, falling through to the reference implementation when nothing else applies. That structure exists so that an optional dependency is never a hard requirement. A machine without Triton, without CUDA, or without the `kernels` package runs the same code and produces the same numbers, only slower. | Function | Purpose | |---|---| | `active()` | Op slot to selected implementation name, or `"unavailable"` | | `resolve(op)` | The callable that would run, or `None` | | `register(op, impl, *, name, predicate=None)` | Add an implementation, newest first | | `unregister(op, name)` | Remove one | | `clear_cache()` | Drop the memoized resolution | **See what is selected** ```python import libreyolo.kernels as kernels # Op slot to selected implementation name, or "unavailable". print(kernels.active()) ``` **Force the reference path** ```bash # off and reference both mean the same thing, and also skip # importing the accelerated providers at all. LIBREYOLO_KERNELS=off python train.py ``` **Turn off Hub kernels without uninstalling** ```bash LIBREYOLO_HUB_KERNELS=0 python predict.py ``` **Switch a family to fused attention** ```python from libreyolo import LibreYOLO from libreyolo.kernels.attention import set_fused_attention model = LibreYOLO("LibreSwinIRs.pt") # Returns how many attention modules switched. print(set_fused_attention(model)) ``` **Register your own** ```python import libreyolo.kernels as kernels kernels.register( "fake_quant_fp8", my_impl, name="mybackend", predicate=my_check, ) ``` A predicate that raises is caught and warned about, never propagated, so a broken third-party implementation degrades to the portable path instead of breaking prediction. ### Layout The tree is organized by purpose first and backend second, so a slot is found by what it computes rather than by which library happens to implement it today. | Directory | Contents | |---|---| | `kernels/quant/simulate/` | Fake-quantization Triton kernels, with straight-through backward, on any device. Used by QAT and by simulated post-training quantization alike | | `kernels/quant/execute/` | Real-precision paths for finalized models only, no backward: the FP8 tensor-core GEMM, its fused Triton prologue and epilogue, and the packed-weight unpack kernels | | `kernels/attention/` | Attention ops shared across families: the `ms_deform_attn` slot, and the fused-SDPA policy | The boundary between `simulate` and `execute` is whether the model is finalized, not whether it is training or deploying. The reference implementations stay in `libreyolo/quant/`, which defines what the numbers mean; `kernels/` only makes them fast. Weight packing has no variants at all, because it is the checkpoint contract. GEMM and attention slots have no reference implementation. A caller has to check that `resolve()` returned something and keep its own portable path, which is why ONNX, TensorRT and `torch.export` graphs always contain the portable math. ### Selection overrides `LIBREYOLO_KERNELS=off` or `=reference` forces reference implementations and short-circuits the import of the accelerated providers entirely. Any other value restricts selection to implementations registered under that name. `LIBREYOLO_QUANT_KERNELS` is honored as a legacy alias from when the registry lived under `libreyolo/quant/`, and is read only when `LIBREYOLO_KERNELS` is unset. Both are listed with the rest on [settings](/docs/reference/settings). ## Hub kernels Compiled CUDA kernels published on the Hugging Face Hub load at runtime through the optional `kernels` package. Nothing is vendored into LibreYOLO; the artifact is fetched and cached by that package, and each provider pins an audited commit revision, so bumping a pin requires a GPU parity run before it lands. Installing the extra is the opt-in: ```bash pip install "libreyolo[hub-kernels]" ``` Without the package nothing changes and no network request is made. `LIBREYOLO_HUB_KERNELS=0` disables the fetch without uninstalling anything. A kernel that fails to load or to run disables itself for the rest of the process and falls back with one warning. One slot is Hub-backed today: `ms_deform_attn`, the compiled multi-scale deformable attention forward and backward from Deformable DETR, under Apache 2.0. It is wired into the whole deformable lineage: RF-DETR, Deformable DETR, DINO-DETR, LW-DETR, Grounding DINO, RT-DETR, RT-DETRv2, D-FINE, RT-DETRv4, DEIM, DEIMv2, EC and OV-DEIM. Because the backward is compiled too, training benefits as well as prediction. Eligibility is narrow on purpose. Inputs must be CUDA and float32, and execution must be eager: the provider declines under `torch.jit.is_tracing()`, `torch.compiler.is_compiling()`, `torch.compiler.is_exporting()` and `torch.onnx.is_in_onnx_export()`. Two input layouts also fall through to the portable path, a per-level point count that varies between levels, and discrete integer-index sampling. The EC pose variant is not wired. ### This kernel is newly reachable Read this before installing the extra on an existing project. In v1.4.0 the slot was consulted from inside a helper, behind a condition that required the spatial-shape pairs to be absent. RF-DETR always threads those pairs through its decoder, so the condition never held and the kernel never executed in any eager forward. The consult moved in v1.5.0, and the kernel now actually runs. The practical consequence is that upgrading to v1.5.0 *and* installing `libreyolo[hub-kernels]` on CUDA means RF-DETR and its lineage take their forward from a compiled binary for the first time. Predictions and metrics can shift at float tolerance as a result. A stock install, without the extra, is unaffected. If you are comparing metrics across the upgrade, hold the extra fixed or set `LIBREYOLO_HUB_KERNELS=0` on both sides. ## Fused attention Fused scaled dot-product attention needs no optional dependency, only stock PyTorch, so it is governed by policy rather than by availability. Two rules apply. First, a graph capture never uses it. Every swapped call site keeps the primitive-op equation available behind an export check, covering ONNX export, whose default opset has no SDPA symbolic, and `torch.jit.trace`, which TorchScript, CoreML and NCNN all go through. Dynamo captures are deliberately outside the gate, because `torch.compile` lowers SDPA better than the manual math, and both Core AI and ExecuTorch decompose SDPA to core ATen on their own. Second, the parity bar for making it the default is byte exact. Families that clear it use SDPA by default: SegFormer, Depth Anything and MoGe-2, BERT, Grounding DINO, SwinIR and PP-OCR. Families that do not keep manual math and expose a `fused_attn` flag instead, which is what `set_fused_attention(model)` flips: Swin, DINO-DETR's Swin backbone, BiRefNet and FeyNobg, OWLv2, LW-DETR, SigLIP 2, ZipDepth and MobileSAM. ViT and DeiT carry the same flag but default it on, following upstream, so the same call with `enabled=False` turns them off. It is worth doing where it applies. On an RTX 5070 Ti under fp16 autocast, Swin window attention goes from 1.278 ms to 0.721 ms, a 1.77x gain, and OWLv2 vision attention from 6.483 ms to 1.735 ms, 3.74x. ## Hardware | Platform | Behavior | |---|---| | CPU and MPS | Every CUDA and Triton predicate fails, so everything runs reference | | NVIDIA CUDA | Triton kernels and eligible Hub and GEMM kernels engage | | AMD ROCm | Triton can engage, since ROCm wheels ship Triton's AMD backend, but parity is only exercised on NVIDIA in CI | ## Adding an implementation Call `register()` with a name and a predicate. Out-of-tree compiled kernels can ship as a separate `libreyolo_kernels` package that registers itself on import, which keeps a private backend out of the LibreYOLO tree entirely. Parity is the gate for anything in-tree: an exact forward match against the reference, and gradients within 1e-6 of the straight-through estimator, over the shape set the test suite carries. Kernel selection interacts with [CUDA graphs](/docs/reference/cuda-graphs): the inference parity matrix ran without the `kernels` package installed, so capture safety with a compiled kernel active is not covered by it. --- # Model API A loaded LibreYOLO model is an instance of BaseModel. This page lists the methods that instance carries, with the signatures and defaults read from libreyolo/models/base/model.py. Verified against LibreYOLO v1.5.0. ## Construction The factory returns a family class instance. Constructing that class directly takes the same arguments, except that `size` is required: ```python Family(model_path, size, nb_classes=80, device="auto", task=None, **kwargs) ``` `device="auto"` selects CUDA when available, then MPS, then CPU. An integer or a digit string is read as a CUDA ordinal, so `device=0` and `device="0"` both mean `cuda:0`. `task` is validated against the family's `SUPPORTED_TASKS`. Passing `model_path=None` builds the architecture and leaves it in training mode; passing a `dict` loads that state dict directly. ## predict and \_\_call\_\_ `predict` is an alias for `__call__`. ```python model( source=None, *, conf=0.25, iou=0.45, imgsz=None, device=None, classes=None, max_det=300, augment=False, save=False, batch=1, stream=False, stream_buffer=False, vid_stride=1, show=False, output_path=None, color_format="auto", tiling=False, overlap_ratio=0.2, output_file_format=None, cuda_graph=False, **kwargs, ) ``` | Argument | Default | Meaning | |---|---|---| | `source` | `None` | Image, list or tuple of in-memory images, directory, video file, or a screen source such as `"screen"`, `"screen 1"`, `"screen 1 100 200 512 256"` | | `conf` | `0.25` | Confidence threshold | | `iou` | `0.45` | IoU threshold for NMS | | `imgsz` | `None` | Input size override; `None` uses the model's native size | | `device` | `None` | Device override for this call | | `classes` | `None` | Keep only these class IDs | | `max_det` | `300` | Maximum detections per image | | `augment` | `False` | Test-time augmentation | | `save` | `False` | Write an annotated image or video | | `batch` | `1` | Images per forward pass for directory and list sources | | `stream` | `False` | Return a generator instead of a materialized list | | `stream_buffer` | `False` | Keep every captured live frame instead of only the newest | | `vid_stride` | `1` | Process every N-th video or screen frame | | `show` | `False` | Display annotated frames in a window | | `output_path` | `None` | Output path when `save=True` | | `color_format` | `"auto"` | Color format hint for in-memory arrays | | `tiling` | `False` | Tiled inference for large images | | `overlap_ratio` | `0.2` | Tile overlap ratio | | `output_file_format` | `None` | `"jpg"`, `"png"` or `"webp"` | | `cuda_graph` | `False` | `True` captures on first use per input shape, `"auto"` waits for a shape to repeat | A single image source returns one `Results`. A list, a tuple or a directory returns a list of them, and `stream=True` returns a generator in every case. Live stream sources are unbounded and require `stream=True`. `tiling` and `augment` cannot be combined. Test-time augmentation raises for the `embed`, `point` and `edge` tasks. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9t.pt") model.info() result = model(SAMPLE_IMAGE, conf=0.25, iou=0.45) print(result.boxes.xyxy) print(result.speed) ``` With `batch > 1`, families whose `SUPPORTS_BATCHED_PREDICT` is true run one stacked forward per chunk; `batch=1` keeps one forward per image. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9t.pt") # stream=True returns a generator, one Results per frame or image. for result in model([SAMPLE_IMAGE, SAMPLE_IMAGE], stream=True): print(len(result)) ``` ## embed ```python model.embed(source=None, **kwargs) -> torch.Tensor ``` A convenience wrapper over `predict` that stacks every embedding row into a single `(N_total, D)` tensor. The model must have been constructed with `task="embed"`, otherwise it raises `NotImplementedError`. ## track ```python model.track( source, *, track_conf=0.25, iou=0.45, imgsz=None, classes=None, max_det=300, save=False, show=False, vid_stride=1, output_path=None, tracker="bytetrack", tracker_config=None, augment=False, **tracker_kwargs, ) -> Generator[Results, None, None] ``` Yields one `Results` per frame with `track_id` set. `tracker` is `"bytetrack"`, `"botsort"`, `"ocsort"` or `"deepocsort"`, and is ignored when `tracker_config` is given because the config type selects the tracker. `track_conf` maps to `track_high_thresh` for ByteTrack and BoT-SORT and to `det_thresh` for OC-SORT and Deep OC-SORT. `output_path` defaults to `runs/track/.mp4`. ## val ```python model.val( data=None, batch=16, imgsz=None, conf=0.001, iou=0.6, workers=4, allow_download_scripts=False, device=None, split="val", augment=False, save_json=False, verbose=True, *, plots=None, **kwargs, ) -> Dict ``` Returns a metrics dictionary whose keys depend on the task; detection returns `metrics/precision`, `metrics/recall`, `metrics/mAP50` and `metrics/mAP50-95`. `imgsz` accepts a square int or a `(height, width)` tuple and defaults to the model's native input size. `plots` is an alias for `save_plots`. `allow_download_scripts` gates the embedded Python that a dataset YAML may carry in its `download` field. `faster_coco_eval` is accepted through `**kwargs` and defaults to `True`, falling back to pycocotools when the package is not installed. The backend that ran is reported on `model.last_eval_backend`. Augmented validation raises for the `obb` and `pose` tasks. ## train `train` is defined per family, so its arguments differ. Two behaviors are shared, because the base class wraps every family's `train`: - `cfg=` takes a YAML path whose keys are merged into the call. Explicit keyword arguments win over the file. - `pretrained=False` on a family in coverage group `g0` or `g1` reinitializes the model from scratch before training, and cannot be combined with `resume=True`. Which augmentation knobs a family actually honors is a per-family question; see the [augmentation matrix](/docs/reference/augmentation-matrix). ## export ```python model.export(format="onnx", **kwargs) -> str ``` Returns the path to the written artifact. `format` is resolved through the exporter registry, where `engine` is an alias for `tensorrt` and `litert` is an alias for `tflite`. Arguments shared by every exporter: | Argument | Default | Meaning | |---|---|---| | `output_path` | `None` | Output file path; generated under `weights/` when omitted | | `imgsz` | `None` | `(height, width)` tuple or a single int; defaults to the native size | | `opset` | `None` | ONNX opset version | | `simplify` | `True` | Run ONNX graph simplification | | `dynamic` | `True` | Enable dynamic axes | | `half` | `False` | FP16 precision | | `int8` | `False` | INT8 precision | | `batch` | `1` | Batch size baked into the artifact | | `device` | `None` | Device to trace on | | `data` | `None` | data.yaml for INT8 calibration | | `fraction` | `1.0` | Fraction of the calibration dataset to use | | `allow_download_scripts` | `False` | Allow embedded Python in dataset YAML downloads | | `verbose` | `False` | Verbose exporter logging | Blocked combinations raise `NotImplementedError` in preflight, before tracing. Coverage and its rules are on the [export matrix](/docs/reference/export-matrix) page. When live LoRA adapters are present they are folded into dense weights, and that merge happens only after every request rejection. ## save ```python model.save(path) -> str ``` Writes a schema v1.0 LibreYOLO checkpoint: the state dict plus the metadata described in the [checkpoint schema](/docs/reference/checkpoint-schema). A quantized model additionally carries its `quant` manifest, so `LibreYOLO(path)` restores the quantized structure and scales. ## quantize, quant_info and dequantize ```python model.quantize( recipe, calib="coco128.yaml", samples=128, batch=8, algorithm="auto", keep_high_precision=None, allow_download_scripts=False, verbose=True, ) ``` Quantizes in place and returns the model. `recipe` is one of the casts `fp16` and `bf16`, the Conv and Linear recipes `int8` and `fp8`, or the Linear-only recipes `w4a16`, `w4a8`, `nvfp4`, `mxfp4` and `int2`, which transformer families such as RF-DETR support. `int2` requires QAT. `calib` takes a data.yaml path or a built-in dataset name and reads images forward-only; labels are never read. Pass `calib=None` to skip calibration. `algorithm` is `"minmax"`, `"percentile"` or `"auto"`. `model.quant_info()` returns the quantization state summary, or `None` for a float model. `model.dequantize()` restores float modules in place while keeping the quantization-trained master weights, which is the bridge from QAT to `export(format="onnx", int8=True, data=...)`. ## info and layers ```python model.info(detailed=False, verbose=True) -> Dict[str, Any] model.get_available_layer_names() -> List[str] model.get_distill_config() -> Dict ``` `info` returns a JSON-friendly dictionary and logs a human-readable summary when `verbose` is true. `get_available_layer_names` lists the layers a distillation or feature-extraction config can name. ## CUDA graphs Available on families whose `SUPPORTS_CUDA_GRAPH` class attribute is true. Replay is bit-identical to eager execution. ```python model.capture_graph(imgsz=None, batch=1, dtype=None) -> None model.cuda_graph_scope(mode=True) # context manager model.graph_info() -> Dict[str, Any] model.release_graphs() -> None ``` A captured graph is valid only for the exact shape it was captured at, so `batch` and `imgsz` must match the later `predict` call. `capture_graph` moves the capture cost off the first request. `mode` accepts `True` or `"on"` to capture on first use, `"auto"` to wait until a shape repeats, and `False` for a no-op. `capture_graph` raises `NotImplementedError` when the family has not opted in and `CudaGraphUnavailable` when capture fails. ## Device and dtype `Results` objects carry `.to()`, `.cpu()`, `.cuda()` and `.numpy()`; see [Results types](/docs/reference/results-types). The model itself is moved by passing `device=` to `predict`, or at construction time. --- # Open-vocabulary API LibreOpenVocab is the factory for text-conditioned detectors. The class list is a prompt rather than a fixed head, so the vocabulary is set by set_classes and the model returns ordinary detection Results against it. Verified against LibreYOLO v1.5.0. ## Install The tier needs the `openvocab` extra. **bash** ```bash pip install 'libreyolo[openvocab]' ``` ## The factory ```python LibreOpenVocab(model: str = "grounding-dino-tiny", **kwargs) -> LibreOpenVocabDetector ``` `model` is an alias, not a path. Underscores fold to hyphens before lookup, so the family-qualified names the CLI inventory prints, such as `omdet_turbo-t` and `grounding_dino-t`, load as given. An unknown alias raises `ValueError` listing every known alias. The constructor accepts `size`, `nb_classes=80`, `names=None`, `device="auto"`, `task=None` and `text_threshold=None`. Passing `names` is the same as calling `set_classes` right after loading. Passing `text_threshold` to a family that does not support it raises `TypeError`. **Python** ```python from libreyolo import LibreOpenVocab, SAMPLE_IMAGE model = LibreOpenVocab("grounding-dino-tiny") model.set_classes(["person", "skateboard", "handrail"]) result = model.predict(SAMPLE_IMAGE) for box, cls in zip(result.boxes.xyxy, result.boxes.cls): print(result.names[int(cls)], box.tolist()) ``` ## Families and aliases | Family | Aliases | Sizes | Weights | |---|---|---|---| | Grounding DINO | `grounding-dino`, `groundingdino`, `grounding-dino-tiny`, `groundingdino-tiny`, `grounding-dino-t`, `groundingdino-t`, `grounding-dino-base`, `groundingdino-base`, `grounding-dino-b`, `groundingdino-b` | `t`, `b` | `LibreYOLO/LibreGroundingDINOt`, `LibreYOLO/LibreGroundingDINOb` | | OWLv2 | `owlv2`, `owl-v2`, `owlv2-base`, `owl-v2-base`, `owlv2-b16`, `owl-v2-b16`, `owlv2-large`, `owl-v2-large`, `owlv2-l14`, `owl-v2-l14` | `b16`, `l14` | `LibreYOLO/LibreOWLv2b16`, `LibreYOLO/LibreOWLv2l14` | | OMDet-Turbo | `omdet-turbo`, `omdet`, `omdetturbo`, `omdet-turbo-tiny`, `omdet-turbo-swin-tiny`, `omdet-turbo-t` | `t` | `LibreYOLO/LibreOMDetTurbot` | | OV-DEIM | `ov-deim`, `ovdeim`, `ov-deim-s`, `ovdeim-s`, `ov-deim-m`, `ovdeim-m`, `ov-deim-l`, `ovdeim-l` | `s`, `m`, `l` | `LibreYOLO/LibreOVDEIMs`, `LibreYOLO/LibreOVDEIMm`, `LibreYOLO/LibreOVDEIMl` | The default alias is `grounding-dino-tiny`. `LibreGroundingDINO`, `LibreOWLv2` and `LibreOMDetTurbo` are exported at package level and can be constructed directly with `size=`. OV-DEIM is reachable through the factory aliases above. ## set_classes ```python model.set_classes(classes: list[str]) -> LibreOpenVocabDetector ``` Sets the vocabulary for every later `predict()` call, and returns the model so calls can chain. The list must be non-empty, must contain only strings, and its entries must be unique when compared case-insensitively; blank labels are rejected. Passing a bare string raises `TypeError`, because it would enumerate into one-character classes. After the call, `model.names` maps `0..N-1` to the labels in the order given, and `model.nb_classes` is `N`. ## Call arguments The tier reuses the standard predict surface with three differences. `conf` defaults to the family's own value rather than the shared 0.25: | Family | Default conf | Suppression | |---|---|---| | Grounding DINO | 0.25 | | | OWLv2 | 0.1 | | | OMDet-Turbo | 0.3 | Its own post-processing, threshold 0.5, honors `iou=` | | OV-DEIM | 0.25 | One-to-one matching with top-K selection, no suppression | `iou=` only means something for a family that runs suppression. OMDet-Turbo takes the threshold as an argument and defaults it to 0.5 when `iou=` is unset. The other three suppress nothing, so passing `iou=` there emits a warning and is ignored. `text_threshold=` is Grounding DINO only, where it defaults to 0.25. It can be passed at construction for a persistent value, or per call. A per-call value cannot be combined with `stream=True`, because streamed results are generated lazily; set it on the constructor instead. Every other family raises `TypeError` for it. `imgsz=` raises `ValueError`: the preprocessing pipeline owns resizing for this tier. `augment=True` raises as well, since test-time augmentation is out of scope here. Input sizes are recorded per family for reference only: Grounding DINO 800, OWLv2 960 and 1008, OMDet-Turbo 640, OV-DEIM 640. ## Not supported `train()`, `val()`, `track()` and `export()` all raise `NotImplementedError`. Fine-tune upstream and load the resulting weights; run `predict()` per frame in place of tracking. Validation would need a dedicated validator, because the shared detection validator calls the model with image tensors while this tier requires text-conditioned inputs. --- # Python API The public Python surface of LibreYOLO is the __all__ list in libreyolo/__init__.py. Everything on this page is importable as from libreyolo import ; anything not on that list is internal. Verified against LibreYOLO v1.5.0. ## Entry points Five callables load a model. They are separated by call contract, not by architecture. | Factory | Loads | Prompt at call time | Extra required | |---|---|---|---| | `LibreYOLO` | Promptless families, by sniffing the checkpoint or file suffix | | | | `LibreSAM` | Promptable segmenters, by size alias | Points, boxes, or concept text | `sam` | | `LibreVLM` | Generative vision-language detectors, by alias | Class vocabulary or a free-form prompt | `vlm` | | `LibreOpenVocab` | Text-conditioned detectors, by alias | Class vocabulary | `openvocab` | | `LibreEnsemble` | Two or more detectors, fused into one surface | | | **The five entry points** ```python from libreyolo import LibreYOLO, LibreEnsemble # Weight-sniffing factory over the promptless families. detector = LibreYOLO("LibreYOLO9t.pt") # Two or more detectors behind one prediction surface. ens = LibreEnsemble(["LibreYOLO9t.pt", "LibreYOLO9s.pt"]) # The other three factories need an extra installed: # pip install 'libreyolo[sam]' -> from libreyolo import LibreSAM # pip install 'libreyolo[vlm]' -> from libreyolo import LibreVLM # pip install 'libreyolo[openvocab]' -> from libreyolo import LibreOpenVocab print(type(detector).__name__, ens.fusion) ``` `LibreYOLO` is the only one that reads a file. The other three take a string alias and resolve it to a Hugging Face repository, so the argument is a model name and not a path. ```python LibreYOLO( model_path: str, size: str | None = None, reg_max: int = 16, nb_classes: int | None = None, device: str = "auto", task: str | None = None, compute_units: str = "all", ) ``` `model_path` accepts a `.pt` checkpoint, an ONNX `.onnx` file, an ExecuTorch `.pte`, an MNN `.mnn`, a TensorRT `.engine`, an OpenVINO, Paddle or ncnn directory, or a Triton HTTP or HTTPS model URL. `size` and `nb_classes` are read from the checkpoint when omitted. `compute_units` is read only for CoreML `.mlpackage` loads and is one of `all`, `cpu_only`, `cpu_and_gpu`, `cpu_and_ne`. `task` takes any canonical task name from `libreyolo.tasks.TASKS`. **Load anything through one factory** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9t.pt") # A single image source returns one Results; a list or directory # returns a list of them. result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) print(result.names) ``` **Import a family class directly** ```python from libreyolo import LibreYOLO9, SAMPLE_IMAGE model = LibreYOLO9("LibreYOLO9t.pt", size="t") result = model(SAMPLE_IMAGE) print(len(result)) ``` ## Family classes Every family the factory can return is also exported by name, so a class can be constructed directly when the checkpoint is known in advance. The constructors follow `BaseModel.__init__`: ```python Family(model_path, size, nb_classes=80, device="auto", task=None, **kwargs) ``` `size` has no default on a family class, which is the difference from the factory. YOLO9 and its variants insert `reg_max: int = 16` after `size`. Detection and multi-task families: `LibreYOLO9`, `LibreYOLO9E2E`, `LibreYOLO9P2`, `LibreYOLONAS`, `LibreYOLOX`, `LibreYOLO7`, `LibreYOLO4`, `LibreYOLO3`, `LibreYOLO2`, `LibreYOLO1`, `LibreRTDETR`, `LibreRTDETRv2`, `LibreRTDETRv4`, `LibreRFDETR`, `LibreDFINE`, `LibreDOMEDETR`, `LibreDEIM`, `LibreDEIMv2`, `LibreDETR`, `LibreDeformableDETR`, `LibreDINODETR`, `LibreLWDETR`, `LibreMaskRCNN`, `LibreFCOS`, `LibreFasterRCNN`, `LibreRetinaNet`, `LibreSSD`, `LibreCenterNet`, `LibreEfficientDet`, `LibreEC`, `LibrePICODET`, `LibreRTMDet`, `LibreFOMO`. Dense-prediction families: `LibreMiDaS`, `LibreDepthAnythingV2`, `LibreDepthAnything3`, `LibreZipDepth`, `LibreMoGe2`, `LibreTEED`, `LibreDexiNed`, `LibreNAFNet`, `LibreRealESRGAN`, `LibreSwinIR`, `LibreBiRefNet`, `LibreFeyNobg`, `LibreFCN`, `LibreEoMT`, `LibreDeepLabv3`, `LibrePIDNet`, `LibreSegformer`, `LibreLingBotVision`. Classification and embedding families: `LibreViT`, `LibreMobileNetV4`, `LibreConvNeXt`, `LibreDeiT`, `LibreSwin`, `LibreEfficientNetV2`, `LibreVGG`, `LibreResNet`, `LibreAlexNet`, `LibreCLIP`, `LibreSigLIP2`, `LibreDINOv2`. Other tasks: `LibreHRNet` (pose), `LibreL2CS` (gaze), `LibrePPOCR` (ocr), `LibreFaceEmbedder` (embed). The sibling tiers export their family classes too: `LibreSAM1`, `LibreSAM2`, `LibreSAM3`, `LibreEdgeTAM`, `LibreMobileSAM`, `LibrePicoSAM3`; `LibreGroundingDINO`, `LibreOWLv2`, `LibreOMDetTurbo`; `LibreLFM2VL`, `LibreQwen3VL`, `LibreSmolVLM2`, `LibreInternVL3`, `LibreFlorence2`, `LibreKosmos2`, `LibreLocateAnything`, `LibreMODUS` (also spelled `LibreModus`). ## Prediction surface Calling a model runs inference. `predict` is an alias for `__call__`, so the two are interchangeable. ```python model( source=None, *, conf=0.25, iou=0.45, imgsz=None, device=None, classes=None, max_det=300, augment=False, save=False, batch=1, stream=False, stream_buffer=False, vid_stride=1, show=False, output_path=None, color_format="auto", tiling=False, overlap_ratio=0.2, output_file_format=None, cuda_graph=False, **kwargs, ) ``` A single image source returns one `Results`. A list, a tuple or a directory returns a list of them, and `stream=True` returns a generator. The other methods on the model object are documented on the [model API page](/docs/reference/model-api). ## Results payloads `Results` and its eighteen payload classes are exported at package level: `Results`, `Boxes`, `Masks`, `Keypoints`, `Points`, `Probs`, `OBB`, `Gaze`, `SemanticMask`, `PanopticSegmentation`, `DepthMap`, `EdgeMap`, `NormalMap`, `RestoredImage`, `Matte`, `Meshes`, `OCRRegions`, `Embeddings`, `Identities`. Each one is described in [Results types](/docs/reference/results-types). ## Backends Exported artifacts load through `LibreYOLO()` by file suffix, so the backend classes are rarely constructed by hand. They are exported for the cases where a backend has to be selected explicitly: `OnnxBackend`, `OpenVINOBackend`, `PaddleBackend`, `TensorRTBackend`, `TritonBackend`, `NcnnBackend`, `CoreMLBackend`, plus `create_triton_config`. `BaseExporter` is the exporter registry behind `model.export()`. ## Validators `model.val()` dispatches to the right validator by task, so these are exported for direct use and for subclassing: `DetectionValidator`, `SegmentationValidator`, `PoseValidator`, `SemanticValidator`, `PanopticValidator`, `DepthValidator`, `NormalValidator`, `EdgeValidator`, and the shared `ValidationConfig`. ## Tracking `model.track()` selects a tracker by name. The tracker classes and their configuration dataclasses are also exported: `ByteTracker` with `TrackConfig`, `BoTSortTracker` with `BoTSortConfig`, and `OCSortTracker` with `OCSortConfig`. ## Data helpers `DATASETS_DIR` is the resolved dataset root, `load_data_config` reads a dataset YAML, and `check_dataset` validates one. The task-specific loaders named in [Dataset formats](/docs/reference/dataset-formats) live in `libreyolo.data` rather than at package level. ## Galleries and distillation `Gallery` and `FaceGallery` hold enrolled identity vectors for the `embed` task and produce the `Identities` payload. `Distiller` and `get_distill_config` drive teacher-student training. ## Assets `SAMPLE_IMAGE` is an absolute path to an image bundled with the package, so every snippet in these docs runs without downloading a picture first. ## Lazy imports and renamed classes Most sibling-tier names, the backends, the validators and the data helpers resolve through the module-level `__getattr__`, so importing `libreyolo` does not import their dependencies. The import still fails with a clear message when the required extra is missing. Two class names were renamed and the old spelling still resolves, with a `DeprecationWarning`: `LibreYOLORTDETR` is now `LibreRTDETR`, and `LibreYOLORFDETR` is now `LibreRFDETR`. --- # Results types Results is the single per-image return type of every LibreYOLO model. It carries eighteen optional payload slots, one per task shape, and populates only the ones the model produced. Verified against LibreYOLO v1.5.0. ## The Results object One `Results` describes one image. A single image source returns one of them, a list source or a directory returns a list, and `stream=True` returns a generator that yields them. | Attribute | Type | Meaning | |---|---|---| | `orig_shape` | `(int, int)` | Original image height and width | | `path` | `str` | Source path when the input came from disk | | `names` | `dict[int, str]` | Class index to class name | | `speed` | `dict[str, float]` | Per-stage milliseconds | | `track_id` | tensor | Track IDs when the result came from `track()` | | `frame_idx` | `int` | Frame index for video and stream sources | | `restore_scale` | `int` | Output-to-input upscale factor of a restore result; `1` everywhere else | **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9t.pt") result = model(SAMPLE_IMAGE) print(result.orig_shape, result.path) print(result.boxes.xyxy) print(result.boxes.conf) print(result.names[int(result.boxes.cls[0])]) ``` ## Payload slots Each slot is `None` unless the model produced it. The slot a family fills is decided by its task. | Slot | Class | Task | |---|---|---| | `boxes` | `Boxes` | detect | | `masks` | `Masks` | segment | | `keypoints` | `Keypoints` | pose | | `probs` | `Probs` | classify | | `obb` | `OBB` | obb | | `gaze` | `Gaze` | gaze | | `points` | `Points` | point | | `semantic_mask` | `SemanticMask` | semantic | | `panoptic` | `PanopticSegmentation` | panoptic | | `depth_map` | `DepthMap` | depth | | `normal_map` | `NormalMap` | normal | | `edges` | `EdgeMap` | edge | | `restored` | `RestoredImage` | restore | | `matte` | `Matte` | matte | | `ocr` | `OCRRegions` | ocr | | `embeddings` | `Embeddings` | embed | | `identities` | `Identities` | embed, with a gallery | | `meshes` | `Meshes` | mesh | `result.normals` is a read-write alias for `result.normal_map`. More than one slot can be set at once. A segmentation model fills both `boxes` and `masks`; a gaze model fills `boxes` with the face boxes and `gaze` with the angles; a mesh model fills `boxes` with person boxes and `meshes` row-aligned to them. ## Boxes Detection boxes for one image. | Member | Returns | |---|---| | `xyxy` | Corner coordinates in original-image pixels | | `xywh` | Center and size in pixels | | `xyxyn` | Corners normalized to `[0, 1]` | | `xywhn` | Center and size normalized to `[0, 1]` | | `conf` | Confidence per box | | `cls` | Class index per box | | `id` | Track ID per box, or `None` | | `is_track` | `True` when track IDs are present | | `data` | The packed tensor | `with_id(id)` and `with_orig_shape(orig_shape)` return a new `Boxes` with that field replaced. ## Masks Instance masks for one image. `data` is the mask tensor; `xy` returns per-instance contours in pixels and `xyn` returns them normalized. ## Keypoints Pose keypoints, row-aligned with `boxes`. `xy` is the coordinate pair per keypoint and `xyn` the normalized pair. `conf` is the third channel when the data carries one, otherwise `None`. `has_visible` is a boolean array, true where `conf > 0`, and all-true when there is no confidence channel. ## Points Point localization for one image. `data` has shape `(N, 4)` with rows `x, y, class, confidence`. Coordinates are absolute pixels; `xy` and `cls` and `conf` split the columns, and `xyn` normalizes the coordinates. ## Probs Classification scores. `top1` is the winning index, `top5` the five best indices, and `top1conf` and `top5conf` their scores. ## OBB Oriented boxes. `data` holds 7 or 8 values per row: `xywhr`, an optional track ID, then confidence and class. | Member | Returns | |---|---| | `xywhr` | Center, size and rotation in radians | | `xyxyxyxy` | The four corners in pixels | | `xyxyxyxyn` | The four corners normalized | | `xyxy` | Axis-aligned hull in pixels | | `conf`, `cls`, `id`, `is_track` | As on `Boxes` | ## Gaze Per-face gaze angles in radians, shape `(N, 2)`, row-aligned with the face boxes in `boxes`. Column 0 is pitch and column 1 is yaw, in the L2CS convention: positive yaw rotates the gaze toward the subject's left and positive pitch rotates it downward. `pitch_deg` and `yaw_deg` convert to degrees, and `direction_3d` returns the unit direction vector. ## SemanticMask Dense semantic map, shape `(H, W)` of integer class IDs on the original image canvas. `255` is the ignore value and never counts as a class (`SemanticMask.IGNORE_INDEX`). `classes` lists the class IDs present, and `class_mask(class_id)` returns the boolean mask for one class. ## PanopticSegmentation Every pixel gets exactly one non-overlapping segment, unifying stuff regions and thing instances. `data` is a `(H, W)` integer segment-ID map; segment ID `0` is unlabeled (`PanopticSegmentation.IGNORE_INDEX`). `segments_info` is a list of dicts, one per segment, each with at least `{"id": int, "category_id": int}`, where `id` matches a value in the map and `category_id` indexes `names`. `segment_ids` lists the IDs present and `segment_mask(segment_id)` returns one segment's boolean mask. Thing-versus-stuff is a property of the category, not of the segment. A payload may denormalize it onto each segment as `"isthing": bool`, and when it does, the value must agree with the category-level map. ## DepthMap Dense relative inverse-depth map, shape `(H, W)` of floats on the original image canvas. Higher values mean closer to the camera. Values are relative, not metric meters. `min`, `max` and `mean` are computed over finite values, and `normalized()` rescales the map to `[0, 1]`. ## NormalMap Dense surface-normal field, float32 `(H, W, 3)` on the original image canvas, in the OpenCV camera frame: `+x` right, `+y` down, `+z` into the scene. Normals face the camera, so a fronto-parallel surface is `(0, 0, -1)`. Every pixel is a unit vector. `assert_normalized(atol=1e-4)` checks that invariant. ## EdgeMap Dense edge-probability map, float32 `(H, W)` on the original image canvas, where `0` is non-edge and `1` is edge. The continuous map is kept so the threshold stays the caller's choice: `binary(threshold=0.5)` applies one, and `array` returns the numpy view. ## RestoredImage The restored RGB image, `(H, W, 3)` uint8. For super-resolution the canvas is `Results.restore_scale` times the input. `array` returns the numpy view and `save(path)` writes the image. ## Matte Soft opacity matte, float32 `(H, W)` in `[0, 1]` on the original image canvas. `1` is fully foreground and `0` is fully background. A soft matte subsumes a hard background-removal mask, thresholded at 0.5, and keeps the anti-aliased edges that a binary mask discards. `array` returns the numpy view. On a matte result, `Results.cutout(image=None)` returns an RGBA `(H, W, 4)` uint8 array whose fourth channel is the matte, and `Results.save(path, image=None)` writes that cutout as a transparent-background PNG. Both take the RGB from `image` when given, otherwise they reload it from `Results.path`. ## OCRRegions Located text with transcripts. `data` is `(N, 4, 2)` float polygons in original-image pixels, ordered top-left, top-right, bottom-right, bottom-left, and regions come in reading order, top to bottom then left to right. `texts` is the list of N transcripts. `conf` is the per-region recognition score and `det_conf` the detection score, both `(N,)`. Detection quads are genuine polygons, so they do not populate `Results.boxes`. `xyxy` gives the axis-aligned hulls. ## Embeddings L2-normalized vectors from the `embed` task, always shape `(N, D)`. A whole-image result carries one row and no boxes; region embeddings are row-aligned with `boxes`. Because each row is normalized, cosine similarity is a dot product. | Member | Returns | |---|---| | `dim` | `D` | | `normalized` | The rows, renormalized | | `similarity(other)` | Pairwise cosine similarity against another `Embeddings` or tensor | | `verify(i, j, threshold=0.4)` | `True` when rows `i` and `j` match | ## Identities Named gallery matches, row-aligned with `embeddings`. Produced when a `Gallery` is passed to an `embed` prediction. `name` is a list where an entry is `None` below the match threshold, and the nearest below-threshold name is never guessed. `score` is the match score array and `data` pairs them. ## Meshes Parametric human body meshes, row-aligned with the person boxes in `boxes`. Everything is in the camera frame of the original image. `transl` is metric in meters with `+z` pointing away from the camera; `vertices` and `joints3d` are metric and already include `transl`; `joints2d` is in pixels on the original image canvas, not on the crop the network saw. No field carries a world or gravity frame. Parameter layouts differ between body models, so nothing about the shapes is hard-coded. `body_model` names the parameterization and the counts are read back from the tensors: `num_vertices`, `num_joints`, `num_betas`, and `has_vertices`. `params` returns the parameter dict, and `save_obj(path, index=0)` writes one mesh. Fields are `global_orient`, `body_pose`, `betas`, `transl`, `vertices`, `faces`, `joints3d`, `joints2d`, `conf`, `focal_length` and `extras`. For `body_model="mhr"` the rotations are Euler angles in radians rather than axis-angle, `body_pose` is a flat per-joint parameter vector rather than one triplet per joint, and `betas` are identity blendshape coefficients. Skeleton scale, hand pose and facial expression live in `extras`. ## Conversion and selection Every payload carries `to(*args, **kwargs)`, `cpu()`, `cuda()` and `numpy()`, and calling one of them on the `Results` applies it to every populated slot at once. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9t.pt") result = model(SAMPLE_IMAGE) # Every payload moves together. result = result.cpu().numpy() # Rows, as plain dicts, then as JSON. print(result.summary()[:1]) print(result.to_json()) ``` `result[idx]` selects rows across the row-aligned payloads. `len(result)` is the number of detections, or of points when there are no boxes. `result.update(...)` returns a copy with the named slots replaced; it accepts every slot plus `track_id` and `restore_scale`. ## summary and to_json `summary(normalize=False, decimals=5, embeddings=False)` returns a list of plain dicts, one row per detection, segment, point or region depending on which slots are set. `to_json(**kwargs)` passes its arguments to `summary` and returns the JSON string. `plot()` renders a dense normal or edge result in its canonical visualization; it raises for other result types. Annotated images for the other tasks come from `predict(save=True)`. --- # Promptable segmentation API LibreSAM is the factory for promptable segmentation. A forward pass needs a per-image prompt supplied at call time, so the tier owns its own predict surface rather than routing through the promptless inference runner. Verified against LibreYOLO v1.5.0. ## Install The tier needs the `sam` extra. **bash** ```bash pip install 'libreyolo[sam]' ``` ## The factory ```python LibreSAM(model: str = "base", **kwargs) -> LibreSAMModel ``` `model` is a size alias, not a path. `**kwargs` reaches the family constructor, which takes `device` and `multimask`. An unknown alias raises `ValueError` and the message lists every known alias. **Point and box prompts** ```python from libreyolo import LibreSAM, SAMPLE_IMAGE model = LibreSAM("base") r = model.predict(SAMPLE_IMAGE, points=[900, 370], labels=[1]) print(r.masks.xy) print(r.boxes.xyxy) r = model.predict(SAMPLE_IMAGE, bboxes=[100, 100, 200, 200]) print(len(r)) ``` **Encode once, prompt many times** ```python from libreyolo import LibreSAM, SAMPLE_IMAGE model = LibreSAM("base") model.set_image(SAMPLE_IMAGE) a = model.predict(points=[500, 375], labels=[1]) b = model.predict(bboxes=[100, 100, 200, 200]) print(len(a), len(b)) model.reset_image() ``` ## Aliases | Family | Aliases | Sizes | Weights | |---|---|---|---| | SAM-1 | `base`, `large`, `huge`, `b`, `l`, `h`, `sam-base`, `sam-large`, `sam-huge`, `sam_b`, `sam_l`, `sam_h` | `base`, `large`, `huge` | `facebook/sam-vit-base`, `-large`, `-huge` | | SAM-2 | `sam2-tiny`, `sam2-small`, `sam2-base-plus`, `sam2-baseplus`, `sam2-large`, and the short forms `sam2-t`, `sam2-s`, `sam2-bp`, `sam2-l`, `sam2_t`, `sam2_s`, `sam2_bp`, `sam2_l` | `tiny`, `small`, `base-plus`, `large` | `LibreYOLO/LibreSAM2tiny`, `-small`, `-base-plus`, `-large` | | EdgeTAM | `edgetam`, `edge-tam`, `edgetam-edge` | `edge` | `LibreYOLO/LibreEdgeTAM` | | SAM 3 | `sam3`, `sam-3`, `sam3-large` | `large` | `facebook/sam3` | | MobileSAM | `mobilesam`, `mobilesam-tiny`, `mobilesam_t`, `mobile-sam`, `mobile-sam-tiny` | `tiny` | `LibreYOLO/LibreMobileSAM` | | PicoSAM3 | `picosam3`, `picosam3-pico`, `picosam3_pico`, `pico-sam3` | `pico` | `LibreYOLO/LibrePicoSAM3` | The default is `base`. SAM-1, SAM-2, EdgeTAM and MobileSAM run at a nominal 1024 pixel canvas, SAM 3 at 1008, PicoSAM3 at 96. SAM 3 weights are gated. They download from `facebook/sam3` under Meta's custom SAM License, which is neither MIT nor Apache-2.0 and is not redistributed by LibreYOLO. Accept the terms on the repository page and authenticate with Hugging Face before loading; the loader logs the notice first. The family classes are exported too, so `LibreSAM1`, `LibreSAM2`, `LibreSAM3`, `LibreEdgeTAM`, `LibreMobileSAM` and `LibrePicoSAM3` can be constructed directly with `size=`. ## predict ```python model.predict( source=None, *, points=None, bboxes=None, labels=None, masks=None, text=None, conf=None, multimask=None, max_det=300, device=None, color_format="auto", points_per_side=None, ) -> Results ``` | Argument | Default | Meaning | |---|---|---| | `source` | `None` | Image to segment; `None` reuses the image cached by `set_image()` | | `points` | `None` | Point prompt in pixel coordinates | | `bboxes` | `None` | Box prompt as `[x1, y1, x2, y2]`, or a list of them for one mask per box | | `labels` | `None` | Point labels, `1` positive and `0` negative, shaped to match `points`; all positive when omitted | | `masks` | `None` | Reserved; passing one raises `NotImplementedError` | | `text` | `None` | Concept prompt; SAM 3 only | | `conf` | `None` | Predicted mask-IoU floor | | `multimask` | `None` | Return all ambiguity masks per prompt; defaults to the construction setting | | `max_det` | `300` | Cap on returned masks | | `device` | `None` | Move the model for this and later calls, invalidating cached embeddings | | `color_format` | `"auto"` | Color format hint for in-memory arrays | | `points_per_side` | `None` | Grid density for segment-everything; defaults to 32 | The return is an ordinary `Results` carrying `masks`, plus tight `boxes` derived from those masks, with class `0` named `"object"`. ## Prompt shapes `points` accepts the nested forms `[x, y]` for one object, `[[x, y], ...]` for N objects, and `[[[x, y], ...], ...]` for points grouped per object. Numpy arrays work everywhere a list does. Coordinates are plain pixels on the source image. Omitting every spatial prompt runs segment-everything, a grid automatic mask generator with a predicted-IoU threshold and box-IoU deduplication. The default `points_per_side` of 32 runs roughly 1024 decoder passes, which is slow on CPU; lower it for interactive use. The generator omits stability-score filtering, multi-crop and mask-IoU deduplication, so it is an approximation of the prompted path rather than a match for it. ## Confidence `conf` filters by predicted mask-IoU, which is a mask-quality score and not a detection confidence. `None` keeps every mask in the prompted path and applies the family grid threshold in segment-everything. `0.0` disables filtering in either mode. On SAM 3's text path, `conf` is the Promptable Concept Segmentation detection score instead. `None` there means the standard 0.3 threshold, and `0.0` keeps all candidates. ## Text prompts `text=` is SAM 3 only; every spatial-prompt family raises `NotImplementedError` for it. Text is mutually exclusive with points and boxes. The returned `names` maps class `0` to the requested concept. A text call with `source=None` re-encodes the cached image, because the tracker and the concept encoder do not share a cache. The keyword `exemplars=` is reserved for a future image-exemplar extension and is not implemented. ## The encode-once lifecycle ```python model.set_image(source, color_format="auto") -> LibreSAMModel model.reset_image() -> LibreSAMModel ``` `set_image` runs the heavy image encoder once and caches the embeddings, so every later `predict()` with `source=None` is cheap. Both methods return the model so calls can chain. Passing `device=` to `predict` moves the model and invalidates the cache. ## PicoSAM3 PicoSAM3 accepts `bboxes=` only. Point, text, mask, multimask and segment-everything prompts raise. The box is expanded by 10 percent and run through a 96 pixel ROI network, and PicoSAM3 is the one family in the tier that exports, to ONNX only. ## Not supported `train()`, `val()` and `track()` raise `NotImplementedError` on every family in the tier. Promptable masks have no fixed class set to score against, so mAP has no meaning here. `export()` raises for SAM-1, SAM-2, SAM 3, EdgeTAM and MobileSAM. Video and memory paths for SAM-2, SAM 3 and EdgeTAM are out of scope for this version, as are SAM 3 image exemplars and mask prompts. --- # Settings LibreYOLO has no configuration file. Behavior that is not a function argument is controlled by environment variables and by a small number of conventional directories, all listed here. Verified against LibreYOLO v1.5.0. ## Environment variables | Variable | Default | Effect | |---|---|---| | `LIBREYOLO_DATASETS_DIR` | `~/datasets` | Dataset root. Read once at import, into `libreyolo.data.DATASETS_DIR` | | `LIBREYOLO_FASTER_COCO_EVAL` | unset | Overrides the `faster_coco_eval` validation flag. `1`, `true`, `yes` or `on` forces the faster backend on, any other value forces it off, unset defers to the config flag | | `LIBREYOLO_KERNELS` | unset | Kernel selection. `off` or `reference` forces the reference implementations; any other value selects only implementations registered under that name | | `LIBREYOLO_QUANT_KERNELS` | unset | Legacy alias for `LIBREYOLO_KERNELS`, read only when that one is unset | | `LIBREYOLO_HUB_KERNELS` | unset | `0`, `false`, `off` or `no` disables Hugging Face Hub kernel loading. Any other value, including unset, leaves it enabled | | `LIBREYOLO_MHR_PATH` | `~/.cache/libreyolo/mhr/mhr_model.pt` | Location of the MHR body model used by the `mesh` task | | `LIBRELABEL_ENABLE_LOCATE` | unset | Must be exactly `1`, `true`, `yes` or `on` to expose the LocateAnything assistant in the labeling tool. Any other value keeps it off | | `SAM_3D_BODY_PATH` | unset | Path to the SAM 3D Body package for the mesh family, when it is not passed to the constructor | | `HF_TOKEN` | unset | Hugging Face access token, used for gated repositories | **Point the dataset root somewhere else** ```bash export LIBREYOLO_DATASETS_DIR=/data/datasets python -c "from libreyolo.data import DATASETS_DIR; print(DATASETS_DIR)" ``` **Read the resolved value from Python** ```python from libreyolo.data import DATASETS_DIR # Defaults to ~/datasets; LIBREYOLO_DATASETS_DIR overrides it at import time. print(DATASETS_DIR) ``` `LIBREYOLO_DATASETS_DIR` is read at import time, so setting it after importing `libreyolo.data` has no effect on `DATASETS_DIR`. Hub kernels are a two-part opt-in. The runtime fetch only happens when the optional `kernels` package is installed, so installing `libreyolo[hub-kernels]` is the opt-in and `LIBREYOLO_HUB_KERNELS=0` is the opt-out. An installation without the extra is unaffected either way. Kernel selection also short-circuits imports: when `LIBREYOLO_KERNELS` forces `off` or `reference`, the in-tree accelerated providers are never imported at all. The registry these three variables control is documented on [kernels](/docs/reference/kernels). ## Variables the library sets These are written rather than read, so setting them by hand is not the supported path. | Variable | Set by | |---|---| | `RANK`, `LOCAL_RANK`, `WORLD_SIZE`, `MASTER_ADDR`, `MASTER_PORT` | The DDP spawn helper, one value per worker process | | `CUDA_VISIBLE_DEVICES` | Temporarily narrowed during distributed setup, then restored | | `PYTORCH_ENABLE_MPS_FALLBACK` | Set to `1` by the EC trainers, with `setdefault`, so an existing value wins | | `MOMENTUM_ENABLED` | Set with `setdefault` by the mesh family loader | `LOCAL_RANK` doubles as the distributed-mode signal: its presence in the environment is how the training code detects that it is running under DDP. ## Logger variables The optional training loggers fall back to environment defaults for the project name. | Variable | Default | Used by | |---|---|---| | `WANDB_PROJECT` | `libreyolo` | The Weights and Biases logger, when no project is passed | | `COMET_PROJECT_NAME` | `libreyolo` | The Comet logger, when no project is passed | Authentication for those services follows their own tooling, not LibreYOLO. ## Tokens `HF_TOKEN` is the Hugging Face access token. When it is unset, the token is read from `~/.cache/huggingface/token`, which is where a Hugging Face CLI login writes it. Either path works. A token is needed only for gated repositories. SAM 3 is the shipped example: its weights download from a gated repository under a custom license, so the terms have to be accepted on the repository page and the session has to be authenticated. ## Directories | Path | Contents | |---|---| | `weights/` | Downloaded checkpoints, downloaded Hugging Face snapshots, and exported artifacts | | `~/datasets` | Dataset root, unless `LIBREYOLO_DATASETS_DIR` says otherwise | | `~/.cache/huggingface/token` | Hugging Face token, when not in `HF_TOKEN` | | `~/.cache/libreyolo/mhr/mhr_model.pt` | MHR body model, unless `LIBREYOLO_MHR_PATH` says otherwise | | `runs/track/` | Default output for `model.track(save=True)` | `weights/` is relative to the working directory. A bare filename resolves through it, so `LibreYOLO("LibreYOLO9t.pt")` looks for `weights/LibreYOLO9t.pt` and downloads there when it is absent. `model.export()` writes into the same directory when `output_path` is not given. The sibling tiers download multi-file snapshots into `weights//`. ## Download behavior Weight downloads are retried three times with backoff, resume from a partial file, and are guarded by a lock file so two processes do not fetch the same checkpoint at once. A family that fetches from a third-party host can pin a checksum and fail closed on a mismatch. Some downloads print a license notice before they start. Those notices are part of the download path and are not suppressible through configuration. ## Validation backend `model.val()` accepts `faster_coco_eval=True` by default and falls back to pycocotools when the package is not installed, warning once. Setting `LIBREYOLO_FASTER_COCO_EVAL` overrides the per-call flag, which is what a benchmark harness that cannot touch per-run configs should use. The backend that actually ran is reported on `model.last_eval_backend`. ## Dataset download scripts A dataset YAML may carry a `download` field containing Python. It is not executed unless `allow_download_scripts=True` is passed to the call that reads it, which is a function argument on `val()` and `export()` rather than an environment variable. --- # Stability tiers LibreYOLO uses the word tier for three separate things: the evidence behind an export path, the call contract a model family answers to, and the coverage group a family is enrolled in. This page defines each one and says what it does not imply. Verified against LibreYOLO v1.5.0. ## Export support tiers The tier that decides whether a call succeeds. It applies to the triple `(family, task, format)`, and every combination has exactly one. | Tier | Meaning | What happens on `export()` | |---|---|---| | `validated` | Numeric parity is covered in CI or a documented nightly run | Runs | | `available` | Conversion is implemented, but numeric runtime parity evidence has not been recorded | Runs | | `blocked` | No supported path | Raises `NotImplementedError` in preflight, with the reason | Validated and available both proceed without an acknowledgement or a blanket warning. The difference is evidence, not permission: a validated entry has a parity test behind it and a `since` release, and an available entry does not yet. A CoreML conversion without a macOS prediction run, for example, is available and not validated. A blocked combination fails before dependency checks, calibration loading, tracing or artifact creation, so nothing partial is written. Every validated cell carries a constraint describing the configuration the parity number came from, typically a fixed input canvas, batch 1, FP32 and a named runtime version. Read it as a claim about that configuration rather than about the format in general. The rules that fill cells with no explicit entry are on the [export matrix](/docs/reference/export-matrix) page. **Read both classifications for one family** ```python from libreyolo.models.registry import GROUPS, group_of from libreyolo.export.support import get_support, validated_alternatives family = "yolo9" group = group_of(family) print(group, GROUPS[group]) print(get_support(family, "detect", "onnx").tier) print(validated_alternatives(family, "detect")) ``` ## API tiers The tier that decides what a call looks like. A family sits in exactly one, chosen by call contract rather than by architecture. | Tier | Factory | Contract | |---|---|---| | Detector factory | `LibreYOLO` | One promptless forward returns every object it found, with calibrated scores. Members register themselves by recognizing a checkpoint | | Promptable segmentation | `LibreSAM` | A forward is meaningless without a per-image spatial or concept prompt supplied at call time. Interactive and stateful: encode once, prompt many times | | Open-vocabulary detection | `LibreOpenVocab` | Text-conditioned discriminative detectors. The class list is a prompt, set by `set_classes` | | Vision-language | `LibreVLM` | A generative model driven as a detector. The class list is a prompt and the confidence is a placeholder | The three sibling tiers deliberately do not register into the detector factory, which is why `LibreYOLO("some-alias")` does not reach them. They load by size alias and autodownload rather than by checkpoint sniffing. All four return the same `Results`, so downstream code is unchanged across them. What differs is which methods work: the sibling tiers raise `NotImplementedError` for `train()`, `val()` and `export()`, and the SAM and open-vocabulary tiers raise for `track()` as well. Each tier page lists its own exclusions. ## Coverage groups The classification that decides which families a cross-family test run includes, and the one a reader is most likely to meet on a model page. Every registered family is enrolled in exactly one group, and a test fails when a registered family is missing from the enrollment. `GROUPS` in `libreyolo/models/registry.py` is the source of the Meaning column below; `MODEL_GROUPS` in the same file assigns every family, and the Families column counts that assignment directly. The Label column is the shorter name the site uses for the same group on a model page header. | Group | Label | Families | Meaning | |---|---|---|---| | `g0` | Flagship | 2 | Flagship anchors required in shared-feature coverage | | `g1` | Core | 10 | Trainable detector coverage set | | `g2` | Supported | 14 | Additional trainable-family coverage set | | `g3` | Inference only | 35 | Families without a training implementation | | `g4` | Museum | 5 | Historical families with inference coverage | | `s` | Sibling tier | 21 | Sibling APIs (SAM, open-vocab, VLM, zero-shot) covered separately | That is 87 families across six groups. `g3` alone holds more families than every other group combined, because most of the registry is inference-only lineage and museum coverage rather than actively trained detectors. For a reader choosing a model, the group says where to expect engineering attention, not how accurate a family is. `g0` and `g1` are where a new feature is designed and land first; `g2` is kept green in CI but a feature lands there opportunistically rather than on the same release wave. `g3` states an absence rather than a limit: predict, validate and, where the family supports it, export all still work, and `train()` on a `g3` or `g4` family raises `NotImplementedError` naming the reason rather than doing something silently partial. `s` families do not sit in this trade-off at all, because they load through their own factory rather than `LibreYOLO()`. See [core concepts](/docs/concepts) for how a group fits alongside task, family and size when reading a checkpoint filename. A group does not grant or restrict a user-facing capability by itself. Support comes from the family's implemented API and from format-specific capability checks, never from group membership alone. Groups classify families, not tasks, so a task-scoped coverage run names the task explicitly, as in "g1 detect". Two places read the group at runtime rather than only in tests. `collect_model_inventory()` in `libreyolo/models/inventory.py` attaches the group to every entry the CLI inventory prints, and `pretrained=False` triggers the special from-scratch reinitialization path only for families in `g0` and `g1`. Outside those two groups the check in `libreyolo/models/base/model.py` is skipped entirely, so `pretrained=False` reaches the family's own `train()` as an ordinary keyword instead. ## Training A family in `g3` or `g4` has no training implementation, and calling `train()` on one raises. That is a property of the family's code, not of its group: the group records the fact rather than causing it. For a family that does train, whether an individual augmentation knob reaches the pipeline is a separate question with its own three-value vocabulary, `used`, `gated_by_mosaic` and `ignored`. See the [augmentation matrix](/docs/reference/augmentation-matrix). ## What a tier does not tell you A tier is not an accuracy claim. A validated export says the artifact reproduces the native model within a stated threshold; it says nothing about how well the native model scores on a dataset. Benchmark numbers live on the model pages. A tier is also not a licensing statement. Weight licenses vary within a family and the repository hosting a specific checkpoint is authoritative. A family being in the detector factory says nothing about whether its published weights permit commercial use. --- # Upstream checkpoints LibreYOLO families are ported from upstream projects whose released checkpoints are almost loadable but carry no LibreYOLO metadata. Auto-conversion recognizes those files, wraps them in schema v1.0, and writes the result beside the source. Verified against LibreYOLO v1.5.0. ## What happens on load When `LibreYOLO()` meets a `.pt` file that is not already a complete v1.0 checkpoint, it calls the auto-converter, which: 1. unwraps the tensor dict from the common upstream layouts; 2. asks every registered family whether it recognizes the layout, remapping keys where the upstream naming differs from the native port; 3. wraps the winner in a strict v1.0 metadata checkpoint, reading size, task and class count from the tensors themselves so fine-tuned checkpoints convert correctly; 4. writes it beside the source as `-[-task].pt` and returns that path, so the factory loads it normally. Nothing is asked of the caller. A file that no family claims returns nothing and the factory reports that it could not load it. **Just pass the file to the factory** ```python from libreyolo import LibreYOLO # A recognized upstream file is converted on load, and the converted # checkpoint is written next to it. # model = LibreYOLO("yolov9-t-converted.pt") # Any LibreYOLO checkpoint loads unchanged. model = LibreYOLO("LibreYOLO9t.pt") print(model.family, model.size, model.task, model.nb_classes) ``` ## Layouts it unwraps The tensor dict is looked for in this preference order, EMA first, and each candidate is tried until one actually holds tensors. An empty or metadata-only EMA block therefore does not mask valid weights underneath. | Key | Note | |---|---| | `ema.module` | The common EMA wrapper | | `ema` | Legacy flat EMA wrappers that store tensors directly | | `ema_state_dict` | Entries under a `module.` prefix are stripped | | `params_ema` | | | `params` | | | `ema_net` | | | `net` | | | `model` | | | `state_dict` | | | The file itself | A plain state dict | Each candidate is then narrowed to its tensor-valued entries and normalized: a leading `module.` or `_orig_mod.` prefix is stripped, and a dict whose keys all start with `model.model.` has that prefix removed. ## Which families recognize what Recognition is a per-family classmethod. The default implementation claims a layout whose keys already match the native port. A family whose upstream key naming differs overrides it with a remap, and returns nothing for layouts it does not recognize. Families that ship a remapping recognizer: `centernet`, `deeplabv3`, `deformable_detr`, `dexined`, `moge2`, `picodet`, `rtdetr`, `rtdetrv2`, `rtdetrv4`, `rtmdet`, `segformer`, `swin`, `teed`, `yolo7`, `yolo9`, `yolo9_e2e`, `yolo9_p2`. Families that decline auto-conversion outright: `efficientdet`, `eomt` and `pidnet` return nothing from the recognizer, so their upstream files go through a conversion script instead. `l2cs` is excluded from the generic recognizer because it is inference-only with redistribution-restricted weights. RF-DETR keeps its own recognizer, because it needs the whole checkpoint rather than just the tensor dict to detect the size and to remap COCO classes. It is registered only when its optional dependencies are installed. Every other registered family uses the default: it claims the file when its own loader already recognizes those keys. ## Which family wins Several families can claim the same file, so the resolution mirrors the factory's dispatch rules. A subclass claim beats its base class. Registration order follows class creation, so a derived family registers after the base it refines, and its positive markers must not lose to the base's broader passthrough. Registry order then decides, because it encodes specificity: the earliest claim is the most specific match. The one tie registry order cannot break is DEIM against D-FINE, whose architecture keys are identical. There, and only there, the filename is the deciding signal, and a file whose name gives no hint is refused rather than guessed. The filename is deliberately not consulted anywhere else, so a broad false-positive claim can never be promoted over a more specific one purely by what the file is called. ## Safe loading Upstream files are loaded through the weights-only unpickler. Some upstream training checkpoints embed library objects that unpickler rejects. Those objects are training metadata rather than weights, so each blocked global is retried with an inert stand-in class that satisfies the unpickler without executing anything. The captured name is used only as a string label, never imported, evaluated or called. Sensitive module names are refused outright and never stubbed: `builtins`, `os`, `sys`, `posix`, `nt` and `subprocess`. The retry loop is bounded at 32 attempts, so a file engineered to introduce an unbounded series of distinct globals fails closed instead of spinning. Only tensors survive into the converted checkpoint. ## Where the converted file goes The output is written beside the source, named `-[-task].pt`. It is always rewritten rather than reused, which keeps repeated loads of the same source fresh while avoiding collisions with official weights or with another fine-tune of the same family, size and task in the same directory. When the source directory is read-only, the conversion falls back to a fresh private temporary directory created per call, and the log line names the path it used. Only if that also fails is the conversion dropped, with a warning. ## Existing LibreYOLO checkpoints A file carrying a LibreYOLO-specific marker, `libreyolo_version` or `model_family`, belongs to the normal load path and is not re-converted. The skip applies only to a passthrough claim, meaning one where the keyset was unchanged. A claim whose conversion changed the keyset is proof of a foreign upstream layout and is accepted even on a marked file. `schema_version` is deliberately not treated as a marker, because other training and export tools use that generic name, and neither are `names`, `nc`, `size`, `task` or `imgsz`, because an upstream fine-tune may carry them too. A foreign fine-tune that merely carries a generic `names` key is therefore not marked, so its native-keyed claim converts normally and derives the class count from the tensor head rather than being mis-loaded as 80 class. ## Metadata the converter reads Class names are taken from a top-level `names` key, or from `class_names` inside an `args` or `hyper_parameters` block. A names map keyed by labels rather than by class index is unusable and is replaced by generated defaults. A names list longer than the detected class count is trimmed, because out-of-range indices would fail the strict validator and silently abort the conversion. Upstream `args` are carried over as plain metadata, with any value that is not a string, number, boolean, list or dict dropped, so nothing unsafe reaches the saved file. ## RF-DETR COCO normalization Upstream RF-DETR checkpoints expose a 91-output classification head, which is COCO's 90 classes plus background. Auto-conversion normalizes a COCO RF-DETR to the COCO-80 convention, with the remap applied at postprocessing. A checkpoint is treated as COCO when it carries exactly 80 names, or declares a class count of 80, or has a `coco` dataset hint, or has no class or dataset metadata at all. That last case matters: a bare upstream state dict is the canonical COCO-pretrained checkpoint, and it is the only metadata-less 91-output RF-DETR in distribution. A genuine custom 90-class RF-DETR is preserved as 90 classes. It is identified by a names list, an explicit non-80 class count, or a non-COCO dataset hint, so the bare-checkpoint fallback does not fire for it. Empty placeholders are ignored when deciding whether a dataset hint is present. ## Limits Auto-conversion recognizes released upstream layouts. It does not rewrite an architecture, and it does not make an unported model loadable. When no family claims a file, the answer is a conversion script rather than a factory argument: the repository ships `weights/convert_*.py` for the families that need one, including EoMT, PIDNet and EfficientDet. Conversion also does not invent metadata it cannot read. Size, task and class count come from the tensors; names come from the file when present, and are generated as `class_i` when not. --- # Vision-language API LibreVLM loads a generative vision-language model and drives it as an object detector. The class list is a prompt rather than a fixed head, and the model returns the same Results any other family returns. Verified against LibreYOLO v1.5.0. ## Install The tier needs the `vlm` extra. **bash** ```bash pip install 'libreyolo[vlm]' ``` ## The factory ```python LibreVLM(model: str = "qwen3-vl-4b", **kwargs) -> LibreVLMModel ``` `model` is an alias, not a path. `**kwargs` reaches the family constructor, which takes `device`, `names` (the initial vocabulary, equivalent to calling `set_classes` after load), `prompt` (override the detection prompt) and `max_new_tokens`. An unknown alias raises `ValueError` listing every alias. **Detect an open vocabulary** ```python from libreyolo import LibreVLM, SAMPLE_IMAGE model = LibreVLM("lfm2-vl-450m") model.set_classes(["person", "skateboard"]) result = model.predict(SAMPLE_IMAGE) for box, cls in zip(result.boxes.xyxy, result.boxes.cls): print(result.names[int(cls)], box.tolist()) ``` **Ask a free-form question** ```python from libreyolo import LibreVLM, SAMPLE_IMAGE model = LibreVLM("lfm2-vl-450m") print(model.chat(SAMPLE_IMAGE, "How many people are in this image?")) ``` ## Aliases | Family | Aliases | Sizes | Weights | |---|---|---|---| | Qwen3-VL | `qwen3-vl`, `qwen3-vl-2b`, `qwen3-vl-4b`, `qwen3-vl-8b` | `2b`, `4b`, `8b` | `Qwen/Qwen3-VL-2B-Instruct`, `-4B-`, `-8B-` | | LFM2-VL | `lfm2-vl`, `lfm2-vl-450m`, `lfm2-vl-1.6b` | `450m`, `1.6b` | `LiquidAI/LFM2.5-VL-450M`, `-1.6B` | | InternVL3 | `internvl3`, `internvl3-1b`, `internvl3-2b`, `internvl3-8b` | `1b`, `2b`, `8b` | `OpenGVLab/InternVL3-1B-hf`, `-2B-hf`, `-8B-hf` | | SmolVLM2 | `smolvlm2`, `smolvlm2-2.2b`, `smolvlm2-500m` | `2.2b`, `500m` | `HuggingFaceTB/SmolVLM2-2.2B-Instruct`, `SmolVLM2-500M-Video-Instruct` | | Florence-2 | `florence-2`, `florence2`, `florence-2-base`, `florence-2-large` | `base`, `large` | `florence-community/Florence-2-base`, `-large` | | Kosmos-2 | `kosmos-2`, `kosmos2` | `224` | `microsoft/kosmos-2-patch14-224` | | LocateAnything | `locate-anything`, `locateanything`, `locate-anything-3b`, `locateanything-3b` | `3b` | `nvidia/LocateAnything-3B` | | SenseNova-Vision | `sensenova-vision`, `sensenova-vision-7b`, `sensenovavision` | `7b` | `LibreYOLO/SenseNovaVision7b` | | LibreMODUS | `libremodus`, `libremodus-14b-a7b`, `modus`, `modus-14b-a7b` | `14b-a7b` | Pinned upstream snapshot | The default alias is `qwen3-vl-4b`. Sizes for the default alias of each family are the ones listed first: `qwen3-vl` resolves to `4b`, `lfm2-vl` to `450m`, `internvl3` to `2b`, `smolvlm2` to `2.2b`, `florence-2` to `base`. `LibreVLM`, `LibreLFM2VL`, `LibreQwen3VL`, `LibreSmolVLM2`, `LibreInternVL3`, `LibreFlorence2`, `LibreKosmos2`, `LibreLocateAnything` and `LibreMODUS` (also spelled `LibreModus`) are exported at package level. ## Tasks Most families serve `detect` only. Two serve more: | Family | Supported tasks | |---|---| | LocateAnything | `detect`, `point` | | SenseNova-Vision | `detect`, `segment`, `panoptic`, `pose`, `point`, `depth`, `ocr` | Because the task is prompt-driven rather than baked into a checkpoint, it can be switched on a loaded model: ```python model.set_task(task: str) -> LibreVLMModel ``` The task is validated against the family's supported list, is sticky across later `predict()` and `track()` calls, and the model is returned so calls can chain. ## set_classes ```python model.set_classes(classes: list[str]) -> LibreVLMModel ``` Sets the open vocabulary. Any words work, because the model is prompted with them rather than constrained to a fixed head. The list must be non-empty and its entries must be unique when compared case-insensitively. Passing a bare string raises `TypeError`, because it would enumerate into one-character classes. The vocabulary is sticky: set it once after loading and it persists until set again. ## chat ```python model.chat(image, prompt, max_new_tokens=None, color_format="auto") -> str ``` Raw multimodal generation: image and prompt in, decoded text out, verbatim. This is the escape hatch under the detection convenience, for free-form questions, counting, or an output format the detection wrapper does not cover. `max_new_tokens` falls back to the family's `MAX_NEW_TOKENS`, which is 1024 on the base class. Decoding is greedy with a mild repetition penalty. ## Confidence Generated output has no calibrated per-box confidence. This version assigns a constant placeholder so `predict`, drawing and `track` behave, which makes `conf=` filtering and mAP soft rather than meaningful. This is also why `val()` raises: COCO mAP over placeholder scores would mislead. ## Predict and track The standard predict surface applies, and `track()` works, so a VLM detector drops into the same pipeline as any other family. Two class-level policies differ from a convolutional detector: test-time augmentation is disabled, because multi-scale augmentation is meaningless for a fixed-resolution generator, and batched predict is off, because generation is autoregressive and preprocessing returns a text-and-image encoding rather than a stackable image tensor. ## Not supported `train()`, `val()` and `export()` raise `NotImplementedError`. Fine-tune upstream and load the resulting weights. ## Remote code Every shipped family loads through a native model class, so LibreYOLO does not execute third-party repository code by default. A family that genuinely needs it must opt in explicitly and pin a snapshot revision; LocateAnything is the one that does, pinned to commit `c32291ca5e996f5a7a485845b4f57a233936bba0`. LibreMODUS is an explicit exception to the checkpoint schema: its alias resolves to a directory of pinned upstream files rather than a LibreYOLO `.pt`, and LibreYOLO neither adds v1.0 metadata to it nor republishes it. --- # Background removal Background removal separates a subject from everything behind it. LibreYOLO exposes it as the matte task, which returns a soft alpha value per pixel rather than a hard foreground mask. Verified against LibreYOLO v1.5.0. ## Definition The `matte` task predicts one alpha value per pixel from a single RGB image: `1` is fully foreground and `0` is fully background. The value is continuous rather than binary, which is the point of the task. A hard mask is one threshold away, at 0.5, while the soft matte additionally carries the partial coverage at hair, fur and motion-blurred edges that a binary mask throws away. A prediction fills `result.matte`, a `Matte` payload holding an `(H, W)` float32 array in `[0, 1]` on the original image canvas, reachable as NumPy through `.array`. `result.cutout()` composites the source image with that alpha into an `(H, W, 4)` uint8 RGBA array, and `result.save(path)` writes the same thing to a transparent-background PNG. `result.boxes` stays empty, so `conf`, `iou` and `max_det` have no effect. ## Models Two families serve `matte`, and they share a forward path. [BiRefNet](/docs/models/birefnet) is the bilateral-reference network the task is built around, published here as one Swin-L tier checkpoint. [FeyNobg](/docs/models/feynobg) is Feyn Inc.'s deepened variant: BiRefNet's architecture with the third Swin stage grown from 18 to 24 blocks, then retrained. LibreYOLO reuses BiRefNet's forward path, preprocessing and single-logit output for it, so predict, validate and checkpoint handling behave identically; the weights and the family identity are FeyNobg's own. The two carry different weight licenses. Both are stated on the model pages, and the license on the Hugging Face repository of the specific checkpoint is the authoritative one. ## Predict Weights download from Hugging Face on first use and are cached locally. **Predict a matte** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreBiRefNetl-matte.pt") result = model(SAMPLE_IMAGE) matte = result.matte print(matte.array.shape, matte.array.dtype) # (H, W) float32 in [0, 1] ``` **Write a transparent PNG** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreBiRefNetl-matte.pt") result = model(SAMPLE_IMAGE) # save() composites the source with the matte as an alpha channel. result.save("subject.png") rgba = result.cutout() # the same (H, W, 4) uint8 array in memory print(rgba.shape) ``` **Composite onto a new background** ```python import numpy as np from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreBiRefNetl-matte.pt") result = model(SAMPLE_IMAGE) rgba = result.cutout() alpha = rgba[..., 3:4].astype(np.float32) / 255.0 backdrop = np.full_like(rgba[..., :3], 255) # white composited = (rgba[..., :3] * alpha + backdrop * (1 - alpha)).astype(np.uint8) print(composited.shape) ``` Both families run at a fixed native 1024x1024 canvas and resize the matte back to the original image. A different resolution is not supported, because the Swin backbone's relative-position tables are tied to that size, and a mismatch interpolates them badly rather than raising. `Results.save()` is defined for matte results only and needs the source image, which it reloads from `Results.path` unless you pass one. See [prediction](/docs/predict) for sources, streaming and result handling. ## Dataset format Matte validation pairs each RGB image with a single-channel ground-truth alpha matte sharing the same stem, where 0 is background and 255 is foreground. ```text my-matte-dataset/ images/ subject.jpg mattes/ subject.png ``` Passing that root as `data=` is enough: the matte directory is auto-detected among `mattes/`, `matte/`, `gt/`, `masks/`, `mask/` and `alpha/`. A dataset YAML is the alternative, with `path` plus `val_images` and `val_mattes` naming directories relative to it: ```yaml path: my-matte-dataset val_images: images val_mattes: mattes nc: 1 names: {0: matte} ``` `nc` and `names` are schema placeholders; a matte model returns `Results.matte`, not detections. Matte values are read as alpha in `[0, 1]` by dividing by 255, and a matte whose shape differs from the prediction canvas is resized bilinearly to match. See [dataset formats](/docs/reference/dataset-formats) for the full contract. ## Train Neither matte family has a training implementation: `train()` raises `NotImplementedError` on both, and matte support covers prediction, validation and export only. Each model page names the upstream project that ships training code and the conversion script that brings a checkpoint back. ## Validate `val()` drives the model's own `predict`, so validation uses the family's exact preprocessing, and both metrics are computed on the original image canvas. **Validate and read the metric keys** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreBiRefNetl-matte.pt") # A directory holding images/ and a matte directory works in place of # a dataset YAML. metrics = model.val(data="my-matte-dataset/") print(metrics["metrics/MAE"]) # lower is better print(metrics["metrics/Smeasure"]) # fitness, higher is better ``` `metrics/MAE` is the mean absolute error against the ground-truth alpha, in `[0, 1]`, and lower is better. `metrics/Smeasure` is the S-measure of Fan et al. (ICCV 2017), a structural similarity that credits getting the subject's shape and its holes right, which a per-pixel average alone misses; higher is better. S-measure is also `fitness`, the number best-checkpoint selection reads. Neither metric depends on resolution. ## Export An exported matte model loads back through `LibreYOLO()` on its file suffix, so the artifact behaves like a checkpoint and returns the same `Results`. **Export** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreBiRefNetl-matte.pt") model.export(format="torchscript") ``` **Run the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreBiRefNetl-matte.torchscript") result = model(SAMPLE_IMAGE) print(result.matte.array.shape) ``` TorchScript is the validated path for this task. ONNX conversion runs but has not cleared the same parity bar, and the remaining formats are not available. Per-format coverage is on the [BiRefNet](/docs/models/birefnet) and [FeyNobg](/docs/models/feynobg) pages and in the [full export matrix](/docs/reference/export-matrix). --- # Body mesh Body mesh recovery turns a single image and a set of person boxes into a parametric 3D body per person: shape and pose parameters, posed vertices, 3D joints, and the camera translation that places them in front of the lens. Verified against LibreYOLO v1.5.0. ## Definition Body mesh recovery returns a `Meshes` payload per image, row-aligned with `result.boxes`: row `i` describes the person in box `i`, the same contract the pose task uses for keypoints. Everything is expressed in the camera frame of the original image. `transl` is metric, in meters, with +z pointing away from the camera. `vertices` and `joints3d` are metric and already include `transl`, so they need no further composition. `joints2d` is in pixels on the original image canvas, not on the crop the network saw. `faces` holds the mesh topology once for the whole image rather than per row, because every person shares it. There is no world or gravity frame in this version, and no field silently stands in for one. Parameter layouts differ between body models, so nothing about the shapes is fixed: `body_model` names the parameterization and the counts are read back from the tensors. For `"mhr"`, the Momentum Human Rig, rotations are Euler angles in radians rather than axis-angle, `body_pose` is a flat per-joint parameter vector rather than one triplet per joint, and `betas` are identity blendshape coefficients. Skeleton scale, hand pose and facial expression live in `extras`. The canonical task key is `mesh`. `body-mesh`, `hmr` and `human-mesh-recovery` normalize to it. ## Models [SAM 3D Body](/docs/models/sam-3d-body) is the only family serving this task, and it is a wrapper rather than a port: Meta's `sam-3d-body` package is published under the SAM License, which LibreYOLO's own code may not derive from, so none of it is vendored. Two backbones share the same MHR body model, `d3` on a DINOv3 ViT-H/16+ encoder and `h` on the original ViT-H. Three requirements apply before a first prediction, and none of them is optional. The upstream package is installed by you, not by LibreYOLO: ```bash git clone https://github.com/facebookresearch/sam-3d-body pip install roma einops yacs omegaconf braceexpand pytorch-lightning timm ``` Point the library at the clone with `sam_3d_body_path=` or the `SAM_3D_BODY_PATH` environment variable. A user who never constructs this family never triggers the import. The checkpoint mirror is gated. Accept the license on the Hugging Face model page and authenticate with `hf auth login`, or the first download fails. The MHR body model itself is a separate Apache-2.0 release, fetched from its own public location and cached locally. Inference needs a CUDA device. The upstream estimator moves its batch to the GPU without checking, so there is no CPU path to fall back to and `device="cpu"` raises. ## Predict **Python** ```python from libreyolo import SAMPLE_IMAGE from libreyolo.models.sam3dbody import LibreSAM3DBody # This family is not registered with the LibreYOLO() factory, so it # is constructed directly. model_path=None triggers the gated # Hugging Face download; a string is treated as an existing local # checkpoint and is never fetched. Inference requires CUDA. model = LibreSAM3DBody(None, size="d3", device="cuda") result = model(SAMPLE_IMAGE, person_boxes=[[34, 12, 220, 400]]) meshes = result.meshes print(meshes.body_model) # the parameterization these tensors use print(meshes.vertices.shape) # (N, V, 3), camera frame, meters print(meshes.joints3d.shape) # (N, J, 3) print(meshes.joints2d.shape) # (N, J, 2), pixels on the source image ``` **With a person detector** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE from libreyolo.models.sam3dbody import LibreSAM3DBody # person_detector accepts a constructed LibreYOLO detector, a plain # callable, or a PersonDetector instance. There is no name shortcut. detector = LibreYOLO("LibreYOLO9s.pt") model = LibreSAM3DBody(None, size="d3", device="cuda") result = model(SAMPLE_IMAGE, person_detector=detector) ``` People reach the model in one of two ways. `person_boxes` passes boxes you already hold, for a single image only: a fixed set of boxes cannot follow people across video frames, so passing it with a video source raises instead of silently reusing frame one's boxes. `person_detector` accepts a constructed LibreYOLO detector, a callable, or a `PersonDetector`, and is the path for video. `focal_length` supplies a known camera intrinsic; left unset, the model uses its own estimate, which is what `meshes.focal_length` reports. This family is not wired into the `LibreYOLO()` factory or the `libreyolo predict` CLI command. `LibreSAM3DBody` is the only entry point. See [prediction](/docs/predict) for sources, streaming and result handling. ## Train No family in this task trains inside LibreYOLO. `LibreSAM3DBody.train()` raises: train at the upstream project and load the resulting checkpoint here. ## Validate There is no mesh validator, and `val()` raises. The usual benchmarks are research-license only, so none is bundled and none can be fetched for you. The metrics themselves are available as `libreyolo.validation.mesh_metrics`, for evaluating against a dataset you already hold. It takes predicted and target joints, optionally predicted and target vertices, and returns a dictionary keyed exactly like a validator's: `metrics/mpjpe` is mean per-joint position error after aligning the root joint, so it scores pose while ignoring where the person stands in the scene. `metrics/pa_mpjpe` is the same quantity after a full Procrustes alignment, rotation, uniform scale and translation, which removes global orientation and body-size error and leaves the articulated pose. `metrics/pve` is mean per-vertex error over the mesh surface after aligning on the vertex centroid; unlike the joint metrics it is sensitive to body shape, and it appears only when both vertex arrays are supplied. All three are lower-is-better. Inputs are assumed metric, in meters, and `scale_to_mm` converts the results to the millimeters the literature reports. ## Export Mesh export is not implemented. LibreYOLO has not defined an exported-graph metadata contract for this task, including how to carry the MHR parameter layout outside PyTorch, so `export()` raises rather than emitting a graph whose output could not be interpreted. --- # Depth estimation Depth estimation predicts how far each pixel is from the camera using a single image. LibreYOLO exposes it as the depth task, which returns a dense relative inverse-depth map on the original image canvas. Verified against LibreYOLO v1.5.0. ## Definition The `depth` task predicts one value per pixel from a single RGB image. LibreYOLO defines that value as relative inverse depth: higher means closer to the camera, and the numbers carry no metric unit and no scale that holds across two images. Comparing depth between two pixels of the same prediction is meaningful; comparing a value to a value from another image is not. A prediction fills `result.depth_map`, a `DepthMap` payload holding an `(H, W)` array on the original image canvas. `.min`, `.max` and `.mean` read the finite values, and `.normalized()` rescales the map to `[0, 1]` for display. `result.boxes` stays empty, so `conf`, `iou` and `max_det` have no effect, and `save=True` writes a colormapped image of the map rather than an annotated photo. ## Models Six families serve `depth`. [Depth Anything V2](/docs/models/depth-anything-v2) pairs a DINOv2 encoder with a DPT decoder and is the general-purpose default here. Licensing decides the size as much as accuracy does: the Small checkpoint is Apache-2.0 while Base and Large are non-commercial, so check the checkpoint table on its page before picking one. [Depth Anything 3](/docs/models/depth-anything-3) ports the DA3MONO-LARGE checkpoint, a plain transformer with no architectural specialization for depth. [ZipDepth](/docs/models/zipdepth) is the compact tier: a reparameterizable CNN distilled from Depth Anything V2 Large, with a second checkpoint whose decoder avoids gather and unfold operations for NPU compilers that lack them. [MiDaS](/docs/models/midas) is the line of work that established the zero-shot relative-depth protocol the other families are measured with. It is the one depth family LibreYOLO does not republish: requesting a checkpoint downloads the official asset from its authors' GitHub release and checks a pinned SHA-256. [LibreMODUS](/docs/models/libremodus) reaches depth as one target of an any-to-any model rather than as a dedicated head. It needs the `modus` extra and your own authenticated Hugging Face account, and it offers neither `val()` nor `export()`. [SenseNova-Vision](/docs/models/sensenova-vision) generates the depth map as an image through a diffusion decode, from the same 7B checkpoint that serves its six other tasks. It needs the `sensenova` extra, and its weights are restricted to non-commercial use; the license is on its page. ## Predict Weights download from Hugging Face on first use and are cached locally, except for the two families noted above. **Predict a depth map** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreDepthAnythingV2s-depth.pt") result = model(SAMPLE_IMAGE, save=True) depth = result.depth_map print(depth.data.shape) # (H, W) on the original canvas print(depth.min, depth.max, depth.mean) ``` **Work with the values** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreDepthAnythingV2s-depth.pt") result = model(SAMPLE_IMAGE) depth = result.depth_map raw = depth.data # higher is closer; no metric unit, no scale gray = depth.normalized() # rescaled to [0, 1] for visualization print(raw.shape, float(gray.max())) ``` **A compact alternative** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # Same task contract, a much smaller network built for edge runtimes. model = LibreYOLO("LibreZipDepthb-depth.pt") result = model(SAMPLE_IMAGE) print(result.depth_map.data.shape) ``` Input resolution is constrained per family. Depth Anything V2 and Depth Anything 3 build on a DINOv2 patch grid, so `imgsz` must divide evenly by 14, which LibreYOLO checks before running. `Results.plot()` does not cover this task; it is defined for surface normals and edges only. See [prediction](/docs/predict) for sources, streaming and result handling. ## Dataset format Depth validation pairs each image with a dense single-channel depth map that has the same resolution, found by substituting the depth directory into the image path. ```text dataset/ data.yaml images/ val/room.jpg depths/ val/room.png ``` ```yaml path: dataset val: images/val depths_dir: depths nc: 1 names: {0: depth} ``` Maps are single-channel PNG or TIF, or `.npy`. Values are plain depth in a unit the dataset keeps consistent, and `0`, negative, NaN and infinite pixels mark invalid samples that are excluded from the metrics. Integer maps are divided by `depth_scale`, which defaults to `256.0`, the 16-bit PNG convention; float `.npy` maps are used as they are. `depth_stem_suffix` and `depth_mask_suffix` cover datasets that name their depth files or validity masks differently. See [dataset formats](/docs/reference/dataset-formats) for the full contract. ## Train No depth family in LibreYOLO has a training implementation: `train()` raises `NotImplementedError` on all six. Each model page names the conversion script that turns a checkpoint trained upstream into one LibreYOLO can load. ## Validate `val()` runs the shared depth validator. Relative depth has no absolute scale, so each prediction is first fitted to the inverse of its ground truth with a per-image least-squares scale and shift, then inverted back to depth. Every metric below is computed per image on that aligned map and averaged over the dataset, counting only pixels the dataset marks valid. **Validate and read the metric keys** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDepthAnythingV2s-depth.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/abs_rel"]) print(metrics["metrics/rmse"]) print(metrics["metrics/delta1"]) # fitness print(metrics["metrics/delta2"], metrics["metrics/delta3"]) ``` `metrics/abs_rel` is the mean absolute relative error, the residual divided by the ground-truth depth, and lower is better. `metrics/rmse` is the root mean squared error in the dataset's own depth unit, also lower is better. `metrics/delta1`, `metrics/delta2` and `metrics/delta3` are the threshold accuracies: the fraction of valid pixels whose ratio to ground truth, taken in whichever direction is larger, falls under 1.25, 1.25 squared and 1.25 cubed, so higher is better. `metrics/delta1` is also `fitness`, the number best-checkpoint selection reads. ## Export An exported depth model loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`, with `depth_map` in place of boxes. **Export** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDepthAnythingV2s-depth.pt") model.export(format="onnx") ``` **Run the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreDepthAnythingV2s-depth.onnx") result = model(SAMPLE_IMAGE) print(result.depth_map.data.shape) ``` Coverage differs per family, and Depth Anything 3 rejects any format outside its validated set rather than attempting an unvalidated conversion. Check the model page and the [full export matrix](/docs/reference/export-matrix) before committing to a target. LibreMODUS and SenseNova-Vision do not export at all. [Export](/docs/export) lists the arguments every format accepts. --- # Edge detection Edge detection predicts how likely each pixel is to lie on an object boundary. LibreYOLO exposes it as the edge task, which returns a dense probability map on the original image canvas rather than a set of line segments. Verified against LibreYOLO v1.5.0. ## Definition The `edge` task predicts one probability per pixel from a single RGB image: `0` means non-edge and `1` means edge. The map stays continuous, so choosing the threshold that turns it into a binary boundary image is left to the caller, and the right threshold depends on the dataset and the downstream use. A prediction fills `result.edges`, an `EdgeMap` payload holding an `(H, W)` float32 array in `[0, 1]` on the original image canvas. `.array` returns that map as NumPy and `.binary(threshold)` returns a boolean mask. `result.boxes` stays empty, so `conf`, `iou` and `max_det` have no effect. `Results.plot()` covers this task and renders the map directly. ## Models Three families serve `edge`. [DexiNed](/docs/models/dexined), the Dense Extreme Inception Network, fuses several side outputs into one probability map and runs at a native 352 px. [TEED](/docs/models/teed), the Tiny and Efficient Edge Detector, is a small network at the same native 352 px, with a downsample stride of 4 against DexiNed's 16, so it accepts more values of `imgsz`. [LibreMODUS](/docs/models/libremodus) produces Canny-style edges as one target of an any-to-any model. It needs the `modus` extra and your own authenticated Hugging Face account, and it offers neither `val()` nor `export()`, so it does not take part in the validation and export sections below. ## Predict LibreYOLO publishes no edge checkpoint. The officially released DexiNed and TEED weights are trained on BIPED, whose published dataset terms restrict use to non-commercial purposes, so LibreYOLO does not mirror them. Convert a checkpoint you are licensed to use, then load the converted file by path: ```bash python weights/convert_dexined_weights.py upstream.pth weights/LibreDexiNedb-edge.pt --verify ``` **Predict an edge map** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # No edge checkpoint ships with LibreYOLO; convert one first (below). model = LibreYOLO("weights/LibreDexiNedb-edge.pt") result = model(SAMPLE_IMAGE, save=True) edges = result.edges print(edges.array.shape) # (H, W) float32 in [0, 1] print(edges.binary(0.5).sum()) # edge-pixel count at 0.5 ``` **Choose your own threshold** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("weights/LibreDexiNedb-edge.pt") result = model(SAMPLE_IMAGE) # The continuous map is kept so the threshold stays your decision. for t in (0.3, 0.5, 0.7): print(t, int(result.edges.binary(t).sum())) ``` **Save the visualization** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("weights/LibreDexiNedb-edge.pt") result = model(SAMPLE_IMAGE) # plot() renders the map; it is defined for edge and normal results. result.plot().save("edges.png") ``` The filename has to carry the `-edge` task suffix for the loader to recognize it. `imgsz` must be divisible by the network's downsample stride, and LibreYOLO raises a clear error naming the divisor when it is not. See [prediction](/docs/predict) for sources, streaming and result handling. ## Dataset format Edge validation pairs each RGB image with a same-stem single-channel map of the same resolution, plus an optional validity mask. ```text dataset/ data.yaml images/ val/scene.jpg edges/ val/scene.png masks/ val/scene.png ``` ```yaml path: dataset train: images/train val: images/val edges_dir: edges masks_dir: masks nc: 1 names: {0: edge} ``` The target is a single-channel PNG or TIF, not an RGB visualization. Integer maps are divided by the maximum of their dtype; float maps must already be finite and in `[0, 1]`. Mask pixels count as valid when nonzero, and padded pixels never contribute to a metric. `edge_invert: true` covers sources that store black edges on white. See [dataset formats](/docs/reference/dataset-formats) for the full contract. ## Train No edge family in LibreYOLO has a training implementation: `train()` raises `NotImplementedError` on all three. Each model page names the conversion script that turns a checkpoint trained elsewhere into one LibreYOLO can load. ## Validate `val()` reports the BSDS-style F-measures. Continuous predictions are thinned first with four-direction gradient non-maximum suppression, then predicted and ground-truth edge pixels are matched one-to-one within a distance tolerance. **Validate and read the metric keys** ```python from libreyolo import LibreYOLO model = LibreYOLO("weights/LibreDexiNedb-edge.pt") metrics = model.val(data="my-dataset.yaml", imgsz=352) print(metrics["metrics/ODS"]) # fitness print(metrics["metrics/OIS"]) print(metrics["metrics/best_threshold"]) ``` **Change the sweep and the match tolerance** ```python from libreyolo import LibreYOLO model = LibreYOLO("weights/LibreDexiNedb-edge.pt") metrics = model.val( data="my-dataset.yaml", imgsz=352, edge_thresholds=(0.1, 0.2, 0.3, 0.4, 0.5), edge_max_dist=0.0075, ) print(metrics["metrics/ODS"], metrics["metrics/best_threshold"]) ``` `metrics/ODS` is the optimal-dataset-scale F-measure: match counts are pooled across the dataset at each threshold, and the best of those pooled F-measures is reported. It is also `fitness`, the number best-checkpoint selection reads. `metrics/OIS` is the optimal-image-scale F-measure, the mean over images of each image's own best F-measure, so it lets every image pick its own threshold. `metrics/best_threshold` is the single threshold that produced ODS, which is the one to reuse in `edges.binary()` at inference. Two arguments shape the sweep. `edge_thresholds` is the set of thresholds tried, defaulting to 0.01 through 0.99 in hundredths. `edge_max_dist` is the match tolerance as a fraction of the image diagonal, defaulting to `0.0075`; a pair further apart than that is not a match. ## Export An exported edge model loads back through `LibreYOLO()` on its file suffix, so a `.onnx` file behaves like a checkpoint and returns the same `Results`. **Export** ```python from libreyolo import LibreYOLO model = LibreYOLO("weights/LibreDexiNedb-edge.pt") model.export(format="onnx", imgsz=352) ``` **Run the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("weights/LibreDexiNedb-edge.onnx") result = model(SAMPLE_IMAGE) print(result.edges.array.shape) ``` Edge export uses a fixed-resolution, batch-1 runtime contract: `dynamic` and a `batch` other than 1 are rejected, and the exported graph emits a single fused probability map. Per-format coverage is on the [DexiNed](/docs/models/dexined) and [TEED](/docs/models/teed) pages and in the [full export matrix](/docs/reference/export-matrix). [Export](/docs/export) lists the arguments every format accepts. --- # Embeddings One task covers every vector LibreYOLO produces. embed returns unit-length float32 rows whose dot product is a similarity score, whether the row describes a whole image, a single detected face, or a line of text, and the same Gallery matches all of them. Verified against LibreYOLO v1.5.0. ## Definition `embed` turns an image, a region of an image, or a string into a float32 row of fixed width whose length is one. Because every row is a unit vector, comparing two of them is a dot product, and comparing two sets of them is a single matrix multiplication. Nothing else in the task is model specific: retrieval, duplicate detection, re-identification and face recognition are all the same arithmetic over different rows. The vector is the output. There is no class list, so a name is attached later by comparing against references you supply rather than by anything the network was trained to predict. ### Three shapes | Shape | `Results.embeddings` | `Results.boxes` | Produced by | |---|---|---|---| | Whole image | `(1, D)` | `None` | Passing an image to a whole-image family | | Region | `(N, D)` | `(N, 4)`, row-aligned | Families that localize first, such as face recognition | | Text | not a `Results` at all | | `model.embed_text(texts)`, returning `(M, D)` | A whole-image result stays two dimensional even for one image. `(D,)` is not a permitted return shape, so a consumer never has to special-case the single-row case. Text returns a plain tensor rather than a `Results`, because a string is not an image source: passing one to `model(...)` still means a path or a URL, and the library never guesses that a string is prose. The canonical task key is `embed`. `embedding`, `embeddings`, `face-recognition`, `facial-recognition`, `recognition`, `face`, `faceid` and `reid` all normalize to it, so `task="reid"` and `task="embed"` select exactly the same thing. ## Models Four families serve the task, and they split cleanly by whether they localize anything first. | Family | Shape | Dimension | Also supports | |---|---|---|---| | [LibreFaceRec](/docs/models/librefacerec) | Region, one row per detected face | 512 | Nothing; `embed` is its only task | | [CLIP](/docs/models/clip) | Whole image, with a paired text tower | 512 for `b32` and `b16`, 768 for `l14` | `classify`, which stays its default | | [SigLIP 2](/docs/models/siglip2) | Whole image, with a paired text tower | 768 for `b16`, 1152 for `so400m` | `classify`, which stays its default | | [DINOv2](/docs/models/dinov2) | Whole image, image only | 384 | `semantic`, `classify` | CLIP and SigLIP 2 keep `classify` as their default task, so `task="embed"` has to be asked for. Their existing `-cls` checkpoint is the shared two-tower artifact; no duplicate `-embed` checkpoint is published for identical weights. `embed_text` exists only on CLIP and SigLIP 2, the two families with a text tower. DINOv2 has none. DINOv2 embedding bypasses the semantic and classification heads and reads the final normalized CLS token at 224 pixels; the `n`, `s`, `m` and `l` variants all share the DINOv2-S encoder, so all four return `D = 384`. The classification-only backbones added in this release, [ViT](/docs/models/vit), [Swin](/docs/models/swin) and [DeiT](/docs/models/deit), declare `classify` only and do not serve this task. **Whole image** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # CLIP defaults to classify, so ask for the vector explicitly. model = LibreYOLO("LibreCLIPb32-cls.pt", task="embed") result = model(SAMPLE_IMAGE) print(result.embeddings.data.shape) # (1, 512), one row per image print(result.boxes) # None: nothing was localized ``` **Per region** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("librefacerec-l.onnx") result = model(SAMPLE_IMAGE) # Row i describes the region in box i. print(result.boxes.xyxy.shape) # (N, 4) print(result.embeddings.data.shape) # (N, 512) ``` **Many images at once** ```python from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="embed") # Every row from every result, concatenated into one tensor. vectors = model.embed(["a.jpg", "b.jpg", "c.jpg"]) print(vectors.shape) # (3, 384) ``` **Text** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreCLIPb32-cls.pt", task="embed") # Text is a method, never a prediction source. A string passed to # model(...) is still a path or a URL. text = model.embed_text(["a photo of a cat", "a photo of a dog"]) print(text.shape) # (2, 512) ``` `model.embed(source, **kwargs)` is the batch shortcut: it runs `predict` and concatenates every row from every result into one `(N_total, D)` CPU float32 tensor, raising if the rows have mixed dimensions. A family without `embed` in its supported tasks raises `NotImplementedError`. ## Result payloads `result.embeddings` is an `Embeddings` payload. Its `data` is always `(N, D)` float32, already L2-normalized by the inference path, and a non-two-dimensional input raises rather than being reshaped silently. | Member | Meaning | |---|---| | `.data` | The `(N, D)` matrix | | `.dim` | `D` | | `.normalized` | The same rows, defensively re-normalized | | `.similarity(other)` | `(N, M)` against another set, or `(N,)` against a single `(D,)` vector | | `.verify(i, j, threshold=0.4)` | Whether rows `i` and `j` are the same subject | `result.identities` is an `Identities` payload, present only when a gallery was passed. It is a plain container, not a tensor, so moving a `Results` between devices leaves it alone. | Member | Meaning | |---|---| | `.name` | List of names, `None` where nothing cleared the threshold | | `.score` | `(N,)` float32 best cosine score, kept even when the name is `None` | | `.data` | List of `(name, score)` tuples | **Compare two sets of rows** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreCLIPb32-cls.pt", task="embed") query = model.embed("query.jpg") # (1, 512) pool = model.embed(["a.jpg", "b.jpg"]) # (2, 512) # Rows are unit length, so cosine similarity is a dot product. scores = model("query.jpg").embeddings.similarity(pool) print(scores.shape) # (1, 2) ``` **Image against text** ```python import torch from libreyolo import LibreYOLO model = LibreYOLO("LibreCLIPb32-cls.pt", task="embed") image = model.embed("photo.jpg") # (1, 512) text = model.embed_text(["a cat", "a dog", "a car"]) # (3, 512) print(torch.matmul(image, text.T)) ``` Vectors are left out of `summary()` and `to_json()` by default, since a 512-float row is about two kilobytes per subject. Each row reports `embedding_dim` instead, plus `identity` and `identity_score` when a gallery was used. Pass `summary(embeddings=True)` to include the numbers. ## Galleries A `Gallery` is a named set of reference rows. It stores each reference separately rather than averaging them, so a name is scored by its single best matching reference, and adding a bad photo cannot drag an identity's centroid around. **Enroll and identify** ```python from libreyolo import Gallery, LibreYOLO model = LibreYOLO("librefacerec-l.onnx") gallery = Gallery(model) gallery.enroll("ada", ["people/ada/1.jpg", "people/ada/2.jpg"]) gallery.enroll("grace", "people/grace/1.jpg") gallery.save("refs.npz") result = model("group.jpg", gallery=gallery, threshold=0.4) for name, score in result.identities.data: print(name, score) # name is None below the threshold ``` **Top-k search** ```python from libreyolo import Gallery from libreyolo.models.dinov2.model import LibreDINOv2 model = LibreDINOv2(size="s", task="embed") gallery = Gallery.load("refs.npz", model=model) result = model("query.jpg") matches = gallery.match(result.embeddings, top_k=5, threshold=0.4) print(matches[0]) # [(name, score), ...] for the first row ``` **Enroll a vector you already hold** ```python from libreyolo import Gallery gallery = Gallery() gallery.enroll_embedding("ada", vector) # normalized on the way in print(gallery.identities, gallery.dim, len(gallery)) ``` `Gallery(model)` binds to the weights that will produce its vectors. `enroll(name, sources, select="best")` runs prediction on each source and keeps the highest-confidence row per result; `select="all"` keeps every row instead, which is what you want when a reference image legitimately contains several subjects. `enroll_embedding(name, vector)` skips inference and takes a vector directly, normalizing it and rejecting an all-zero row. `FaceGallery` is a permanent alias of the same class, and archives written by earlier face-only releases still load. ### Matching and thresholds Matching is a dense matrix multiplication against every stored reference, reduced to one score per name by taking the maximum. There is no approximate index, which keeps the numbers exact and puts a practical ceiling on gallery size. Two entry points differ in what they do below the threshold. `match()` returns `[(name, score), ...]` per row with everything under the threshold dropped, so a row with no match is an empty list. `identify()` returns an `Identities` payload that always keeps the best score and sets the name to `None` when it is under the threshold. Neither ever substitutes the nearest below-threshold name. The default threshold is `0.4` throughout. It is a cosine value, not a probability, and the right operating point is a property of your data and your tolerance for false matches, so sweep it on labeled pairs rather than accepting the default. `libreyolo enroll` and the `gallery=` prediction argument use the same number. ### Persistence `save(path)` writes a compressed `.npz` holding the vectors, the names and a metadata block carrying the format version, the embedding dimension and a fingerprint of the weights that produced the rows. `Gallery.load(path, model=...)` checks both before comparing anything, so pointing a gallery at a different model raises instead of silently scoring vectors from two unrelated spaces against each other. Saving an empty gallery is refused. ## Command line | Command | Purpose | |---|---| | `libreyolo enroll` | Walk a folder-per-identity tree and write or extend a `.npz` gallery | | `libreyolo compare` | Embed the primary subject in two images and report cosine similarity | | `libreyolo verify` | The same command under a second name | | `libreyolo predict gallery=...` | Attach identities to an ordinary prediction run | **Enroll a folder tree** ```bash # source//*.jpg. An existing gallery is extended in place. libreyolo enroll model=librefacerec-l.onnx source=people/ gallery=refs.npz ``` **Identify while predicting** ```bash libreyolo predict model=librefacerec-l.onnx source=group.jpg \ gallery=refs.npz gallery_threshold=0.45 ``` **Compare two images** ```bash libreyolo compare model=librefacerec-l.onnx \ source=a.jpg source2=b.jpg threshold=0.4 # verify is the same command under a second name. libreyolo verify model=librefacerec-l.onnx source=a.jpg source2=b.jpg --json ``` Every LibreYOLO command accepts both `key=value` and `--key value`, so `gallery=refs.npz` and `--gallery refs.npz` are the same argument. `enroll` takes `model`, `source` and `gallery`, plus optional `face-detector`, `device`, `--json` and `--quiet`. It reads one folder per identity, where the folder name is the identity and every image inside contributes references: ```text people/ ada/ 1.jpg 2.jpg grace/ 1.jpg ``` An image that yields nothing is skipped with a line on stderr rather than aborting the run, and the summary reports how many references were stored for each name. An existing gallery file is extended in place, so identities can be added over time. `compare` and `verify` are one function registered twice. They take `model`, `source`, `source2` and an optional `threshold`, and print the cosine similarity, the same-or-different verdict and the threshold that produced it. `--json` prints the same three fields as an object. On `predict`, `gallery` points at a saved `.npz` and `gallery_threshold` overrides the `0.4` default. Passing a gallery to a model whose task is not `embed` is an error rather than a silent no-op, and a missing gallery file suggests the `libreyolo enroll` command that would create it. ## Faces Face recognition is the region shape of this task, and it is the only shipped implementation of that shape. It adds a detection and alignment stage in front of the embedding head, plus a `verify()` method, a bring-your-own-boxes argument, published accuracy numbers and calibration guidance for the threshold. All of that lives on [face recognition](/docs/tasks/face-recognition), which is the walkthrough to follow when the subject is faces. Everything on this page applies to it unchanged. ## Train, validate and export Nothing in this task trains inside LibreYOLO. The face embedding head is an ONNX artifact whose `train()`, `val()` and `export()` all raise; train a head upstream and load the file by path. CLIP, SigLIP 2 and DINOv2 train and export through their classification and segmentation tasks, not through `embed`. There is no retrieval validator. Measure verification accuracy on labeled pairs by sweeping `threshold`, and identification accuracy by enrolling a gallery and reading `identities.name` and `identities.score` on held-out images, counting a `None` name as a rejection. --- # Face recognition Face recognition is the embed task applied to faces. A detector locates and aligns every face, a recognition head returns an L2-normalized vector per face, and identity is decided by cosine similarity against enrolled references rather than by a fixed class list. Verified against LibreYOLO v1.5.0. ## Definition Face recognition returns a vector per face, not a label. Prediction runs two stages: a face detector locates each face and its five landmarks, the crop is warped to a canonical 112x112 alignment, and a recognition head emits an L2-normalized embedding. `result.embeddings` is an `Embeddings` payload of shape `(N, D)`, row-aligned with `result.boxes`, so row `i` describes the face in box `i`. Because rows are unit vectors, cosine similarity is a dot product, and `embeddings.similarity()` computes it against another `Embeddings` or a whole matrix in one call. Naming a face is a separate step. A `Gallery` holds named reference vectors; passing `gallery=` to `predict()` attaches `result.identities`, row-aligned with the embeddings, carrying a name and its best cosine score per face. A face below the match threshold keeps `None` as its name, and the nearest below-threshold name is never substituted. The library's canonical task key is `embed`. `face-recognition`, `facial-recognition`, `reid` and `face` all normalize to it, so `task="face-recognition"` and `task="embed"` select the same thing. Faces are the region shape of that wider task; [embeddings](/docs/tasks/embeddings) covers the whole-image and text shapes, the shared `Embeddings`, `Identities` and `Gallery` API, and the models that produce vectors without detecting anything. ## Models [LibreFaceRec](/docs/models/librefacerec) is the family for this task. It is two ONNX artifacts behind one call: `librefacerec-l.onnx`, an iResNet100 recognition head producing 512-d embeddings, and `librefacerec-det.onnx`, the default face detector with five landmarks, taken from the OpenCV zoo. Both download from the LibreYOLO Hugging Face org on first use. Any other ArcFace-convention ONNX file (aligned 112x112 in, `(N, D)` out) can replace the recognition head by passing its path instead of a `librefacerec-*` name. The `embed` task key is wider than faces. [CLIP](/docs/models/clip), [SigLIP2](/docs/models/siglip2) and [DINOv2](/docs/models/dinov2) also support `task="embed"` and return one whole-image vector, which is image retrieval rather than face identity. They share the `Gallery` and `Embeddings` API, so the enroll-and-match workflow below transfers, but they do not detect or align faces. The recognition head runs through `onnxruntime`, which the base install does not carry: ```bash pip install "libreyolo[onnx]" ``` ## Predict **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # librefacerec-* names route to the face-embedding family regardless # of file suffix, and download from the LibreYOLO Hugging Face org on # first use along with the default face detector. model = LibreYOLO("librefacerec-l.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) # (N, 4) face boxes print(result.embeddings.data.shape) # (N, D), one row per face print(result.embeddings.dim) ``` **CLI** ```bash libreyolo predict model=librefacerec-l.onnx source=photo.jpg ``` **Compare two images** ```python from libreyolo import LibreYOLO model = LibreYOLO("librefacerec-l.onnx") # Runs detection and embedding on both images and compares their # most confident face. Cosine similarity is in [-1, 1]. outcome = model.verify("person_a.jpg", "person_b.jpg", threshold=0.4) print(outcome["similarity"], outcome["same_person"]) ``` **Enroll a gallery and identify** ```python from libreyolo import Gallery, LibreYOLO model = LibreYOLO("librefacerec-l.onnx") gallery = Gallery(model) gallery.enroll("ada", ["people/ada/1.jpg", "people/ada/2.jpg"]) gallery.enroll("grace", "people/grace/1.jpg") gallery.save("faces.npz") result = model("group_photo.jpg", gallery=gallery, threshold=0.4) for name, score in result.identities.data: print(name, score) # name is None below the threshold ``` **Enroll and identify from the CLI** ```bash libreyolo enroll model=librefacerec-l.onnx source=people/ gallery=faces.npz libreyolo predict model=librefacerec-l.onnx source=group_photo.jpg gallery=faces.npz ``` **Bring your own face boxes** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("librefacerec-l.onnx") # face_boxes skips detection entirely; face_detector accepts a # callable, a LibreYOLO detection model, or a FaceDetector instance. result = model(SAMPLE_IMAGE, face_boxes=[[34, 12, 90, 80]]) print(result.embeddings.data.shape) ``` Left alone, `predict()` downloads and pairs the default detector. `face_detector` overrides it with a callable, a LibreYOLO detection model, or a `FaceDetector` instance, and can be set on the constructor or per call. `face_boxes` bypasses detection with boxes you already hold. On the CLI, `face_detector=` accepts a face-detector `.onnx` path or a LibreYOLO detector name. `model.verify(image_a, image_b)` is the two-image shortcut: it embeds the most confident face in each and returns `{"similarity", "same_person", "threshold"}`. `model.embed(sources)` returns every face row across one or more images stacked into a single `(N_total, D)` tensor. See [prediction](/docs/predict) for sources, streaming and result handling. ## Dataset format Enrollment reads a folder per identity. The folder name becomes the identity, and every image inside it contributes references for that name: ```text people/ ada/ 1.jpg 2.jpg grace/ 1.jpg ``` `libreyolo enroll` walks that tree and writes a `.npz` gallery. An existing gallery file is extended in place rather than replaced, so identities can be added over time. Galleries are bound to the weights that produced them by embedding dimension and a file fingerprint; matching with a different model raises instead of comparing incompatible vector spaces. By default each source image contributes one reference row, the most confident face, so a portrait containing bystanders enrolls only its subject. Pass `select="all"` to `Gallery.enroll` to store every returned row. ## Train No family in this task trains inside LibreYOLO. `LibreFaceEmbedder.train()` raises: train a recognition head upstream, export it to ONNX in the ArcFace convention, and load the file by path. ## Validate There is no dataset validator for this task, and `val()` raises rather than pretending otherwise. Verification accuracy is measured on labeled image pairs with `model.verify()`, sweeping `threshold` to pick the operating point you want. Identification accuracy is measured by enrolling a gallery and reading `result.identities.name` and `result.identities.score` on held-out images, counting a `None` name as a rejection. ## Export The recognition head is already an ONNX graph, so there is nothing to convert: `LibreFaceEmbedder.export()` raises. Deploy the `.onnx` file directly, or point LibreYOLO at it and let the family handle detection, alignment and normalization. --- # Gaze estimation Gaze estimation returns a look direction for every face in an image. LibreYOLO models it as a two-stage task: a face detector runs first, and a gaze head reads pitch and yaw from each face crop it returns. Verified against LibreYOLO v1.5.0. ## Definition Gaze estimation returns two angles per face. `result.gaze` is a `Gaze` payload of shape `(N, 2)`, column 0 pitch and column 1 yaw, in radians, aligned row by row with `result.boxes`, the detected face boxes. The convention is the one L2CS-Net uses: positive yaw rotates the gaze toward the subject's left, positive pitch rotates it downward. The same payload exposes `pitch_deg` and `yaw_deg` for degrees, and `direction_3d`, an `(N, 3)` unit vector in the camera frame with columns `(x, y, z)`. Because the task is two-stage, a prediction depends on two models. Faces the detector misses have no gaze row, and boxes it places badly produce angles from a badly cropped face. The canonical task key is `gaze`; `gaze-estimation` normalizes to it. ## Models [L2CS-Net](/docs/models/l2cs) is the only family serving this task. It pairs a ResNet trunk with two parallel angle-bin classification heads, one for pitch and one for yaw, over 448x448 face crops. Five backbone depths are supported architecturally, and one, the ResNet-50, has a published checkpoint. The weights carry a license restriction. They are trained on Gaze360, whose license permits research and non-commercial use only and forbids redistribution, so LibreYOLO mirrors nothing for this family. The one checkpoint the library can fetch automatically comes straight from the authors' own Google Drive distribution, over `gdown`, after printing the license terms. Read [L2CS-Net](/docs/models/l2cs) before deploying it. That download path needs the `gaze` extra: ```bash pip install "libreyolo[gaze]" ``` Without it the library prints manual download instructions instead of attempting the transfer. Predicting on and exporting a checkpoint you already hold needs no extra at all. ## Predict **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # With no face_detector given, prediction falls back to OpenCV's # bundled detector, so nothing downloads beyond the checkpoint. model = LibreYOLO("LibreL2CSr50.pt") result = model(SAMPLE_IMAGE) gaze = result.gaze print(gaze.pitch, gaze.yaw) # radians, one row per face print(gaze.pitch_deg, gaze.yaw_deg) # the same angles in degrees print(gaze.direction_3d) # (N, 3) unit vectors ``` **CLI** ```bash # Unlike the Python path, the CLI has no automatic fallback: gaze # models require an explicit face detector, and it must be a # LibreYOLO detector whose boxes are faces. libreyolo predict model=LibreL2CSr50.pt source=photo.jpg face_detector=face-detector.pt save=True ``` **Choose the face source** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreL2CSr50.pt") # Hand the gaze head boxes from a detector you already ran. result = model(SAMPLE_IMAGE, face_boxes=[[34, 12, 90, 80]]) # Or name one of the bundled detectors. result = model(SAMPLE_IMAGE, face_detector="yunet") ``` The face source is chosen in one of three ways. `face_boxes` passes boxes you already computed and skips detection. `face_detector` accepts `"auto"`, `"haar"`, `"yunet"`, a LibreYOLO detection model, or a plain callable, and can be set on the constructor or per call. Left unset in Python, prediction falls back to OpenCV's bundled detector, so a bare call works with no wiring. On OpenCV 4 that is the Haar cascade shipped inside the wheel, which needs no download at all; on OpenCV 5, where the Haar API was removed, it is YuNet, which fetches a small model file from the OpenCV zoo once. The CLI does not share that fallback. `libreyolo predict` rejects a gaze model without `face_detector=`, and the value it takes is a LibreYOLO detector name or checkpoint path. See [prediction](/docs/predict) for sources, streaming and result handling. ## Train No family in this task trains inside LibreYOLO. `LibreL2CS.train()` raises: train at the upstream L2CS-Net project and load the resulting state dict here. ## Validate Validation against gaze ground-truth datasets is out of scope, and `val()` raises rather than returning metrics it did not compute. There is no `metrics/` dictionary for this task. Evaluate upstream, on the dataset the checkpoint was trained for. ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreL2CSr50.pt") model.export(format="onnx") ``` **CLI** ```bash libreyolo export model=LibreL2CSr50.pt format=onnx ``` The gaze export contract covers ONNX, TorchScript, ExecuTorch, TensorRT and OpenVINO. What leaves the library is the ResNet trunk and the two angle-bin heads alone: the graph takes a preprocessed 448x448 face crop and returns raw yaw and pitch logits. Face detection, cropping, the softmax, the bin expectation and the conversion to angles all stay in Python, in `libreyolo.models.l2cs.utils`. See [export](/docs/export) for the formats and their arguments. --- # Image classification Image classification assigns one label distribution to a whole image and locates nothing inside it. The task key is classify. Verified against LibreYOLO v1.5.0. ## Definition Image classification produces one score per class for the whole image and no coordinates at all. It answers what is in the picture, never where, which is what separates it from [object detection](/docs/tasks/object-detection). `classify` is the canonical task key, and the `-cls` suffix in a checkpoint filename selects it. That suffix is required rather than optional on classification families, so `LibreResNet50.pt` is not read as a classifier and only `LibreResNet50-cls.pt` is. `predict()` fills `result.probs` and leaves `boxes` empty. `.data` is the full score vector, `.top1` the index of the highest score and `.top1conf` its value, `.top5` the five highest indices in descending order and `.top5conf` their scores. Indices point into `result.names`. Slicing a `Results` object never truncates `probs`, because the vector belongs to the image rather than to one row. ## Models Five families both train and predict: [ResNet](/docs/models/resnet), [ConvNeXt](/docs/models/convnext), [MobileNetV4](/docs/models/mobilenetv4), [EfficientNetV2](/docs/models/efficientnetv2) and [DINOv2](/docs/models/dinov2). The first four run on the base package and ship published weights. DINOv2 needs `pip install "libreyolo[rfdetr]"` and has no LibreYOLO-hosted checkpoint: it loads the upstream backbone with a randomly initialized linear head, so it is a fine-tuning starting point rather than a ready predictor. Five more predict, validate and export, but their `train()` raises `NotImplementedError`: [ViT](/docs/models/vit), [Swin](/docs/models/swin), [VGG](/docs/models/vgg), [AlexNet](/docs/models/alexnet) and [DeiT](/docs/models/deit). [CLIP](/docs/models/clip) and [SigLIP2](/docs/models/siglip2) classify without a fixed label set. They score the image against text prompts, so `set_classes()` defines the classes at call time and there is no training step for a new label set at all. Both also serve the `embed` task. ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The -cls suffix in the filename selects the task, so no task # argument is needed. model = LibreYOLO("LibreResNet50-cls.pt") result = model(SAMPLE_IMAGE, save=True) print(result.names[result.probs.top1], float(result.probs.top1conf)) print(result.probs.top5) ``` **CLI** ```bash libreyolo predict model=LibreResNet50-cls.pt save=True \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` **The whole distribution** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE result = LibreYOLO("LibreResNet50-cls.pt")(SAMPLE_IMAGE) probs = result.probs # .data is the full (C,) vector; top5/top5conf are ordered views. print(probs.data.shape) for index, score in zip(probs.top5, probs.top5conf): print(result.names[index], float(score)) ``` **Zero-shot, no training** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # CLIP scores the image against text prompts, so the label set is # set at call time instead of baked into the checkpoint. model = LibreYOLO("LibreCLIPb32-cls.pt") model.set_classes(["a person jumping", "an empty street", "a parked car"]) result = model(SAMPLE_IMAGE) print(model.names[result.probs.top1], float(result.probs.top1conf)) ``` `conf`, `iou` and `max_det` have no effect here: there are no candidates to threshold or suppress, only one distribution. See [prediction](/docs/predict) for sources, streaming and result handling. ## Dataset format Classification uses a directory tree, not label files and not a YAML. `data` is the dataset root. ```text dataset/ train/ tench/000001.jpg parachute/000002.jpg val/ tench/000101.jpg parachute/000102.jpg ``` `train/` is required for training and it defines the class-to-index mapping by sorted folder name, so the first folder alphabetically becomes class 0. `val/` is required for validation. A `test/` split may be present and the default train and validate commands do not use it. Any split other than `train` has to contain the same class folder names as the expected class set, which is what makes a mismatch fail loudly rather than score as a wrong prediction. The accepted image extensions are `.jpg`, `.jpeg`, `.png`, `.bmp`, `.webp`, `.tif` and `.tiff`. `data` accepts three things: a path to a directory containing a `train/` split, a `.zip` URL, or one of the known dataset names, `imagenette160` and `smoke10`, which download and cache on first use. The canonical loader is `libreyolo.data.classify_dataset`. ## Train **Python** ```python from libreyolo import LibreYOLO # imagenette160 is a known dataset name and downloads on first use. # Pass a directory with a train/ split for your own data. model = LibreYOLO("LibreResNet50-cls.pt") model.train(data="imagenette160", epochs=5) ``` **CLI** ```bash libreyolo train model=LibreResNet50-cls.pt data=imagenette160 epochs=5 ``` **Multi-GPU** ```bash libreyolo train model=LibreResNet50-cls.pt data=imagenette160 \ epochs=50 device=0,1 batch=-1 ``` There is no `nc` to declare: the class count comes from the folder names under `train/`, and the final linear layer is rebuilt to match it while the backbone transfers unchanged. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a plain dictionary of `metrics/` keys, computed over the `val/` split of the dataset root. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreResNet50-cls.pt") # val() returns a plain dict, not an object. metrics = model.val(data="imagenette160") print(metrics["metrics/accuracy_top1"]) print(metrics["metrics/accuracy_top5"]) ``` **CLI** ```bash libreyolo val model=LibreResNet50-cls.pt data=imagenette160 ``` `metrics/accuracy_top1` is the share of images whose highest-scoring class is the true one, and it is the headline number, the one training uses to pick the best epoch. `metrics/accuracy_top5` is the share whose true class appears anywhere in the five highest-scoring classes, which says less the fewer classes the dataset has. The dictionary also carries `fitness`, a copy of the top-1 value. ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreResNet50-cls.pt") model.export(format="onnx") ``` **CLI** ```bash libreyolo export model=LibreResNet50-cls.pt format=onnx ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like a checkpoint and returns the same Results object. model = LibreYOLO("LibreResNet50-cls.onnx") result = model(SAMPLE_IMAGE) print(result.probs.top1, result.probs.top1conf) ``` An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Format coverage differs by family; the matrix on each model page is generated from the validated set rather than typed by hand. See [export and deploy](/docs/export) for the formats, their extras and their constraints. --- # Image restoration Image restoration takes a degraded image and returns a clean one. LibreYOLO exposes it as the restore task, which covers denoising, deblurring and super-resolution behind a single output contract: one RGB image in, one RGB image out. Verified against LibreYOLO v1.5.0. ## Definition The `restore` task maps one image to another image. Denoising, deblurring and super-resolution are all the same task here, because they share one contract: the model consumes an RGB image and returns an RGB image, and the degradation it was trained to undo is a property of the checkpoint rather than of the API. A prediction fills `result.restored`, a `RestoredImage` payload holding an `(H, W, 3)` uint8 RGB array. `.array` returns it as NumPy and `.save(path)` writes it to disk. `result.restore_scale` records the upscale factor the output canvas carries, which is `1` for a checkpoint that preserves resolution. `result.boxes` stays empty, so `conf`, `iou` and `max_det` are accepted for signature parity but have no effect, and `save=True` writes the restored image directly rather than an annotated photo. ## Models Three families serve `restore`, split by the degradation they undo. [NAFNet](/docs/models/nafnet) is the denoiser, and the only restore family LibreYOLO can train. Its architecture replaces the nonlinear activations of a UNet block with elementwise multiplication, and the published checkpoint is trained on SIDD real-image noise. Output stays at the input resolution. [Real-ESRGAN](/docs/models/real-esrgan) is the practical upscaler: three checkpoints trained against synthetic degradations rather than only bicubic downscaling, at 4x, 2x, and a smaller, faster 4x generator built for lower latency. [SwinIR](/docs/models/swinir) upscales 4x with a Swin Transformer backbone, in three sizes covering the official lightweight generator and two real-world generators. ## Predict Weights download from Hugging Face on first use and are cached locally. **Upscale an image** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The compact 4x generator; tile bounds peak memory on a large source. model = LibreYOLO("LibreRealESRGANx4t-restore.pt") result = model(SAMPLE_IMAGE, tile=512, tile_pad=10) result.restored.save("upscaled.png") print(result.restored.array.shape) # 4x the input in each axis ``` **Denoise an image** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # Trained on SIDD real-image noise; output stays at the input size. model = LibreYOLO("LibreNAFNetl-restore-sidd.pt") result = model(SAMPLE_IMAGE) result.restored.save("denoised.png") print(result.restore_scale) # 1: no upscale for this checkpoint ``` Restoration runs at the source image's own resolution rather than a fixed network canvas, padding only to the network's downsample factor, so both time and memory scale with the pixel count of your input. `tile` splits the forward pass into overlapping tiles and blends the seams back together, and `tile_pad` is the halo added around each tile before it is cropped back out; both are Python keyword arguments. See [prediction](/docs/predict) for sources, streaming and result handling. ## Dataset format Restoration pairs each degraded input image with a clean target image of exactly the same resolution, matched by filename stem. ```text dataset/ data.yaml inputs/ train/photo.jpg val/photo.jpg targets/ train/photo.jpg val/photo.jpg ``` ```yaml path: dataset train: inputs/train val: inputs/val input_dir: inputs target_dir: targets degradation: denoise dataset: MyDataset nc: 1 names: {0: image} ``` `nc` and `names` are schema placeholders; a restore model returns `Results.restored`, not detections. `degradation` and `dataset` are optional provenance labels. `target_stem_suffix` covers datasets that name the clean image differently from its degraded pair. Validation keeps native resolution and pads only enough to stack a batch, so the metrics are computed on the original canvas. See [dataset formats](/docs/reference/dataset-formats) for the full contract. ## Train NAFNet is the only restore family with a training implementation. `Real-ESRGAN.train()` and `SwinIR.train()` both raise `NotImplementedError`: those checkpoints come from GAN training over synthetic degradation pipelines, and the paired restore trainer would run without reproducing that recipe. **Fine-tune NAFNet on paired images** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreNAFNetl-restore-sidd.pt") model.train(data="my-dataset.yaml", epochs=100, imgsz=256, batch=16, lr0=1e-3) ``` **Record the provenance on the checkpoint** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreNAFNetl-restore-sidd.pt") # degradation and dataset are written into the saved checkpoint for # provenance; they take no part in training. model.train( data="my-dataset.yaml", epochs=100, degradation="denoise", dataset="MyDataset", ) ``` The trainer takes coupled crops of the input and target pair, so both sides stay aligned. See [training](/docs/train) for datasets, multi-GPU and loggers, and the [NAFNet page](/docs/models/nafnet) for this family's defaults and the inference-time pooling it detaches while training. ## Validate `val()` compares the restored output against the clean target, in RGB, on the original canvas, with no border crop and no resizing. **Validate and read the metric keys** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreNAFNetl-restore-sidd.pt") # val() returns a plain dict, not an object. metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/PSNR"]) # fitness print(metrics["metrics/SSIM"]) ``` `metrics/PSNR` is the peak signal-to-noise ratio in decibels, and it is also `fitness`, the number best-checkpoint selection reads. `metrics/SSIM` is structural similarity in `[0, 1]`, computed with an 11x11 Gaussian window at sigma 1.5 and averaged over the three color channels. Higher is better for both. ## Export An exported restore model loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`, with `restored` carrying the output image. **Export** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreNAFNetl-restore-sidd.pt") # imgsz is fixed into the graph, so pass the size your deployment # actually feeds the model. model.export(format="onnx", imgsz=256) ``` **Run the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreNAFNetl-restore-sidd.onnx") result = model(SAMPLE_IMAGE) result.restored.save("denoised.png") ``` Restore export fixes the spatial resolution into the graph, so pass the `imgsz` your deployment will actually feed the model. For NAFNet that size must divide by the network's downsample factor, and only the batch dimension stays dynamic under `dynamic=True`. For Real-ESRGAN and SwinIR, leaving `imgsz` out falls back to a small internal patch size rather than your working resolution. Per-format coverage is on each model page and in the [full export matrix](/docs/reference/export-matrix). [Export](/docs/export) lists the arguments every format accepts. --- # Instance segmentation Instance segmentation locates every object instance and returns a per-pixel mask for each one, alongside the box, class and score a detector returns. The task key is segment. Verified against LibreYOLO v1.5.0. ## Definition Instance segmentation is detection plus shape. Each object instance still gets a box, a class and a score, and it also gets a binary mask covering the pixels that belong to it. Masks may overlap, and pixels belonging to no object are left unassigned, which is what separates the task from [semantic segmentation](/docs/tasks/semantic-segmentation) and [panoptic segmentation](/docs/tasks/panoptic-segmentation). `segment` is the canonical task key, and the `-seg` suffix in a checkpoint filename selects it, so `task=` is not needed when loading published weights. `predict()` fills `result.masks` alongside `result.boxes`. `.data` is an `(N, H, W)` stack on the original image canvas, row-aligned with the boxes, so mask `i` belongs to box `i`. `.xy` converts each mask to its largest outer contour as a `(P, 2)` pixel array, and `.xyn` gives the same contour normalized. ## Models Four families both train and predict masks: [RF-DETR](/docs/models/rf-detr), [EdgeCrafter](/docs/models/edgecrafter), [D-FINE](/docs/models/d-fine) and [RTMDet](/docs/models/rtmdet). RF-DETR needs its own extra, `pip install "libreyolo[rfdetr]"`; the other three run on the base package. [Mask R-CNN](/docs/models/mask-rcnn) predicts, validates and exports masks, but its `train()` raises `NotImplementedError`. [EoMT](/docs/models/eomt) predicts and validates masks and also cannot train, and its export is narrower still: `export()` only accepts the semantic task, and raises `NotImplementedError` for `segment` and `panoptic`, because the query-mask runtime contract those two need has not been defined. Use EoMT for instance masks in Python, not through an exported graph. A separate group segments from a prompt rather than a class list: a click, a box or a phrase picks the object, and the model returns its mask. [SAM](/docs/models/sam), [SAM 2](/docs/models/sam-2), [SAM 3](/docs/models/sam-3), [MobileSAM](/docs/models/mobilesam), [EdgeTAM](/docs/models/edgetam) and [PicoSAM3](/docs/models/picosam3) work this way, as does [SenseNova-Vision](/docs/models/sensenova-vision), whose segmentation is referring: it takes a phrase naming one object. They load through their own factory and extras, and each model page carries the exact call. ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The -seg suffix in the filename selects the mask head, so no task # argument is needed. model = LibreYOLO("LibreDFINEn-seg.pt") result = model(SAMPLE_IMAGE, save=True) print(result.masks.data.shape) # (N, H, W), one mask per detection print(result.boxes.xyxy.shape) # (N, 4), the same N rows ``` **CLI** ```bash libreyolo predict model=LibreDFINEn-seg.pt save=True \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` **Mask outlines** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreDFINEn-seg.pt") result = model(SAMPLE_IMAGE) # .xy is a list of (P, 2) contours in pixels, .xyn the same normalized. for name, contour in zip(result.boxes.cls, result.masks.xy): print(result.names[int(name)], contour.shape) ``` **Another family, same call** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreRTMDets-seg.pt") result = model(SAMPLE_IMAGE) print(result.masks.data.shape) ``` `conf` and `max_det` shape the output the same way they do for detection, and masks are filtered along with the boxes they belong to. See [prediction](/docs/predict) for sources, streaming and result handling. ## Dataset format The layout is the detection layout: one `.txt` label file per image, found by swapping `images` for `labels` in the image path and changing the extension. ```text dataset/ data.yaml images/ train/000001.jpg val/000101.jpg labels/ train/000001.txt val/000101.txt ``` What changes is the row. A segment is a class index followed by a flat polygon: ```text ... ``` At least three points, so the coordinate count after the class index is even and at least six, and the polygon must be non-degenerate. Coordinates are floats in `[0, 1]` relative to the original image width and height. A five field detection row is also accepted in a segmentation dataset and is read as a rectangular segment, which makes a box-only dataset loadable without a conversion pass. The YAML is the detection YAML: ```yaml path: dataset train: images/train val: images/val names: 0: person 1: bicycle ``` Native COCO JSON works as well: add an `annotations` mapping of split name to JSON file, and the split path gives the image root. ## Train **Python** ```python from libreyolo import LibreYOLO # Continues from published segmentation weights, mask head included. # data must point at a dataset whose labels carry polygons. model = LibreYOLO("LibreDFINEn-seg.pt") model.train(data="my-dataset.yaml", epochs=50, imgsz=640, batch=8, lr0=2e-4) ``` **CLI** ```bash libreyolo train model=LibreDFINEn-seg.pt data=my-dataset.yaml \ epochs=50 imgsz=640 batch=8 lr0=2e-4 ``` **From detection weights** ```bash # Detection weights carry no mask head, so this is an explicit # transfer: the head starts untrained. Asking for task=segment is # what authorizes it. libreyolo train model=LibreDFINEn.pt data=my-dataset.yaml \ task=segment epochs=50 imgsz=640 ``` Training continues from a published `-seg` checkpoint by default. Starting from detection weights is possible but is a deliberate transfer: those weights carry no mask head, so it starts untrained, and passing `task=segment` is what authorizes the swap. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a plain dictionary of `metrics/` keys. Boxes and masks are scored separately, both with COCO evaluation, and the mask numbers are the primary ones. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDFINEn-seg.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"]) # masks print(metrics["metrics/mAP50-95(M)"]) # masks, explicit print(metrics["metrics/mAP50-95(B)"]) # boxes ``` **CLI** ```bash libreyolo val model=LibreDFINEn-seg.pt data=my-dataset.yaml ``` The unsuffixed keys hold mask results: `metrics/mAP50-95`, `metrics/mAP50`, `metrics/mAP75`, then `metrics/mAP_small`, `metrics/mAP_medium` and `metrics/mAP_large` by object area, and `metrics/AR1`, `metrics/AR10`, `metrics/AR100`, `metrics/AR_small`, `metrics/AR_medium`, `metrics/AR_large` for average recall. `metrics/AR_max_det` and `metrics/max_det` record the detection cap the run used. Four figures are also published under an explicit suffix, `(M)` for mask and `(B)` for box, so that a comparison never depends on which number the family decided to call primary: `metrics/mAP50-95(M)` and `metrics/mAP50-95(B)`, `metrics/mAP50(M)` and `metrics/mAP50(B)`, `metrics/precision(M)` and `metrics/precision(B)`, `metrics/recall(M)` and `metrics/recall(B)`. There is no unsuffixed `metrics/precision` or `metrics/recall` on this task. Read the precision and recall keys carefully. They are kept for backward compatibility and are aliases, not an operating point: `metrics/precision(M)` holds the same value as `metrics/mAP50-95(M)`, and `metrics/recall(M)` the same value as mask AR at 100 detections, with `(B)` behaving the same way for boxes. Plotting a pair of them reports one number twice. ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreDFINEn-seg.pt") model.export(format="onnx", imgsz=640) ``` **CLI** ```bash libreyolo export model=LibreDFINEn-seg.pt format=onnx imgsz=640 ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like a checkpoint and returns the same Results object. model = LibreYOLO("LibreDFINEn-seg.onnx") result = model(SAMPLE_IMAGE) print(result.masks.data.shape) ``` An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Segmentation coverage is narrower than detection coverage on the same family. The matrix on each model page is generated from the validated set and names the reason a target is unavailable. See [export and deploy](/docs/export) for the formats, their extras and their constraints. --- # Object detection Object detection locates every object instance in an image and returns an axis-aligned rectangle, a class label and a score for each one. The task key is detect. Verified against LibreYOLO v1.5.0. ## Definition Object detection answers where each object is and what it is. One image in, one row per instance out: four numbers for the rectangle, a class index and a score. Nothing about pixel shape, orientation or parts is included, which is what separates it from [instance segmentation](/docs/tasks/instance-segmentation), [oriented boxes](/docs/tasks/oriented-detection) and [pose](/docs/tasks/pose-estimation). `detect` is the canonical task key and the default: a checkpoint whose filename carries no task suffix loads as a detector. `predict()` fills `result.boxes`. `.xyxy` gives pixel corners on the original image canvas, `.conf` the score, and `.cls` the class index into `result.names`. `.xywh`, `.xyxyn` and `.xywhn` are derived views of the same rows, and `.id` carries a track id once a tracker is attached. Iterating a `Boxes` object yields one-row slices, so `box.cls`, `box.conf` and `box.xyxy` all work per detection. ## Models Twelve families both train and predict: [YOLOv9](/docs/models/yolov9), [RF-DETR](/docs/models/rf-detr), [EdgeCrafter](/docs/models/edgecrafter), [RT-DETR](/docs/models/rt-detr), [D-FINE](/docs/models/d-fine), [DEIM](/docs/models/deim), [Dome-DETR](/docs/models/dome-detr), [YOLO-NAS](/docs/models/yolo-nas), [YOLOX](/docs/models/yolox), [YOLOv7](/docs/models/yolov7), [RTMDet](/docs/models/rtmdet) and [PicoDet](/docs/models/picodet). YOLOv9 and RF-DETR are the two flagship families, and features land on them first. RF-DETR needs its own extra, `pip install "libreyolo[rfdetr]"`; the rest run on the base package. Eleven more predict, validate and export, but their `train()` raises `NotImplementedError`: [LW-DETR](/docs/models/lw-detr), [DETR](/docs/models/detr), [Deformable DETR](/docs/models/deformable-detr), [DINO-DETR](/docs/models/dino-detr), [Faster R-CNN](/docs/models/faster-rcnn), [Mask R-CNN](/docs/models/mask-rcnn), [FCOS](/docs/models/fcos), [RetinaNet](/docs/models/retinanet), [SSD](/docs/models/ssd), [CenterNet](/docs/models/centernet) and [EfficientDet](/docs/models/efficientdet). The Darknet lineage, [YOLOv1](/docs/models/yolov1), [YOLOv2](/docs/models/yolov2), [YOLOv3](/docs/models/yolov3) and [YOLOv4](/docs/models/yolov4), is kept as a frozen exhibit: predict, validate and export work, training does not. A separate group takes its class list at runtime rather than from the checkpoint, so it detects names never seen in training: [Grounding DINO](/docs/models/grounding-dino), [OWLv2](/docs/models/owlv2), [OMDet-Turbo](/docs/models/omdet-turbo) and [OV-DEIM](/docs/models/ov-deim), plus the vision-language families [Florence-2](/docs/models/florence-2), [Kosmos-2](/docs/models/kosmos-2), [Qwen3-VL](/docs/models/qwen3-vl), [SmolVLM2](/docs/models/smolvlm2), [InternVL3](/docs/models/internvl3), [LFM2-VL](/docs/models/lfm2-vl), [LocateAnything](/docs/models/locate-anything), [SenseNova-Vision](/docs/models/sensenova-vision) and [LibreMODUS](/docs/models/libremodus). These load through their own factory and extras; each model page carries the exact call. ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreYOLO9t.pt") result = model(SAMPLE_IMAGE, save=True) for box in result.boxes: print(result.names[int(box.cls)], float(box.conf), box.xyxy) ``` **CLI** ```bash libreyolo predict model=LibreYOLO9t.pt save=True \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` **Another family, same call** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the checkpoint, and every detector returns the # same Results object, so switching family is a one line change. model = LibreYOLO("LibreDFINEn.pt") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy.shape) ``` **Video and streams** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") # Any source the library accepts: file, folder, URL, webcam index, # RTSP stream, or a .streams list. for result in model.predict("clip.mp4", stream=True, save=True): print(len(result.boxes)) ``` `conf` sets the confidence threshold and `max_det` caps the number of rows. `iou` is the NMS threshold, so it only has an effect on a family that runs NMS; RF-DETR and the end-to-end YOLOv9 head decode a fixed set of predictions and ignore it. See [prediction](/docs/predict) for sources, streaming and result handling. ## Dataset format One `.txt` label file per image, found by swapping `images` for `labels` in the image path and changing the extension. ```text dataset/ data.yaml images/ train/000001.jpg val/000101.jpg labels/ train/000001.txt val/000101.txt ``` Each row is exactly five fields, a class index followed by a normalized center-and-size box: ```text ``` Coordinates are floats in `[0, 1]`, relative to the original image width and height. `w` and `h` must be positive. A missing or empty label file means the image has no objects. Rows carry no confidence and no track id. The YAML names the splits and the classes: ```yaml path: dataset train: images/train val: images/val names: 0: person 1: bicycle ``` `train` and `val` may be image directories, image-list `.txt` files, or lists of either. `nc` is optional and must match `names` when present. Native COCO JSON works too: add an `annotations` mapping of split name to JSON file, and the split path then gives the image root. When `names` is present it defines the label ids, so the JSON category names have to match it. ## Train **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") # coco128.yaml downloads a 128-image sample on first use. Point data # at your own dataset YAML for a real run. model.train(data="coco128.yaml", epochs=50, imgsz=640, batch=8) ``` **CLI** ```bash libreyolo train model=LibreYOLO9t.pt data=coco128.yaml \ epochs=50 imgsz=640 batch=8 ``` **Multi-GPU** ```bash libreyolo train model=LibreYOLO9t.pt data=coco128.yaml \ epochs=50 device=0,1 batch=-1 ``` `epochs`, `imgsz`, `batch` and `lr0` are the arguments that move first. `lr0` is the one that does not carry across families: a rate a convolutional detector tolerates will diverge a transformer one, so take the value from the model page rather than from another family's example. A family can also ignore an argument outright, and its page lists which. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a plain dictionary of `metrics/` keys, computed with COCO evaluation over the split named by `val` in the dataset YAML. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") # val() returns a plain dict, not an object. metrics = model.val(data="coco128.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"], metrics["metrics/mAP75"]) print(metrics["metrics/AR100"]) ``` **CLI** ```bash libreyolo val model=LibreYOLO9t.pt data=coco128.yaml ``` `metrics/mAP50-95` is mean average precision averaged over IoU thresholds 0.50 to 0.95, and it is the headline number. `metrics/mAP50` and `metrics/mAP75` are the single-threshold versions. `metrics/mAP_small`, `metrics/mAP_medium` and `metrics/mAP_large` split the same average by object area, and `metrics/AR1`, `metrics/AR10`, `metrics/AR100`, `metrics/AR_small`, `metrics/AR_medium` and `metrics/AR_large` are the matching average-recall figures. `metrics/AR_max_det` and `metrics/max_det` record the detection cap the run used. Read `metrics/precision` and `metrics/recall` carefully on this task. They are kept for backward compatibility and are aliases, not an operating point: `metrics/precision` holds the same value as `metrics/mAP50-95`, and `metrics/recall` the same value as `metrics/AR100`. Plotting them as a precision-recall pair reports one number twice. Four keys also repeat under a `(B)` suffix, for box, so that a detection key reads the same on a model that also predicts masks: `metrics/mAP50-95(B)`, `metrics/mAP50(B)`, `metrics/precision(B)` and `metrics/recall(B)`. ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9t.pt") model.export(format="onnx", imgsz=640) ``` **CLI** ```bash libreyolo export model=LibreYOLO9t.pt format=onnx imgsz=640 ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like a checkpoint and returns the same Results object. model = LibreYOLO("LibreYOLO9t.onnx") result = model(SAMPLE_IMAGE) print(result.boxes.xyxy) ``` An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Format coverage differs by family; the matrix on each model page is generated from the validated set rather than typed by hand. See [export and deploy](/docs/export) for the formats, their extras and their constraints. --- # Object tracking Tracking assigns a stable identity to each detection across video frames. LibreYOLO does not model it as a task with its own weights: it is a predict mode, model.track(), that runs a chosen tracker over the per-frame output of a detection, segmentation or pose model. Verified against LibreYOLO v1.5.0. ## Definition Tracking is not one of LibreYOLO's task keys, and there is no tracking checkpoint to download. It is a method on the model, `model.track(source)`, which runs detection on each frame and associates the results across time. The method is a generator: it yields one `Results` per processed frame, with `result.track_id` set to an `(N,)` integer tensor aligned with `result.boxes`. The same IDs are also on `result.boxes.id`. Only confirmed, currently tracked objects are yielded. A track the association loses stays alive for a configured number of frames before it is dropped, `track_buffer` for ByteTrack and BoT-SORT and `max_age` for the two OC-SORT variants, so an object recovered inside that window keeps its original ID. Because association happens after detection, the frame's other payloads survive it: the tracked `Results` is the detection `Results` sliced to the matched rows, so masks and keypoints come through with the boxes. ## Models Two independent choices go into a tracking run: the model that produces boxes each frame, and the tracker that links them. Any native LibreYOLO model whose task is detection, segmentation or pose exposes `track()`, so the choice of detector is the ordinary one. See [the model index](/docs/models) for the full list, or start from [YOLO9](/docs/models/yolov9), [RF-DETR](/docs/models/rf-detr), [D-FINE](/docs/models/d-fine) or [RTMDet](/docs/models/rtmdet). Tasks whose results have no box to associate refuse the call rather than returning meaningless IDs: classification, oriented boxes, points, depth, surface normals, edges, semantic and panoptic segmentation, restoration, OCR and body mesh all raise from `track()`. Two of LibreYOLO's model tiers also decline it. Models loaded through `LibreSAM` are image segmenters, and models loaded through `LibreOpenVocab` are per-frame detectors; both raise from `track()` and are used with `predict()` per frame instead. Tracking runs on native PyTorch models. An exported artifact loaded through `LibreYOLO("model.onnx")` returns a runtime backend object, which carries `predict()` but not `track()`. Four trackers ship with the library, selected by the `tracker` argument: `"bytetrack"` is the default. It is motion only, with a Kalman filter and a three-stage association: high-confidence detections first, then a second pass that gives low-confidence detections a chance to match an existing track before they are discarded, then unconfirmed tracks. Configured with `TrackConfig`. `"botsort"` keeps ByteTrack's three-stage lifecycle but uses a center-width-height Kalman state and compensates predicted tracks for camera motion before matching. This is the motion-only variant of BoT-SORT; it runs no appearance model. Configured with `BoTSortConfig`, which adds `enable_cmc`, `cmc_method` and `cmc_downscale`. `"ocsort"` is also motion only, and adds a velocity-direction term to the association cost, a second association pass against each track's last real observation, and a smoothing of the Kalman state along a virtual trajectory when a track is re-found. Configured with `OCSortConfig`. `"deepocsort"` extends OC-SORT with appearance. Each track keeps a confidence-weighted moving average of re-identification embeddings, and a cosine similarity term joins the association cost, so identities survive long occlusions and crossing targets. It costs one small embedding network forward per frame, and its OSNet weights download on first use. Configured with `DeepOCSortConfig`. ## Predict **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") # track() is a generator: one Results per processed frame. for result in model.track("video.mp4"): print(result.track_id) # (N,) int tensor, aligned with boxes print(result.boxes.xyxy) ``` **Choose a tracker** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") # "bytetrack" (default), "botsort", "ocsort" or "deepocsort". for result in model.track("video.mp4", tracker="botsort"): print(result.track_id) ``` **Save an annotated video** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") # Without output_path, the file lands in runs/track/.mp4. for result in model.track("video.mp4", save=True, vid_stride=2): pass ``` **Tune a tracker** ```python from libreyolo import BoTSortConfig, LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") # The config type selects the tracker, so tracker= is redundant here. config = BoTSortConfig(track_buffer=60, frame_rate=25, enable_cmc=False) for result in model.track("video.mp4", tracker_config=config): print(result.track_id) # Or pass the same fields as keyword arguments and let track() build it. for result in model.track("video.mp4", tracker="botsort", track_buffer=60): print(result.track_id) ``` `track_conf` sets the threshold for the first association stage: `track_high_thresh` for ByteTrack and BoT-SORT, `det_thresh` for OC-SORT and Deep OC-SORT. It is not `predict()`'s `conf`, and for ByteTrack, BoT-SORT and OC-SORT the detector runs at a lower threshold internally so weak detections stay available for the recovery pass. Deep OC-SORT runs the detector at `det_thresh` itself. For ByteTrack and BoT-SORT, `track_conf` must be at or above `track_low_thresh`, which defaults to 0.1. Tracker settings arrive in one of two ways. Pass a config instance to `tracker_config=`, and its type selects the tracker, making `tracker=` redundant. Or pass the fields as keyword arguments and let `track()` build the config for the tracker you named; unknown keys warn rather than being applied silently. Either way, `track_conf` is ignored once the matching key is set explicitly. The remaining arguments mirror prediction: `iou`, `imgsz`, `classes`, `max_det`, `vid_stride`, `show`, and `save` with `output_path`. The source is a video file path. See [prediction](/docs/predict) for result handling. ## Train Trackers are not trained. Three of the four are pure motion models with no learned parameters at all, and Deep OC-SORT's appearance network is a published re-identification checkpoint that downloads on first use. Improving tracking quality means improving the detector, or tuning the association thresholds above. --- # OCR OCR locates text in an image and reads it. LibreYOLO exposes it as the ocr task, which returns one four-point polygon plus one transcript per text region, in reading order. Verified against LibreYOLO v1.5.0. ## Definition The `ocr` task does two things in one call: it locates every text region in an image and transcribes it. Regions come back as four-point polygons rather than axis-aligned boxes, because scene text is often rotated, and in reading order, top to bottom then left to right. A prediction fills `result.ocr`, an `OCRRegions` payload. `.data` is an `(N, 4, 2)` float array of polygons in original-image pixels, ordered top-left, top-right, bottom-right, bottom-left; `.texts` is the list of N transcripts; `.conf` is the per-region recognition score and `.det_conf` the detection score; `.xyxy` gives the axis-aligned hull of each polygon. Because the quads are genuine polygons, they do not populate `result.boxes`. Slicing an `OCRRegions` carries the transcripts and both score arrays along with the geometry. ## Models Two families serve `ocr`. [PP-OCRv5](/docs/models/pp-ocrv5) is the dedicated pipeline: a differentiable-binarization detector finds the text quads and an SVTR/CTC recognizer reads them, with both stages bundled into one `.pt` file along with the recognition charset. It ships in two tiers, a lighter one for CPU and a server one for higher accuracy, and one dictionary covers Simplified and Traditional Chinese, English, Japanese and pinyin. [SenseNova-Vision](/docs/models/sensenova-vision) reaches OCR by generating the words as tagged text from the same 7B checkpoint that serves its six other tasks, loaded with `LibreVLM("sensenova-vision", task="ocr")`. It needs the `sensenova` extra, and its weights are restricted to non-commercial use; the license is on its page. ## Predict Weights download from Hugging Face on first use and are cached locally. **Read the text in an image** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The t tier is the lighter of the two, built for CPU. SAMPLE_IMAGE # keeps this runnable; point it at an image with text of your own. model = LibreYOLO("LibrePPOCRt-ocr.pt") result = model(SAMPLE_IMAGE) regions = result.ocr print(len(regions), "regions") for text, score in zip(regions.texts, regions.conf): print(repr(text), float(score)) ``` **Read the quads** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibrePPOCRt-ocr.pt") result = model(SAMPLE_IMAGE) regions = result.ocr print(regions.data.shape) # (N, 4, 2) polygons, TL TR BR BL print(regions.xyxy) # axis-aligned hulls of those polygons print(regions.det_conf) # detection score, separate from .conf ``` **Filter by recognition confidence** ```python import numpy as np from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibrePPOCRt-ocr.pt") result = model(SAMPLE_IMAGE) # Index with positions, not a boolean mask: slicing carries the # transcripts and both score arrays along with the geometry. regions = result.ocr.numpy() keep = regions[np.flatnonzero(regions.conf >= 0.9)] print(keep.texts) ``` PP-OCRv5 runs detection at a fixed long-side limit and then recognizes the cropped regions in batches, with `rec_batch` controlling how many crops go through the recognizer per forward pass. Multi-image sources run sequentially, because a two-stage pipeline does not batch across images. See [prediction](/docs/predict) for sources, streaming and result handling. ## Dataset format OCR labels are one JSONL file per split, one JSON object per image, beside the images themselves. ```text my-ocr-dataset/ images/ val/receipt.jpg labels/ val.jsonl ``` Each line names an image and lists its regions: ```json {"image": "receipt.jpg", "regions": [{"polygon": [[10, 12], [118, 14], [117, 40], [9, 38]], "text": "TOTAL 12.50"}]} ``` `polygon` is a four-point quad in absolute pixel coordinates, ordered top-left, top-right, bottom-right, bottom-left. A region whose text cannot be read is labeled `"text": "###"`, the ICDAR don't-care convention: it is excluded from recognition scoring, and a prediction overlapping it is ignored rather than counted as a false positive. Passing the root directory as `data=` is enough. A dataset YAML is the alternative, with `path` plus optional `images` and `labels` directory names, and `nc: 1` with `names: {0: text}` as schema placeholders, since an OCR model returns `Results.ocr` rather than detections. See [dataset formats](/docs/reference/dataset-formats) for the full contract. ## Train Neither OCR family has a training implementation: `train()` raises `NotImplementedError` on both, and OCR support covers prediction and validation only. PP-OCRv5's page names the Apache-2.0 upstream training code and the conversion script that brings a fine-tuned checkpoint back into LibreYOLO. ## Validate `val()` scores the whole pipeline, detection and recognition together, matching predicted polygons to ground-truth polygons one-to-one at IoU above 0.5. **Validate and read the metric keys** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibrePPOCRt-ocr.pt") metrics = model.val(data="my-ocr-dataset") print(metrics["metrics/det_precision"], metrics["metrics/det_recall"]) print(metrics["metrics/det_hmean"]) print(metrics["metrics/e2e_f1"]) # fitness print(metrics["metrics/rec_1-NED"]) ``` `metrics/det_precision`, `metrics/det_recall` and `metrics/det_hmean` score localization alone: a match needs only the polygon overlap, whatever the transcript says. `metrics/e2e_precision`, `metrics/e2e_recall` and `metrics/e2e_f1` add the reading: a match needs the same polygon overlap and an exact transcript match after NFKC normalization and whitespace removal, and comparison stays case-sensitive. `metrics/e2e_f1` is also `fitness`, the number best-checkpoint selection reads. `metrics/rec_1-NED` grades the recognizer on its own, over the pairs detection already matched: one minus the normalized edit distance, so a transcript off by a character scores near 1 where end-to-end F1 scores it 0. ## Export No export format is available for this task. PP-OCRv5 is two networks moving together rather than one traceable graph, and `export()` raises for every format on both families. To deploy outside LibreYOLO, fine-tune upstream and use the upstream deployment path. --- # Open-vocabulary detection Open-vocabulary detection replaces a checkpoint's fixed class list with words you choose at call time. In LibreYOLO it is not a separate task: it is the detect task served by a separate model tier, loaded through the LibreOpenVocab factory instead of LibreYOLO. Verified against LibreYOLO v1.5.0. ## Definition Open-vocabulary detection returns ordinary detection `Results`: boxes, confidences and class indices, with `result.names` mapping those indices back to the strings you asked for. What changes is where the class list comes from. A conventional detector is trained against a fixed set of categories and can never emit a category outside it. These models take the vocabulary as text at inference time, so `set_classes(["forklift", "safety cone"])` is enough to make those the classes. LibreYOLO has no `open-vocabulary` task key. These models declare `SUPPORTED_TASKS = ("detect",)` like any other detector. What separates them is the loading path: they are Hugging Face snapshots rather than LibreYOLO state-dict checkpoints, so they stay out of the `LibreYOLO()` factory and are constructed through `LibreOpenVocab()` instead. That factory is a sibling of `LibreSAM()` and `LibreVLM()`, not a replacement for `LibreYOLO()`. Scores are real detection scores, not a generated caption parsed after the fact. Each family scores image regions against the text embedding of every prompt. ## Models Four families make up the tier, all of them predict only. Load any of them by alias through `LibreOpenVocab`. [Grounding DINO](/docs/models/grounding-dino), from IDEA Research, in `t` and `b` sizes. It is the tier default, and the only family that accepts `text_threshold`, a second cutoff on the decoded phrase's token score. [OWLv2](/docs/models/owlv2), from Google Research, in `b16` and `l14` sizes. It scores image regions against text embeddings from a CLIP-style encoder. [OMDet-Turbo](/docs/models/omdet-turbo), from Om AI Lab, in one `t` size. It decouples class embeddings from a language task prompt, and is the one family here that suppresses overlapping boxes inside its own post-processing, so `iou=` is honored. [OV-DEIM](/docs/models/ov-deim), in `s`, `m` and `l` sizes, a DETR-style detector that matches decoder queries to text embeddings from a bundled MobileCLIP text tower. It is one-to-one matching with top-K selection, so no NMS runs anywhere. OV-DEIM's weights are the restricted case in this tier. The detector weights are CC BY-NC 4.0, non-commercial. The bundled text tower is under Apple's Machine Learning Research Model license, research use only. The `l` checkpoint adds a DINOv3-S backbone fine-tune under Meta's DINOv3 License. All three license texts ship inside the weight repository, and the library logs the same summary when it resolves the weights, before the model is built. Read [OV-DEIM](/docs/models/ov-deim) before deploying it. The tier needs one extra: ```bash pip install "libreyolo[openvocab]" ``` That covers `transformers` and `timm` for the three wrapped families, and the `huggingface_hub`, `safetensors`, `regex` and `ftfy` packages OV-DEIM needs as a native port. A second tier also takes a text vocabulary: `LibreVLM()` loads generative vision-language models, such as [Qwen3-VL](/docs/models/qwen3-vl) and [Florence-2](/docs/models/florence-2), and turns their output into the same `Results`. It shares the `set_classes()` surface. The difference is what produces the boxes: the families on this page are discriminative detectors that emit scores directly, while the VLM tier generates them. ## Predict **Python** ```python from libreyolo import LibreOpenVocab, SAMPLE_IMAGE model = LibreOpenVocab("grounding-dino-t") model.set_classes(["person", "dog", "skateboard"]) result = model.predict(SAMPLE_IMAGE, conf=0.25) print(result.names) for box in result.boxes: print(box.cls, box.conf, box.xyxy) ``` **Swap the vocabulary** ```python from libreyolo import LibreOpenVocab, SAMPLE_IMAGE model = LibreOpenVocab("owlv2-b16") # set_classes is sticky: it holds until the next call to it. # Labels must be unique once lowercased and stripped of articles. model.set_classes(["a red backpack", "traffic cone"]) result = model.predict(SAMPLE_IMAGE) model.set_classes(["bicycle wheel"]) result = model.predict(SAMPLE_IMAGE) ``` **Grounding DINO text threshold** ```python from libreyolo import LibreOpenVocab, SAMPLE_IMAGE model = LibreOpenVocab("grounding-dino-b") model.set_classes(["remote control", "school bus"]) # conf filters by box score, text_threshold by the decoded phrase's # token score. Both default to 0.25 when left unset. Only Grounding # DINO accepts text_threshold; the others raise. result = model.predict(SAMPLE_IMAGE, conf=0.25, text_threshold=0.3) ``` `set_classes()` takes a non-empty list of label strings and holds until it is called again. Labels must be unique once lowercased and stripped of leading articles, so `"a bus"` and `"bus"` cannot coexist in one vocabulary. Multi-word phrases are labels like any other, and each family turns the list into its own text input before tokenizing, so `"traffic cone"` is a different query from `"cone"`. Three prediction arguments behave differently here than on a native detector. `imgsz=` is rejected, because the processor owns resizing for these families. `augment=True` is rejected, since test-time augmentation is out of scope for the tier. `iou=` applies only to the family whose processor runs its own suppression; where nothing is suppressed, passing it warns and is ignored. Left unset, `conf` takes the loaded family's own default rather than `predict()`'s usual 0.25, and that default is not the same across the tier. Set it explicitly when comparing two families on the same image. `track()` raises across the tier. Run `predict()` per frame instead. See [prediction](/docs/predict) for sources, streaming and result handling. ## Train No family in this tier trains inside LibreYOLO. `train()` raises: fine-tune upstream and load the resulting weights. The vocabulary passed to `set_classes()` is the only setting that changes what a loaded model detects. ## Validate There is no validator for this tier, and `val()` raises. Open-vocabulary validation needs a dedicated one, because the standard detection validator feeds image tensors straight to the model, while these families require text-conditioned inputs built alongside them. ## Export Export is out of scope for the tier and `export()` raises. These models run through `predict()` in PyTorch. --- # Oriented detection Oriented object detection locates each instance with a rotated rectangle rather than an axis-aligned one, so a tilted object is bounded tightly instead of by a box full of background. The task key is obb. Verified against LibreYOLO v1.5.0. ## Definition Oriented detection adds one number to a detection: the angle. Each instance gets a rotated rectangle, a class and a score. The gain is tightness. A ship at 45 degrees, a warehouse roof, a row of parked trucks: an axis-aligned box around any of them is mostly background, and two neighboring boxes overlap even when the objects do not. That is why the task is standard in aerial imagery and document layout, and why the reference dataset for it is DOTA. `obb` is the canonical task key, and the `-obb` suffix in a checkpoint filename selects it, so `task=` is not needed when loading published weights. `predict()` fills `result.obb`. `.xywhr` is the canonical `(N, 5)` form: center x, center y, width, height, and an angle in radians giving the rotation of the width side around the center. `.conf` and `.cls` carry the score and the class index into `result.names`, and `.id` a track id when tracking. `.xyxyxyxy` converts each row to its four corner points as `(N, 4, 2)` pixels, `.xyxyxyxyn` normalizes those corners, and `.xyxy` gives the enclosing axis-aligned box, which is what to use when downstream code only understands rectangles. `result.boxes` is filled as well, with the axis-aligned form. ## Models Two families serve this task, and which one to reach for depends on whether you need to train. [RF-DETR](/docs/models/rf-detr) is the one that trains. It predicts, trains, validates and exports oriented boxes, and it ships published oriented checkpoints in four sizes, n, s, m and l. It needs its own extra, `pip install "libreyolo[rfdetr]"`, and its model page carries the weights license and the provenance. Read the section below on what those checkpoints actually predict before you plan around them. [RT-DETRv2](/docs/models/rt-detr) is the one with aerial weights. It publishes `LibreRTDETRv2n-obb.pt` through `LibreRTDETRv2x-obb.pt`, the official DOTA v1.0 single-scale checkpoints converted into LibreYOLO's format, covering DOTA's 15 classes at 1024 px. It needs no extra beyond the base package, the oriented graph is recognized from the checkpoint's own tensors, and prediction, validation and ONNX and TorchScript export are all supported. Training is not: the oriented task is inference only on that family, `train()` raises, and there is no transfer from its detection weights, which use a different backbone. Tracking and test-time augmentation are also unavailable for oriented boxes. So: DOTA categories out of the box, RT-DETRv2. Your own oriented labels, RF-DETR. ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python # Needs the rfdetr extra: pip install "libreyolo[rfdetr]" from libreyolo import LibreYOLO, SAMPLE_IMAGE # The -obb suffix in the filename selects the task, so no task # argument is needed. model = LibreYOLO("LibreRFDETRs-obb.pt") result = model(SAMPLE_IMAGE, save=True) obb = result.obb print(obb.xywhr) # (N, 5): center x, center y, width, height, radians print(obb.conf, obb.cls) ``` **CLI** ```bash libreyolo predict model=LibreRFDETRs-obb.pt save=True \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` **Corners instead of angles** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE result = LibreYOLO("LibreRFDETRs-obb.pt")(SAMPLE_IMAGE) obb = result.obb print(obb.xyxyxyxy.shape) # (N, 4, 2) corner points in pixels print(obb.xyxyxyxyn.shape) # the same, normalized print(obb.xyxy.shape) # (N, 4) enclosing axis-aligned box ``` **A smaller checkpoint** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreRFDETRn-obb.pt") result = model(SAMPLE_IMAGE) print(result.obb.xywhr.shape) ``` **RT-DETRv2** ```python from libreyolo import LibreYOLO # DOTA v1.0 weights, 15 aerial classes at 1024 px. The oriented graph # is recognized from the checkpoint's own tensors, so no task argument. model = LibreYOLO("LibreRTDETRv2n-obb.pt") result = model("aerial.png", save=True) obb = result.obb print(obb.xywhr) print(result.names) # plane, ship, harbor, helicopter, and 11 more ``` Know what RF-DETR's published checkpoints are before you run them. Despite DOTA being the reference benchmark for this task, those weights were not trained on it. All four were initialized from the RF-DETR detection weights and fine-tuned on a single Roboflow Universe dataset of UAV footage, with six vehicle classes: bike, bus, car, other_vehicle, taxi and truck. Their model cards describe them as development weights, produced while validating oriented training support, and say they should not be read as production or benchmark-official weights. In practice that means they are a working starting point for oriented boxes on vehicles seen from above, and for verifying that your pipeline runs end to end. Any other domain means training on your own oriented labels, and for the aerial categories DOTA is known for, the RT-DETRv2 checkpoints are the ones actually trained on that data. `conf` and `max_det` shape the output as they do for detection. See [prediction](/docs/predict) for sources, streaming and result handling. ## Dataset format The layout is the detection layout: one `.txt` label file per image, found by swapping `images` for `labels` in the image path and changing the extension. ```text dataset/ data.yaml images/ train/P0001.png val/P0101.png labels/ train/P0001.txt val/P0101.txt ``` A row is exactly nine fields, a class index followed by four corner points in order: ```text ``` The four points are normalized floats in `[0, 1]` and have to form a non-degenerate oriented rectangle. No angle is stored in the label file: the loader derives the canonical `xywhr` from the corners. The parser is strict by default and rejects out-of-range coordinates, while dataset and validation ingestion may first clip to `[0, 1]` for otherwise valid crop-boundary labels, then still reject degenerate boxes. Row parsing is task-aware. Nine fields mean an oriented box only in `obb` mode; in `segment` mode the same row is read as a four-point polygon. The YAML is the detection YAML: ```yaml path: dataset train: images/train val: images/val names: 0: plane 1: ship ``` Native COCO JSON loads too, with an `annotations` mapping of split name to JSON file. Annotations are read in priority order: an `obb` field of eight pixel-space corners, an `obb` field of `[cx, cy, w, h, angle]` with the angle in radians, a `segmentation` polygon or RLE refit to its minimum-area rectangle, or a plain COCO `bbox`, which is treated as an axis-aligned rectangle and canonicalized to `xywhr`. The canonical row parser is `libreyolo.data.parse_yolo_obb_label_line`. ## Train **Python** ```python from libreyolo import LibreYOLO # Continues from published oriented weights. data must point at a # dataset whose label rows carry four corners. model = LibreYOLO("LibreRFDETRs-obb.pt") model.train(data="my-obb-dataset.yaml", epochs=50, imgsz=512, batch=8, lr0=1e-4) ``` **CLI** ```bash libreyolo train model=LibreRFDETRs-obb.pt data=my-obb-dataset.yaml \ epochs=50 imgsz=512 batch=8 lr0=1e-4 ``` **From detection weights** ```bash # Detection weights carry no angle prediction, so this is an explicit # transfer. Asking for task=obb is what authorizes it. libreyolo train model=LibreRFDETRs.pt data=my-obb-dataset.yaml \ task=obb epochs=50 imgsz=512 ``` Training on this task means RF-DETR. Training continues from a published `-obb` checkpoint by default. Starting from detection weights is a deliberate transfer: those weights predict no angle, and passing `task=obb` is what authorizes the swap. Keep `lr0` at or below `1e-4`, as with the family's other tasks. RT-DETRv2's oriented checkpoints cannot be fine-tuned; use them as they are, or train an RF-DETR model on your own labels. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a plain dictionary of `metrics/` keys. Matching uses rotated IoU, computed between oriented rectangles rather than between their enclosing axis-aligned boxes, so a prediction with the right position and the wrong angle scores as a miss. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRFDETRs-obb.pt") # val() returns a plain dict, not an object. metrics = model.val(data="my-obb-dataset.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"], metrics["metrics/mAP75"]) print(metrics["metrics/precision"], metrics["metrics/recall"]) ``` **CLI** ```bash libreyolo val model=LibreRFDETRs-obb.pt data=my-obb-dataset.yaml ``` **RT-DETRv2** ```bash libreyolo val model=LibreRTDETRv2n-obb.pt data=my-obb-dataset.yaml ``` `metrics/mAP50-95` is mean average precision averaged over IoU thresholds 0.50 to 0.95 in steps of 0.05, and it is the headline number. Unlike the COCO path used by detection, this task honors `iou_thresholds` in the validation config, so the sweep can be changed. `metrics/mAP50` and `metrics/mAP75` are the single-threshold versions. `metrics/precision` and `metrics/recall` are real precision and recall at IoU 0.50, read at the loosest operating point: every prediction that survived the confidence threshold is counted, and that threshold defaults to 0.001 during validation. Raising `conf` therefore moves them, while the mAP figures, which use the whole precision-recall curve, stay put. Four of these repeat under an `(OBB)` suffix, `metrics/mAP50-95(OBB)`, `metrics/mAP50(OBB)`, `metrics/precision(OBB)` and `metrics/recall(OBB)`, which is how a caller tells an oriented result from an axis-aligned one when both sit in the same table. `metrics/mAP75` has no suffixed twin. Two options do nothing on this task. `save_json` and `save_plots` are accepted and log a warning: oriented prediction dumps and validation plots are not implemented. ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRFDETRs-obb.pt") model.export(format="onnx", imgsz=512) ``` **CLI** ```bash libreyolo export model=LibreRFDETRs-obb.pt format=onnx imgsz=512 ``` **RT-DETRv2** ```bash # ONNX and TorchScript are the validated targets here, at FP32, # batch 1, on a fixed 1024 by 1024 canvas. libreyolo export model=LibreRTDETRv2n-obb.pt format=onnx imgsz=1024 libreyolo export model=LibreRTDETRv2n-obb.pt format=torchscript imgsz=1024 ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like a checkpoint and returns the same Results object. model = LibreYOLO("LibreRFDETRs-obb.onnx") result = model(SAMPLE_IMAGE) print(result.obb.xywhr) ``` An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Format coverage differs by task on the same family, and the matrix on the model page is generated from the validated set and names the reason a target is unavailable. See [export and deploy](/docs/export) for the formats, their extras and their constraints. --- # Panoptic segmentation Panoptic segmentation assigns every pixel to exactly one non-overlapping segment, unifying countable object instances with amorphous background regions. The task key is panoptic. Verified against LibreYOLO v1.5.0. ## Definition Panoptic segmentation is the union of the other two segmentation tasks. Every pixel gets exactly one segment, segments never overlap, and a segment is either a thing, a countable object instance, or stuff, an amorphous region such as sky or road. That makes it stricter than [instance segmentation](/docs/tasks/instance-segmentation), which leaves background pixels unassigned and lets masks overlap, and stricter than [semantic segmentation](/docs/tasks/semantic-segmentation), which labels every pixel but merges touching instances of one class. `panoptic` is the canonical task key, and the `-panoptic` suffix in a checkpoint filename selects it, so `task=` is not needed when loading published weights. `predict()` fills `result.panoptic`. `.data` is an `(H, W)` integer segment-id map on the original image canvas. `.segments_info` is a list of dicts, one per segment, each carrying at least `{"id", "category_id"}`, where `id` matches a value in the map and `category_id` indexes `result.names`. `.segment_ids` lists the ids present in sorted order and `.segment_mask(id)` returns the boolean `(H, W)` selection for one segment. Segment id `0` is the void value: unlabeled pixels, excluded from the metric and left out of `.segment_ids`. Thing versus stuff is a property of the category, not of the individual segment. It is carried on the label set's category metadata, and a prediction payload may copy it onto each segment as `"isthing"` for convenience, but the category metadata remains authoritative. ## Models [EoMT](/docs/models/eomt) is the family that serves this task through `LibreYOLO()`. It runs on the base package and ships panoptic checkpoints in three sizes, s, b and l, trained on COCO. [SenseNova-Vision](/docs/models/sensenova-vision) also emits panoptic maps. It is a prompted generative model with its own factory, `LibreVLM`, and its own extra; with no vocabulary set it falls back to the COCO panoptic categories it was tuned on. Its weights are non-commercial. Per-image latency is far higher than a purpose-built segmenter, because every prediction is a diffusion decode. ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The -panoptic suffix in the filename selects the task, so no task # argument is needed. model = LibreYOLO("LibreEoMTl-panoptic.pt") result = model(SAMPLE_IMAGE, save=True) pan = result.panoptic print(pan.data.shape) # (H, W) segment ids print(pan.segments_info) # [{"id": ..., "category_id": ...}, ...] ``` **CLI** ```bash libreyolo predict model=LibreEoMTl-panoptic.pt save=True \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` **One segment at a time** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE result = LibreYOLO("LibreEoMTl-panoptic.pt")(SAMPLE_IMAGE) pan = result.panoptic for segment in pan.segments_info: pixels = pan.segment_mask(segment["id"]) # boolean (H, W) print(result.names[segment["category_id"]], int(pixels.sum())) ``` **A smaller checkpoint** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreEoMTs-panoptic.pt") result = model(SAMPLE_IMAGE) print(len(result.panoptic.segment_ids)) ``` `conf` filters query selection. See [prediction](/docs/predict) for sources, streaming and result handling. ## Dataset format LibreYOLO adopts the COCO-panoptic format verbatim, from Kirillov et al., CVPR 2019. There is no LibreYOLO-specific panoptic layout. ```text dataset/ data.yaml images/ val/000000000139.jpg annotations/ panoptic_val.json panoptic_val/000000000139.png ``` Each image is paired with one RGB PNG at the same resolution, where each pixel's color encodes the id of the segment it belongs to: ```text segment_id = R + 256 * G + 256 * 256 * B ``` Segment id `0`, RGB black, is void: unlabeled pixels that neither reward nor penalize a prediction. Every other pixel belongs to exactly one segment. The JSON lists, per image, the segment-id PNG and the segments inside it: ```json { "images": [{"id": 139, "file_name": "000000000139.jpg"}], "annotations": [{"image_id": 139, "file_name": "000000000139.png", "segments_info": [ {"id": 3226956, "category_id": 1, "area": 2840, "bbox": [413, 158, 53, 138], "iscrowd": 0}]}], "categories": [{"id": 1, "name": "person", "isthing": 1}] } ``` `annotations[].file_name` names the PNG inside the panoptic directory, and `segments_info[].id` matches a value in that PNG. `iscrowd` marks group regions: they are never counted as false negatives, and a prediction that mostly covers one is not a false positive. `isthing` lives on `categories` and never on an individual segment. The YAML points at both: ```yaml path: dataset val: images/val annotations: val: annotations/panoptic_val.json panoptic_dir: val: annotations/panoptic_val names: 0: person 1: bicycle ``` `annotations` and `panoptic_dir` each accept a single path or a per-split mapping. Raw COCO category ids are typically non-contiguous, while models predict a contiguous `0..nc-1`, so ids are remapped through `names` by category name. A JSON category missing from `names` is an error rather than a silent drop, because dropping it would score as a permanent false negative. The canonical loader is `libreyolo.data.PanopticDataset`. ## Train No family trains panoptic segmentation in LibreYOLO today: EoMT's `train()` raises `NotImplementedError`, so panoptic checkpoints are used as published. ## Validate `val()` returns a plain dictionary of `metrics/` keys, computed at the ground truth resolution over the split named by `val` in the dataset YAML. A predicted and a true segment of the same category match when their IoU exceeds 0.5, and that match is unique. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreEoMTl-panoptic.pt") # val() returns a plain dict, not an object. metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/PQ"]) print(metrics["metrics/SQ"], metrics["metrics/RQ"]) print(metrics["metrics/PQ_things"], metrics["metrics/PQ_stuff"]) ``` **CLI** ```bash libreyolo val model=LibreEoMTl-panoptic.pt data=my-dataset.yaml ``` `metrics/PQ` is Panoptic Quality, the headline number. Within one category it is the product of two factors. Segmentation quality is the mean IoU over matched segments and says how well the matched shapes line up. Recognition quality is `TP / (TP + 0.5 FP + 0.5 FN)`, the F1 score of the matching itself, and says how many segments were found at all. All three figures are then averaged over the categories that appeared, and reported as `metrics/PQ`, `metrics/SQ` and `metrics/RQ`, so the reported PQ is the mean of per-category products rather than the product of the two reported means. `metrics/PQ_things` and `metrics/PQ_stuff` average the same per-category PQ over thing categories and stuff categories separately, and `metrics/categories` counts the categories that appeared and were therefore averaged over. The dictionary also carries `fitness`, a copy of the PQ value. ## Export Panoptic checkpoints do not export. `export()` raises `NotImplementedError` for this task, because the query-mask output has no runtime export contract yet. EoMT's semantic task does export; see [semantic segmentation](/docs/tasks/semantic-segmentation) and [export and deploy](/docs/export). --- # Point detection Point detection returns one x, y location per object instead of a bounding box. LibreYOLO exposes it as the point task, and a prediction carries one row of x, y, class and confidence per object. Verified against LibreYOLO v1.5.0. ## Definition The `point` task locates each object with a single x, y coordinate and a class, with no width, height or mask. Because a prediction is a flat list of objects, the row count is the object count, which is what makes this the counting task. A prediction fills `result.points`, a `Points` payload wrapping an `(N, 4)` array of `x, y, class, confidence` rows in original-image pixels. `.xy` returns the coordinates, `.xyn` the same coordinates divided by the image size, `.cls` the class indices and `.conf` the scores; `len()` returns the number of points. `result.boxes` stays empty, so `iou` and `max_det` have nothing to act on. ## Models Three families serve `point`, and they are not interchangeable. [FOMO](/docs/models/fomo) is the fixed-vocabulary option: a grid classifier that labels each cell of a low-resolution grid as background or an object center. It is the only point family LibreYOLO can train, and the only one that exports. [LocateAnything](/docs/models/locate-anything) takes text instead of a class index, so the vocabulary is whatever phrase you write. It needs the `vlm` extra, is constructed as `LibreLocateAnything` rather than through the `LibreYOLO()` factory, and its weights are restricted to non-commercial use. The exact terms, and the two further licenses the checkpoint composes, are on its page. [SenseNova-Vision](/docs/models/sensenova-vision) reaches `point` through the same prompted-generation checkpoint it uses for six other tasks, loaded with `LibreVLM("sensenova-vision", task="point")`. It needs the `sensenova` extra, and every prediction is a generation pass over a 7B model, so expect noticeably higher per-image latency than a purpose-built detector. Its weights are non-commercial; the license is on its page. ## Predict LibreFOMO weights are the one exception to automatic download on this site. `LibreYOLO("LibreFOMOs-point.pt")` looks for that file on disk and raises a `ValueError` naming it rather than fetching it. Download a checkpoint from the [LibreYOLO organization](https://huggingface.co/LibreYOLO) on Hugging Face first and load it by local path, or train your own. **Predict points and count them** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # LibreFOMO weights are not auto-downloaded. Fetch a checkpoint from # https://huggingface.co/LibreYOLO first and load it by local path. model = LibreYOLO("./LibreFOMOs-point.pt") result = model(SAMPLE_IMAGE, save=True) points = result.points print(len(points)) # object count print(points.xy) # (N, 2) centers in original-image pixels print(points.cls, points.conf) ``` **Normalized coordinates and per-class counts** ```python from collections import Counter from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("./LibreFOMOs-point.pt") result = model(SAMPLE_IMAGE) points = result.points.numpy() print(points.xyn) # same centers in [0, 1] print(Counter(points.cls.astype(int).tolist())) ``` The filename has to carry the `-point` task suffix for the loader to recognize it. `predict(..., nms_radius=1)` controls how many grid cells apart two FOMO detections must be to both survive. See [prediction](/docs/predict) for sources, streaming and result handling. ## Dataset format `point` has no label format of its own. The point families read the standard YOLO detection layout and derive one center from each box row, so `cx cy` is the point and `w h` only decide whether the row is valid. ```text dataset/ data.yaml images/ train/scene.jpg val/scene.jpg labels/ train/scene.txt val/scene.txt ``` Each label file holds one row per object, with normalized coordinates: ```text ``` ```yaml path: dataset train: images/train val: images/val nc: 1 names: {0: seedling} ``` A missing or empty label file means no objects. See [dataset formats](/docs/reference/dataset-formats) for the full contract. ## Train FOMO is the only point family with a training implementation. `train()` on LocateAnything and on SenseNova-Vision raises `NotImplementedError`; fine-tune those upstream and load the result. **Train FOMO on a YOLO dataset** ```python from libreyolo import LibreYOLO model = LibreYOLO("./LibreFOMOs-point.pt") model.train(data="my-dataset.yaml", epochs=40, batch=32, lr0=3e-4) ``` **Predict with the trained checkpoint** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("./LibreFOMOs-point.pt") results = model.train(data="my-dataset.yaml", epochs=40) # train() reloads the best checkpoint into the same object, so the # model predicts with the trained weights when the call returns. print(results["best_checkpoint"]) print(model(SAMPLE_IMAGE).points.xy) ``` `imgsz` is not a free choice for FOMO: it defaults to the loaded checkpoint's native resolution, and passing a different value raises `ValueError` naming the size it expects. See [training](/docs/train) for datasets, loggers and multi-GPU, and the [FOMO page](/docs/models/fomo) for this family's defaults. ## Validate `val()` matches predicted points to ground-truth points one-to-one with the Hungarian algorithm, over a sweep of distance thresholds. A threshold is a Euclidean distance in normalized image coordinates, and the default sweep is ten values from 0.01 to 0.10. **Validate and read the metric keys** ```python from libreyolo import LibreYOLO model = LibreYOLO("./LibreFOMOs-point.pt") metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/precision"], metrics["metrics/recall"]) print(metrics["metrics/f1"]) print(metrics["metrics/mAP@[0.01:0.10]"]) # fitness print(metrics["metrics/MLE"]) # mean localization error print(metrics["metrics/MAE"], metrics["metrics/RMSE"]) # count error ``` **Change the distance thresholds** ```python from libreyolo import LibreYOLO model = LibreYOLO("./LibreFOMOs-point.pt") # The sweep bounds are part of the key text, so a custom sweep # renames the mAP keys it produces. metrics = model.val(data="my-dataset.yaml", dist_thresholds=[0.02, 0.05]) print(metrics["metrics/mAP@0.02"]) print(metrics["metrics/mAP@[0.02:0.05]"]) ``` `metrics/precision`, `metrics/recall` and `metrics/f1` are macro-averaged over classes at the strictest threshold in the sweep, 0.01 by default. `metrics/mAP@0.01` is average precision at that same threshold, and `metrics/mAP@[0.01:0.10]` is the mean over the whole sweep. That sweep value is also `fitness`, the number best-checkpoint selection reads. Both mAP keys are built from the thresholds in use, so passing `dist_thresholds=` renames them. `metrics/MLE` is the mean distance between matched pairs at the strictest threshold, in the same normalized units. `metrics/MAE` and `metrics/RMSE` are counting metrics rather than localization ones: they measure the per-image difference between the number of predicted and ground-truth points. FOMO adds a second, grid-level group on top of these. It sweeps confidence and `nms_radius` and publishes the best-F1 combination as `metrics/grid_F1`, `metrics/grid_precision`, `metrics/grid_recall`, `metrics/grid_mean_distance`, `metrics/grid_TP`, `metrics/grid_FP` and `metrics/grid_FN`, with the settings that produced it under `decode/threshold` and `decode/nms_radius`. ## Export FOMO exports through the shared export path, and an exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. **Export** ```python from libreyolo import LibreYOLO model = LibreYOLO("./LibreFOMOs-point.pt") model.export(format="onnx") ``` **Run the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("./LibreFOMOs-point.onnx") result = model(SAMPLE_IMAGE) print(result.points.xy) ``` Per-format coverage is on the [FOMO page](/docs/models/fomo) and in the [full export matrix](/docs/reference/export-matrix). LocateAnything and SenseNova-Vision do not export: `export()` raises on both, because a generative model has no traceable detection graph. --- # Pose estimation Pose estimation locates each instance and returns an ordered set of named keypoints for it, so the output carries the object's internal structure rather than only its extent. The task key is pose. Verified against LibreYOLO v1.5.0. ## Definition Pose estimation returns structure, not just extent. Each instance still gets a box, a class and a score, and it also gets `K` keypoints in a fixed order, so index 5 means the same body part on every instance and in every image. The label set defines that order; nothing in the output identifies a keypoint by name. `pose` is the canonical task key, and the `-pose` suffix in a checkpoint filename selects it, so `task=` is not needed when loading published weights. `predict()` fills `result.keypoints` alongside `result.boxes`. `.data` is `(N, K, 2)` or `(N, K, 3)`, row-aligned with the boxes, so instance `i` in one is instance `i` in the other. `.xy` slices the pixel coordinates and `.xyn` normalizes them by the original image size. `.conf` is the third column when the checkpoint predicts one and `None` when it does not, and `.has_visible` is the boolean mask derived from it, all-true when there is no third column. Two architectures reach this output. A one-stage model predicts boxes and keypoints in a single pass. A top-down model runs a detector first, crops each instance and regresses keypoints inside the crop, so its accuracy depends on the detector in front of it. ## Models Three families both train and predict: [RF-DETR](/docs/models/rf-detr), [EdgeCrafter](/docs/models/edgecrafter) and [YOLO-NAS](/docs/models/yolo-nas), all one-stage. RF-DETR needs its own extra, `pip install "libreyolo[rfdetr]"`. RF-DETR and EdgeCrafter ship published pose checkpoints and both fine-tune on single-class, person-only datasets; EdgeCrafter's keypoint head is fixed at construction and rejects a dataset declaring a different count, while RF-DETR reinitializes its head for one. YOLO-NAS pulls its weights from Deci.AI's own CDN under a non-commercial license, and LibreYOLO publishes none of them; its pose head also rebuilds for a new keypoint count, and it is the only one of the three whose class count is not fixed at one, so it is the family for a multi-class or non-human skeleton, such as animal pose. [HRNet](/docs/models/hrnet) is the top-down option. It predicts, validates and exports, and its `train()` raises `NotImplementedError`. Given no person source, it pairs itself with a LibreYOLO9t detector automatically; `cropped=True` treats the whole image as one instance, `person_boxes=` takes boxes you already have, and `person_detector=` names a different detector. [SenseNova-Vision](/docs/models/sensenova-vision) also emits keypoints. It is a prompted generative model with its own factory, `LibreVLM`, and its own extra; with no vocabulary set, `set_task("pose")` falls back to the person category. Its weights are non-commercial, and per-image latency is far higher than a purpose-built pose head, because every prediction is a diffusion decode. ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The -pose suffix in the filename selects the keypoint head, so no # task argument is needed. model = LibreYOLO("LibreECs-pose.pt") result = model(SAMPLE_IMAGE, save=True) print(result.keypoints.xy.shape) # (N, K, 2) pixel coordinates print(result.boxes.xyxy.shape) # (N, 4), the same N instances ``` **CLI** ```bash libreyolo predict model=LibreECs-pose.pt save=True \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` **Visible keypoints only** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE result = LibreYOLO("LibreECs-pose.pt")(SAMPLE_IMAGE) kpts = result.keypoints # .has_visible is derived from the third keypoint column, and is # all-true when the checkpoint predicts only (x, y). for person, visible in zip(kpts.xy, kpts.has_visible): print(person[visible]) ``` **Top-down instead** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # HRNet is top-down: it crops each person first. With no person source # given it pairs itself with a LibreYOLO9t detector and logs the choice. model = LibreYOLO("LibreHRNetw32-pose.pt") result = model(SAMPLE_IMAGE) print(result.keypoints.xy.shape) ``` Keypoint counts and orders are properties of the checkpoint, not of the library, so a model trained on a different skeleton returns a different `K` and a different meaning per index. What the third keypoint column holds is also a checkpoint property: EdgeCrafter writes a constant there rather than a per-point score, and it has no box head at all, so each of its pose boxes is the bounding extent of that instance's own keypoints. See [prediction](/docs/predict) for sources, streaming and result handling. ## Dataset format The layout is the detection layout: one `.txt` label file per image, found by swapping `images` for `labels` in the image path and changing the extension. ```text dataset/ data.yaml images/ train/000001.jpg val/000101.jpg labels/ train/000001.txt val/000101.txt ``` A row is a detection row with the keypoints appended: ```text [] ... [] ``` The field count is exactly `5 + K * D`, where `D` is the second value of `kpt_shape`. Box and keypoint coordinates are normalized floats relative to the original image width and height. Visibility `v`, present only when `D` is 3, is `0`, `1` or `2`. The YAML adds two keys to the shared contract: ```yaml path: dataset train: images/train val: images/val kpt_shape: [17, 3] flip_idx: [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15] names: 0: person ``` `kpt_shape` is required and is `[K, 2]` or `[K, 3]`. `flip_idx` is optional and is a permutation of `0..K-1` giving, for each keypoint, the index it takes after a horizontal flip, which is how a left wrist stays a left wrist. Omit it and horizontal flip augmentation is switched off for keypoints rather than applied with the wrong index order. ## Train **Python** ```python from libreyolo import LibreYOLO # coco8-pose.yaml carries an embedded download script, so it needs # explicit permission unless the data is already local. model = LibreYOLO("LibreECs-pose.pt") model.train( data="coco8-pose.yaml", epochs=50, imgsz=640, batch=4, allow_download_scripts=True, ) ``` **CLI** ```bash libreyolo train model=LibreECs-pose.pt data=coco8-pose.yaml \ epochs=50 imgsz=640 batch=4 allow_download_scripts=True ``` **Your own dataset** ```python from libreyolo import LibreYOLO # data.yaml must declare kpt_shape, and the label rows must carry # exactly 5 + K * D fields. model = LibreYOLO("LibreECs-pose.pt") model.train(data="my-pose-dataset.yaml", epochs=50, imgsz=640, batch=8) ``` Training continues from a published `-pose` checkpoint, which already carries the keypoint head; the task is read from the checkpoint you load, not a flag passed at train time, so a detection checkpoint does not become a pose run by asking for one. `kpt_shape` in your YAML has to match the head exactly for EdgeCrafter, since its head is fixed at construction, while RF-DETR and YOLO-NAS resize the head for a different count instead. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a plain dictionary of `metrics/` keys. Scoring is COCO keypoint evaluation over Object Keypoint Similarity, which weighs each keypoint's distance error by the instance scale and by a per-keypoint tolerance, so it plays the role IoU plays for boxes. It needs `pycocotools`, which is in the base install. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreECs-pose.pt") # val() returns a plain dict, not an object. metrics = model.val(data="coco8-pose.yaml", allow_download_scripts=True) print(metrics["metrics/keypoints_mAP50-95"]) print(metrics["metrics/keypoints_mAP50"], metrics["metrics/keypoints_mAP75"]) ``` **CLI** ```bash libreyolo val model=LibreECs-pose.pt data=coco8-pose.yaml \ allow_download_scripts=True ``` `metrics/keypoints_mAP50-95` is the headline number, mean average precision averaged over OKS thresholds 0.50 to 0.95, and it is what training uses to pick the best epoch. `metrics/keypoints_mAP50` and `metrics/keypoints_mAP75` are the single-threshold versions, and `metrics/keypoints_mAP_M` and `metrics/keypoints_mAP_L` split the average by instance area, medium and large; COCO keypoint evaluation defines no small bucket. The matching average recall figures are `metrics/keypoints_AR50-95`, `metrics/keypoints_AR50`, `metrics/keypoints_AR75`, `metrics/keypoints_AR_M` and `metrics/keypoints_AR_L`. Every key on this task is prefixed `keypoints_`, so the box `mAP` keys a detector returns do not appear. ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreECs-pose.pt") model.export(format="onnx", imgsz=640) ``` **CLI** ```bash libreyolo export model=LibreECs-pose.pt format=onnx imgsz=640 ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like a checkpoint and returns the same Results object. model = LibreYOLO("LibreECs-pose.onnx") result = model(SAMPLE_IMAGE) print(result.keypoints.xy) ``` An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Format coverage differs by family; the matrix on each model page is generated from the validated set rather than typed by hand. See [export and deploy](/docs/export) for the formats, their extras and their constraints. --- # Promptable segmentation Promptable segmentation turns a click into a mask: you point at an object, or draw a box around it, and the model returns its outline. In LibreYOLO it is not a separate task key but a model tier, loaded through the LibreSAM factory, whose results are ordinary segmentation Results. Verified against LibreYOLO v1.5.0. ## Definition Promptable segmentation takes an image plus a spatial prompt and returns the mask of whatever the prompt points at. Nothing is classified: there is no class list, and `result.boxes` holds tight boxes derived from the masks rather than detections in their own right. `result.masks` carries the mask data and `result.masks.xy` its polygons. The prompt is the interface. `points` is `[x, y]` pixel coordinates, one set per object, with `labels` marking each point positive (1, include this) or negative (0, exclude this). `bboxes` is `[x1, y1, x2, y2]`, one mask per box. Points and boxes can be combined, in which case they pair per object and must be the same length. Omitting every prompt runs the segment-everything path, a grid of points over the image. A single point is ambiguous by construction. Clicking a sleeve could mean the sleeve, the shirt or the person, so `multimask=True` returns all three whole-versus-part masks per prompt instead of the single best one. `conf` filters on the model's predicted IoU, a mask-quality score, not a detection confidence. LibreYOLO has no `promptable` task key. The tier registers as `segment`, the same key instance segmentation uses. What separates it is the call shape, which is why it has its own factory, `LibreSAM()`, a sibling of `LibreYOLO()`, `LibreOpenVocab()` and `LibreVLM()`. A single `predict(image)` signature cannot express the loop these models are built for: `set_image()` runs the image encoder once and caches the embeddings, every later `predict()` call with `source=None` pays only for prompt decoding, and `reset_image()` clears the cache. The image encoder is the dominant cost and runs once per image, so a second prompt on the same image skips it entirely. ## Models Six families load through `LibreSAM` by alias. [SAM](/docs/models/sam) is the default, in `base`, `large` and `huge` sizes, also spelled `b`, `l` and `h`. [SAM 2](/docs/models/sam-2), as `sam2-tiny`, `sam2-small`, `sam2-base-plus` and `sam2-large`. LibreYOLO supports its image path. [SAM 3](/docs/models/sam-3), as `sam3`, is the one family that accepts a text concept prompt: `text="yellow school bus"` returns every matching instance. Passing `text=` to any other family raises with a message naming SAM 3. Its weights come from Meta under the custom SAM License rather than LibreYOLO's MIT license, and the repository is gated: accept the terms on the model page and authenticate with `hf auth login` before the first download. Read [SAM 3](/docs/models/sam-3) before deploying it. [EdgeTAM](/docs/models/edgetam), as `edgetam`, is an on-device variant of SAM 2. LibreYOLO supports its image path. [MobileSAM](/docs/models/mobilesam), as `mobilesam`, replaces SAM's ViT-H encoder with a distilled TinyViT one. [PicoSAM3](/docs/models/picosam3), as `picosam3`, is a compact CNN for box-prompted regions on edge sensors. Box prompts are the whole contract here: points, text, mask, multimask and segment-everything all raise with a message pointing at SAM 2 or SAM 3. The tier's extra covers the four families that load through `transformers`: ```bash pip install "libreyolo[sam]" ``` MobileSAM and PicoSAM3 are native LibreYOLO ports and need no `transformers` install to run. ## Predict **Point and box prompts** ```python from libreyolo import LibreSAM, SAMPLE_IMAGE model = LibreSAM("base") # A point is [x, y] in pixels; labels are 1 positive, 0 negative. result = model.predict(SAMPLE_IMAGE, points=[640, 420], labels=[1]) print(result.masks.xy) # polygons print(result.boxes.xyxy) # tight boxes derived from the masks # A box prompt gives one mask per box. result = model.predict(SAMPLE_IMAGE, bboxes=[300, 200, 900, 700]) ``` **Encode once, prompt many times** ```python from libreyolo import LibreSAM, SAMPLE_IMAGE model = LibreSAM("base") # set_image runs the heavy image encoder once and caches it. model.set_image(SAMPLE_IMAGE) first = model.predict(points=[640, 420], labels=[1]) second = model.predict(bboxes=[300, 200, 900, 700]) model.reset_image() ``` **Segment everything** ```python from libreyolo import LibreSAM, SAMPLE_IMAGE model = LibreSAM("base") # No prompt means a grid of points over the whole image. The default # grid of 32 per side is ~1024 decoder passes, which is slow on CPU. result = model.predict(SAMPLE_IMAGE, points_per_side=8) print(len(result.masks)) ``` **Ambiguity masks** ```python from libreyolo import LibreSAM, SAMPLE_IMAGE model = LibreSAM("base") # One point can mean a sleeve, a shirt, or a person. multimask=True # returns all three whole-versus-part masks instead of the best one. result = model.predict( SAMPLE_IMAGE, points=[640, 420], labels=[1], multimask=True ) print(len(result.masks)) ``` `source` and `set_image()` are alternatives, not a sequence: pass an image to `predict()` for a one-shot call, or call `set_image()` first and then `predict(source=None)` for each prompt. Passing `device=` to `predict()` moves the model for that call and every later one, and invalidates any cached embeddings. Segment-everything is the expensive mode. `points_per_side` defaults to 32, which is roughly 1024 decoder passes over the image; lower it for anything interactive on CPU. In that mode `conf` applies the family's grid threshold when left unset, while in the prompted path an unset `conf` keeps every mask. Pass `conf=0.0` to disable filtering in either mode, and `max_det` to cap how many masks come back. Mask prompts are not supported in this version, and `masks=` raises rather than being ignored. `track()` also raises across the tier: these are image segmenters, so run `predict()` per frame. See [prediction](/docs/predict) for sources and result handling. ## Train No family in this tier trains inside LibreYOLO. `train()` raises: fine-tune upstream and load the resulting weights. ## Validate There is no validator for this tier, and `val()` raises. A promptable mask has no fixed class set to score against, so the usual detection and segmentation metrics have nothing to key on. Scoring a prompted mask means comparing it to a reference mask you supply yourself, against the prompts you care about. ## Export Export is out of scope for the tier as a whole and `export()` raises, with one exception. [PicoSAM3](/docs/models/picosam3) exports its raw 96x96 region CNN to ONNX as `roi_image -> mask_logits`; box cropping and the mask resize back to image coordinates stay in Python. Every other family runs through `predict()` in PyTorch. See [export](/docs/export) for the formats available elsewhere in the library. --- # Semantic segmentation Semantic segmentation assigns a class to every pixel of an image and draws no distinction between instances of the same class. The task key is semantic. Verified against LibreYOLO v1.5.0. ## Definition Semantic segmentation labels pixels, not objects. Every pixel receives one class id, and two cars touching in the image become one region of the car class with no boundary between them. Counting instances is [instance segmentation](/docs/tasks/instance-segmentation); labeling every pixel and separating instances at the same time is [panoptic segmentation](/docs/tasks/panoptic-segmentation). `semantic` is the canonical task key, and the `-sem` suffix in a checkpoint filename selects it, so `task=` is not needed when loading published weights. `predict()` fills `result.semantic_mask`. `.data` is an `(H, W)` integer class map on the original image canvas, `.classes` lists the ids present in sorted order, and `.class_mask(id)` returns the boolean `(H, W)` selection for one class. The value `255` is the ignore label: it is never a class, it is excluded from loss and metrics, and `.classes` leaves it out. ## Models Three families both train and predict: [SegFormer](/docs/models/segformer), [LingBot-Vision](/docs/models/lingbot-vision) and [DINOv2](/docs/models/dinov2). SegFormer and LingBot-Vision run on the base package and ship published weights. DINOv2 needs `pip install "libreyolo[rfdetr]"` and has no LibreYOLO-hosted checkpoint: it loads the upstream backbone and its dense head starts at random initialization, so it is a training starting point rather than a ready predictor. Four more predict, validate and export, but their `train()` raises `NotImplementedError`: [FCN](/docs/models/fcn), [DeepLabv3](/docs/models/deeplabv3), [PIDNet](/docs/models/pidnet) and [EoMT](/docs/models/eomt). Class sets differ by checkpoint, not by family. The published weights come from datasets whose label spaces have little in common, ADE20K's 150 classes against Cityscapes' 19 among them, so a checkpoint's `names` is what tells you what it can label, and two checkpoints are only comparable when they were trained on the same one. ## Predict Weights download from Hugging Face on first use and are cached locally. **Python** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The -sem suffix in the filename selects the task, so no task # argument is needed. model = LibreYOLO("LibreSegformerb0-sem.pt") result = model(SAMPLE_IMAGE, save=True) mask = result.semantic_mask print(mask.data.shape) # (H, W) class ids on the original canvas print(mask.classes) # sorted class ids present, ignoring 255 ``` **CLI** ```bash libreyolo predict model=LibreSegformerb0-sem.pt save=True \ source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg ``` **One class at a time** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE result = LibreYOLO("LibreSegformerb0-sem.pt")(SAMPLE_IMAGE) mask = result.semantic_mask for class_id in mask.classes: pixels = mask.class_mask(class_id) # boolean (H, W) print(result.names[class_id], int(pixels.sum())) ``` **Another family, same call** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibrePIDNets-sem.pt") result = model(SAMPLE_IMAGE) print(result.semantic_mask.data.shape) ``` The map is an argmax per pixel, so there is no NMS step and `iou` never has an effect. `conf` and `max_det` are accepted for API parity and do nothing on SegFormer, PIDNet and the other dense predictors; EoMT is the exception, where `conf` filters query selection. See [prediction](/docs/predict) for sources, streaming and result handling. ## Dataset format Each image is paired with a dense single-channel mask rather than a `.txt` label file, found by swapping `images` for the mask directory in the image path. ```text dataset/ data.yaml images/ train/000001.jpg val/000101.jpg masks/ train/000001.png val/000101.png ``` Masks are lossless single-channel images, normally PNG, and palette-mode PNGs are read as palette indices. Each pixel value is a class id in `0..nc-1`, the value `255` means ignore, and the mask resolution has to equal the paired image resolution. The YAML takes two keys on top of the shared contract: ```yaml path: dataset train: images/train val: images/val masks_dir: masks nc: 19 names: 0: road 1: sidewalk ``` `masks_dir` is the directory name substituted for `images`, defaulting to `masks`. `label_mapping` is an optional `{source_id: train_id}` remap applied to mask pixel values at load time, which is how a dataset numbered 1 to 150 becomes 0 to 149; any source value left unmapped becomes ignore, and every train id has to fall in `0..nc-1`. Leaving `masks_dir` out switches the loader to a fallback: masks are rasterized at load time from polygon labels resolved through the usual `images` to `labels` convention, and a `background` class is appended after the object classes, so `nc` grows by one. The canonical loader is `libreyolo.data.SemanticDataset`. ## Train **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreSegformerb0-sem.pt") model.train(data="my-dataset.yaml", epochs=160, imgsz=512, batch=8) ``` **CLI** ```bash libreyolo train model=LibreSegformerb0-sem.pt data=my-dataset.yaml \ epochs=160 imgsz=512 batch=8 ``` **On ADE20K** ```bash # ade20k.yaml carries an embedded download script for the ~1 GB # archive, so it needs explicit permission unless the data is local. libreyolo train model=LibreSegformerb0-sem.pt data=ade20k.yaml \ epochs=160 imgsz=512 batch=8 allow_download_scripts=True ``` `imgsz` is constrained here in a way it is not on a detector. Each family declares a divisor its input has to be a multiple of, set by its patch grid or output stride, and both training and validation raise a `ValueError` before the run starts when `imgsz` does not divide evenly. The divisor is 32 for SegFormer, 16 for LingBot-Vision and EoMT, 14 for DINOv2, and 8 for FCN and PIDNet. See [training](/docs/train) for datasets, augmentation, multi-GPU and loggers. ## Validate `val()` returns a plain dictionary of `metrics/` keys, computed over the split named by `val` in the dataset YAML. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreSegformerb0-sem.pt") # val() returns a plain dict, not an object. metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mIoU"]) print(metrics["metrics/pixel_accuracy"]) ``` **CLI** ```bash libreyolo val model=LibreSegformerb0-sem.pt data=my-dataset.yaml ``` `metrics/mIoU` is mean intersection over union: for each class, the overlap between predicted and true pixels divided by their union, averaged over classes. It is the headline number and the one used to pick the best epoch during training. `metrics/pixel_accuracy` is the share of pixels given the correct class, which a large background class can inflate, so mIoU is the figure to compare on. Pixels marked `255` count toward neither. The dictionary also carries `fitness`, a copy of the mIoU value. ## Export **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreSegformerb0-sem.pt") model.export(format="onnx", imgsz=512) ``` **CLI** ```bash libreyolo export model=LibreSegformerb0-sem.pt format=onnx imgsz=512 ``` **Use the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like a checkpoint and returns the same Results object. model = LibreYOLO("LibreSegformerb0-sem.onnx") result = model(SAMPLE_IMAGE) print(result.semantic_mask.data.shape) ``` An exported artifact loads back through `LibreYOLO()` on its file suffix, so a `.onnx` or `.engine` file behaves like a checkpoint and returns the same `Results`. Format coverage differs by family; the matrix on each model page is generated from the validated set rather than typed by hand. See [export and deploy](/docs/export) for the formats, their extras and their constraints. --- # Surface normals Surface-normal estimation predicts the direction each visible surface faces. LibreYOLO exposes it as the normal task, which returns a dense field of unit vectors on the original image canvas. Verified against LibreYOLO v1.5.0. ## Definition The `normal` task predicts a three-component unit vector per pixel from a single RGB image: the direction the surface at that pixel faces. Unlike depth, the output has no free scale, so two predictions are directly comparable without alignment. A prediction fills `result.normal_map`, a `NormalMap` payload holding an `(H, W, 3)` float32 array on the original image canvas, also reachable as `result.normals`. Vectors use LibreYOLO's OpenCV camera frame, with `+x` right, `+y` down and `+z` into the scene, and they face the camera, so a fronto-parallel surface reads `(0, 0, -1)`. `.assert_normalized()` checks that every pixel is finite and unit length within a tolerance. `result.boxes` stays empty, so `conf`, `iou` and `max_det` have no effect, and `Results.plot()` covers this task. ## Models Two families serve `normal`. [MoGe-2](/docs/models/moge-2) is the dedicated one: a single-forward monocular geometry model in three encoder sizes. LibreYOLO does not copy these checkpoints into its own organization; loading one downloads the matching size from the official repositories at a pinned revision and verifies it against a recorded SHA-256. [LibreMODUS](/docs/models/libremodus) produces normals as one target of an any-to-any model, and can take a depth map rather than an RGB image as its input. It needs the `modus` extra and your own authenticated Hugging Face account, and it offers neither `val()` nor `export()`, so it does not take part in the validation and export sections below. ## Predict MoGe-2 weights download on first use and are cached locally. **Predict a normal field** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreMoGe2s-normal.pt") result = model(SAMPLE_IMAGE, save=True) normals = result.normal_map print(normals.data.shape) # (H, W, 3) float32 unit vectors normals.assert_normalized() # raises if any pixel is not unit length ``` **Read one pixel** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreMoGe2s-normal.pt") result = model(SAMPLE_IMAGE) # OpenCV camera frame: +x right, +y down, +z into the scene. A surface # facing the camera reads close to (0, 0, -1). field = result.normals.data h, w = field.shape[:2] print(field[h // 2, w // 2]) ``` **Save the visualization** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreMoGe2s-normal.pt") result = model(SAMPLE_IMAGE) # plot() renders the field; it is defined for normal and edge results. result.plot().save("normals.png") ``` `imgsz` must be divisible by the ViT encoder's patch size, which LibreYOLO checks before the run starts. Predicting a list of images runs one forward pass per image; this task has no stacked-batch fast path. See [prediction](/docs/predict) for sources, streaming and result handling. ## Dataset format Normal validation pairs each image with a same-stem three-channel 16-bit PNG of the same resolution, plus an optional validity mask. ```text dataset/ data.yaml images/ val/room.jpg normals/ val/room.png masks/ val/room.png ``` ```yaml path: dataset train: images/train val: images/val normals_dir: normals masks_dir: masks nc: 1 names: {0: normal} ``` The target PNG is exactly three-channel `uint16` with channels stored as RGB. Decoding is `n = png / 65535 * 2 - 1` followed by renormalizing each vector, and the decoded vectors use the same OpenCV camera frame as the predictions. A mask pixel counts as valid when nonzero; without a mask file, every finite nonzero decoded vector is valid. Invalid and padded target pixels are held internally as `(0, 0, 0)` and never contribute to a metric. See [dataset formats](/docs/reference/dataset-formats) for the full contract. ## Train Neither normal family has a training implementation: `train()` raises `NotImplementedError` on both. MoGe-2's page points at its pinned official checkpoints for predict, validate and export. ## Validate `val()` measures the angle between each predicted vector and its ground-truth vector, over the pixels the dataset marks valid. **Validate and read the metric keys** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreMoGe2s-normal.pt") metrics = model.val(data="my-dataset.yaml", imgsz=518) print(metrics["metrics/mean_angular_error"]) # degrees print(metrics["metrics/median_angular_error"]) # degrees print(metrics["metrics/within_11_25"]) # percent of pixels print(metrics["metrics/within_22_5"], metrics["metrics/within_30"]) ``` `metrics/mean_angular_error` and `metrics/median_angular_error` are that angle in degrees, and lower is better. `metrics/within_11_25`, `metrics/within_22_5` and `metrics/within_30` are the percentage of valid pixels whose angular error falls within 11.25, 22.5 and 30 degrees, so higher is better. Note the unit: those three are percentages, not fractions. `fitness` is `metrics/within_11_25` divided by 100, which puts best-checkpoint selection on the same `[0, 1]` scale as every other task. ## Export An exported normal model loads back through `LibreYOLO()` on its file suffix, so a `.onnx` file behaves like a checkpoint and returns the same `Results`. **Export** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreMoGe2s-normal.pt") model.export(format="onnx", imgsz=518) ``` **Run the exported file** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads # like any checkpoint and returns the same Results object. model = LibreYOLO("LibreMoGe2s-normal.onnx") result = model(SAMPLE_IMAGE) print(result.normal_map.data.shape) ``` Normal export uses a fixed-resolution, batch-1 runtime contract: `dynamic` and a `batch` other than 1 are rejected, and `imgsz` must be divisible by the encoder's patch size. Per-format coverage is on the [MoGe-2 page](/docs/models/moge-2) and in the [full export matrix](/docs/reference/export-matrix). [Export](/docs/export) lists the arguments every format accepts. --- # Augmentations Augmentation is configured by knobs on TrainConfig, but each model family runs its own training pipeline, and a pipeline that has no mosaic branch ignores mosaic_prob rather than approximating it. Verified against LibreYOLO v1.5.0. ## Setting the knobs The augmentation knobs are ordinary `train()` arguments. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") model.train( data="my-dataset.yaml", epochs=100, mosaic_prob=1.0, mixup_prob=0.15, hsv_prob=1.0, flip_prob=0.5, no_aug_epochs=15, ) ``` **CLI** ```bash # The CLI spells mosaic_prob as mosaic and mixup_prob as mixup. libreyolo train model=LibreYOLO9s.pt data=my-dataset.yaml \ epochs=100 mosaic=1.0 mixup=0.15 hsv_prob=1.0 \ flip_prob=0.5 no_aug_epochs=15 ``` Two of them have shorter CLI spellings: `mosaic` maps to `mosaic_prob` and `mixup` maps to `mixup_prob`. Every other knob is spelled identically in both places. ## Three states, not two Whether a knob does anything depends on the family. The library keeps a declarative table of that, and each entry is one of three states. `used` means the knob reaches the pipeline and changes samples. `ignored` means it never reaches the pipeline, so setting it does nothing. `gated_by_mosaic` means it only applies to samples that took the mosaic branch, so with `mosaic_prob=0` it never fires even though it is wired up. That third state is the one that surprises people. On a YOLOX-style pipeline the affine warp runs on the mosaic canvas and MixUp blends a mosaic sample, so `mosaic_prob=0` silently disables `degrees`, `translate`, `shear`, `perspective`, `mosaic_scale`, `mixup_prob` and `mixup_scale` all at once. The trainer logs a warning for the MixUp case specifically: ```text mixup_prob=0.15 has no effect for YOLOv9: mixup only applies to mosaic samples and mosaic_prob=0. Set mosaic_prob > 0 to enable mixup. ``` The CLI warns about ignored knobs too, listing only the ones you actually typed: ```text Warning: RF-DETR ignores these parameters: degrees, mosaic ``` ## Four pipeline shapes Families cluster into four training pipelines, and the pipeline determines almost all of the answers. The YOLOX-style mosaic pipeline applies HSV jitter and flips per sample, then runs affine and MixUp inside the mosaic branch. It covers YOLOX, YOLOv7, YOLOv9 and its E2E and P2 variants, RTMDet, PicoDet, RT-DETR, RT-DETRv2 and FOMO. The DETR-style pass-through pipeline has no mosaic and no affine warp. Its photometric distortion, zoom-out and IoU crop are recipe constants rather than config knobs, so only `flip_prob` and `no_aug_epochs` are live. It covers D-FINE, Dome-DETR, DEIM, DEIMv2, RT-DETRv4, EC and, with one change, RF-DETR. The classification ImageFolder pipeline ignores every detection knob. Its horizontal flip is a fixed 0.5 that `flip_prob` does not reach. It has its own knob pack instead, described below. YOLO-NAS is a shape of its own: no mosaic at all, an always-on per-sample affine, and MixUp applied independently rather than gated. Its `mosaic_scale` value is reused as the affine scale range. SegFormer and NAFNet each run a task-specific pipeline whose randomness is fixed in the family rather than configurable. For SegFormer the live knobs are the class attributes `semantic_scale_jitter` and `semantic_hsv_prob`, not `mosaic_scale` and `hsv_prob`. NAFNet's crop and flips are coupled input and target operations at a fixed 0.5 probability. ## Which family honors which knob The table below is the shipped spec at `libreyolo/data/augment/spec.py`, which is asserted against the real pipeline plumbing by the library's own tests. Read it there rather than inferring from the architecture. **Read the support table for a family** ```python from libreyolo.data.augment.spec import AUG_KNOBS, aug_support for knob, description in AUG_KNOBS.items(): support = aug_support("yolo9")[knob] print(f"{knob:16} {support.status:16} {support.note or description}") ``` **Just the ignored ones** ```python from libreyolo.data.augment.spec import ignored_aug_params print(sorted(ignored_aug_params("rfdetr"))) ``` Summarized by pipeline, for the base knobs: | Knob | YOLOX-style | YOLO-NAS | DETR-style | Classification | |---|---|---|---|---| | `mosaic_prob` | used | ignored | ignored | ignored | | `mixup_prob` | gated by mosaic | used | ignored | ignored | | `hsv_prob` | used | used | ignored | ignored | | `flip_prob` | used | used | used | ignored | | `flipud` | used | used | ignored | ignored | | `degrees` | gated by mosaic | used | ignored | ignored | | `translate` | gated by mosaic | used | ignored | ignored | | `shear` | gated by mosaic | used | ignored | ignored | | `perspective` | gated by mosaic | used | ignored | ignored | | `mosaic_scale` | gated by mosaic | used | ignored | ignored | | `mixup_scale` | gated by mosaic | used | ignored | ignored | | `no_aug_epochs` | used | used | used | used | Exceptions inside those columns, all of them narrowing: - RTMDet, PicoDet, RT-DETR, RT-DETRv2 and FOMO have no vertical flip, so `flipud` is ignored. FOMO's mosaic wrapper is also built without perspective. - RF-DETR's native pipeline has no HSV jitter, so `hsv_prob` is ignored on top of the DETR-style column. - EC honors `hsv_prob`, `degrees` and `translate`, but only for `task="pose"`, whose keypoint-aware transform reads them. Its detect and segment paths use fixed photometric recipes. - DINOv2 follows the DETR-style column for its detect and semantic tasks and adds the classification pack for `task="classify"`. `no_aug_epochs` is `used` everywhere, but it does not mean the same thing everywhere. On the mosaic pipelines it turns mosaic and MixUp off for the final epochs. On the DETR-style pipelines it stops the photometric, zoom-out and crop augmentations and shapes the schedule's tail. On the classification and semantic pipelines it only shapes the tail. ## The classification pack Four knobs drive the classification pipeline and nothing else. Detection families ignore all four. **Classification pack** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreConvNeXtt-cls.pt") model.train( data="my-classification-dataset", epochs=50, auto_augment="randaugment", erasing=0.25, mixup=0.2, cutmix=0.2, ) ``` `auto_augment` takes `"randaugment"`, `"autoaugment"`, `"augmix"` or `None`. `erasing` is the RandomErasing probability. `mixup` and `cutmix` are per-batch probabilities producing soft labels; at most one runs per batch, MixUp first, so the two are additive and should sum to at most 1. All four default off, so classification training is unchanged unless you ask. One naming collision is worth stating plainly: on the CLI, `mixup` is the alias for the detection `mixup_prob`. The classification `mixup` field has no CLI spelling of its own and is reachable only through `model.train(mixup=...)` in Python. ## Family-specific knobs Some knobs live on a family's config subclass rather than on the base class, so they exist for that family only and have no CLI flag. | Family | Knob | Effect | |---|---|---| | YOLOv9, YOLOv9-E2E, YOLOv9-P2 | `copy_paste` | Copy-paste instance augmentation probability, `task="segment"` only | | YOLOv9, YOLOv9-E2E, YOLOv9-P2 | `copy_paste_mode` | `"flip"` reuses the same sample mirrored, `"mixup"` pulls a second sample | | YOLOv9, YOLOv9-E2E, YOLOv9-P2 | `rot90` | Random 90 degree rotation probability | | YOLOv9 | `max_labels` | Per-image ground-truth cap in the train transforms, default 100 | | RF-DETR | `copy_paste`, `copy_paste_mode` | Copy-paste for `task="segment"`, `"flip"` mode only | | RF-DETR, D-FINE, EC | `crop_resize_prob` | Random crop-resize probability | | EC, YOLO-NAS | `brightness_contrast_prob`, `affine_prob` | Pose-path jitter and keypoint-aware affine probabilities | `max_labels` is the one that silently loses data. Boxes past the cap are dropped without an error, so dense imagery such as aerial photography needs it raised. Mosaic and MixUp are disabled for oriented-box training regardless of the knobs, because corner-aware augmentation for rotated boxes is not implemented. ## Related - [Hyperparameters](/docs/train/hyperparameters) for `no_aug_epochs` as a schedule argument and the rest of `train()`. - [Datasets](/docs/train/datasets) for the label formats these transforms consume. --- # Training on a rented GPU A rented GPU turns a training run into a job with a start, an end, and a bill. The work is the same as training locally; what changes is getting the data in, watching from outside, getting the weights out, and shutting the machine down. Verified against LibreYOLO v1.5.0. ## Before you rent anything Two decisions cost more later than they do now. Get the dataset onto a CDN first. Packing it as a single tar in a Hugging Face dataset repository works the same on every provider, serves fast to all of them, and needs nothing but an `HF_TOKEN` in the job environment when the repo is private. Copying a dataset up from a home connection, or pulling it from a slow origin on the box, is billed GPU time spent waiting. **Pack and upload once, from your machine** ```bash tar cf my-dataset.tar my-dataset/ huggingface-cli upload my-org/my-dataset my-dataset.tar --repo-type dataset ``` **Stage on the box** ```python import tarfile from huggingface_hub import hf_hub_download path = hf_hub_download( "my-org/my-dataset", "my-dataset.tar", repo_type="dataset" ) with tarfile.open(path) as archive: archive.extractall("/root/data") ``` Then size the disk. Providers that bill storage bill on allocated capacity, not used capacity, and a disk cannot be shrunk after creation. Add up the staged data, the checkpoints, and roughly 30 percent of headroom, and stop there. ## Install on the box **On the box** ```bash pip install libreyolo # Add only the extras the run needs. rfdetr for RF-DETR training, # lora for parameter-efficient fine-tuning, onnx to export afterwards. pip install "libreyolo[rfdetr,lora]" ``` **Check the GPU before anything else** ```python import torch print(torch.__version__, torch.cuda.is_available()) print(torch.cuda.get_device_name(0)) # A wheel built for another architecture reports True and then fails # on the first real kernel, so run one. x = torch.rand(2000, 2000, device="cuda") print(float((x @ x).sum())) ``` Install PyTorch first if the image does not already carry a CUDA build matching the card, then LibreYOLO, so pip does not resolve its own CPU-only torch. The second snippet is not optional ceremony: a wheel built for the wrong GPU architecture reports `torch.cuda.is_available() == True` and then fails on the first real operation with `CUDA error: no kernel image is available for execution on the device`. One matrix multiply catches it before an hour of setup does not. Point `HF_HOME` at persistent storage if the provider offers a volume, so checkpoint and dataset downloads survive between runs. ## Launch Run the job detached. An interactive session that dies with your network connection takes the training with it. **Detached, so the job survives a disconnect** ```bash nohup libreyolo train \ model=LibreYOLO9s.pt \ data=/root/data/my-dataset/data.yaml \ epochs=100 batch=-1 imgsz=640 \ project=/root/runs name=run1 \ > /root/train.log 2>&1 & ``` **Multi-GPU, from a Python file** ```python from libreyolo import LibreYOLO if __name__ == "__main__": model = LibreYOLO("LibreYOLO9s.pt") model.train( data="/root/data/my-dataset/data.yaml", epochs=100, batch=64, # global batch across all GPUs device="0,1,2,3", project="/root/runs", name="run1", ) ``` `batch=-1` is worth using here specifically, because you are usually on a card you have not trained on before. It probes the model in training mode with a real backward pass and picks the largest power of two that fits, which is faster than discovering the ceiling with an out-of-memory error twenty minutes in. See [Hyperparameters](/docs/train/hyperparameters). On a multi-GPU box, `device="0,1,2,3"` spawns one worker per GPU by itself, and `batch` stays the global batch across all of them. The `__main__` guard is mandatory, because each worker re-imports the script. That, and the rest of the distributed behavior, is on [Multi-GPU training](/docs/train/multi-gpu). ## Watch it from outside Every run writes `status.json` into its run directory, rewritten atomically each epoch. It is the cheap read: a few hundred bytes carrying the state, the current epoch, the ETA and the latest metrics, without parsing a log. **One cheap read** ```bash cat /root/runs/run1/status.json ``` **From a script** ```python import json with open("/root/runs/run1/status.json") as handle: status = json.load(handle) print(status["state"], status["current_epoch"], status["eta_seconds"]) print(status.get("metrics")) ``` **In a browser, over an SSH tunnel** ```bash # On the box (binds 127.0.0.1:8420 by default): libreyolo monitor /root/runs/run1 --no-browser # From your machine, then open http://localhost:8420 locally: # ssh -L 8420:localhost:8420 @ ``` `metrics.jsonl` alongside it has the full per-epoch history, and `train.log` has the console output. `libreyolo monitor` serves a browser dashboard over all three using only the standard library, so it needs nothing installed on the box beyond LibreYOLO itself. Reach it over an SSH port forward. None of these touch the training process, so they attach to a live run, reopen a finished one, or inspect a crashed one. ## Get the weights out before you stop paying The box is disposable. Push checkpoints at milestones, not only at the end, because a crash, a preemption or running out of credit otherwise loses the whole run. **Push the weights somewhere permanent** ```bash huggingface-cli upload my-org/my-run \ /root/runs/run1/weights/best.pt best.pt ``` `weights/best.pt` and `weights/last.pt` are written every epoch and on every improvement. `save_period=N` adds `weights/epoch_.pt` snapshots on top, which is what makes a mid-run push cheap. `summary.json` and `results.csv`, where the family writes them, are small and worth taking too. A callback on `on_train_epoch_end` is the clean way to automate the push. See [Experiment loggers](/docs/train/loggers), where the hosted backends also give you the metrics without touching the box at all. ## Stop paying This is the part that costs real money when it goes wrong, and the rule differs by provider model. On a marketplace where you rent a raw machine, billing runs on wall clock until the instance is destroyed. An idle GPU bills exactly like a busy one, so killing the training process saves nothing on its own. A stopped instance still bills its disk. On a serverless platform where the job is a decorated function, the container scales to zero when the function returns, so a forgotten box is much less likely. A hung job with no timeout still bills, so always set one. Stopping instead of destroying is a real lever, and a real trap. Measured on a rented 8x RTX 4090 with a 250 GB disk on 2026-07-31: running billed $3.4828 per hour, stopped billed $0.0694 per hour for the disk alone, and destroyed billed nothing. That is a 98 percent saving while keeping the environment, the staged data and the checkpoints in place. The stopped rate is arithmetic you can do before renting: ```text stopped $/hr = allocated_GB * storage_cost_per_GB_per_month / 730 = 250 * 0.20 / 730 = $0.0694/hr ``` Compare it against what a rebuild costs: renting again, pulling the image, installing, and re-staging the data. On that same box a rebuild was about 15 minutes of setup plus 43 GB of inbound transfer, roughly $1.00 all in. Against $0.0694 per hour, coming back within about 14 hours favors stopping and a longer gap favors destroying and rebuilding from the staged copy. One risk makes stopping unsafe for scarce hardware: stopping releases the GPUs. Nothing reserves them, so restarting only succeeds if the host still has them free. Your disk is safe; your GPUs are not. ## Serverless, as a function If you would rather not manage a machine, both Modal and Beam run a decorated Python function on a GPU and scale to zero when it returns. LibreYOLO's own nightly test suite runs on Modal, and `tools/ci/modal_nightly.py` in the library repository is the working in-repo example to copy from. ```python import modal image = ( modal.Image.debian_slim(python_version="3.11") .apt_install("git", "libgl1", "libglib2.0-0") # OpenCV system libraries .pip_install("libreyolo[rfdetr]") ) app = modal.App("libreyolo-train") cache = modal.Volume.from_name("libreyolo-cache", create_if_missing=True) @app.function(gpu="A100", timeout=6 * 60 * 60, volumes={"/cache": cache}) def train(): import os os.environ["HF_HOME"] = "/cache/hf" # cache weights across runs from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") model.train(data="coco8.yaml", epochs=100, project="/cache/runs") cache.commit() # persist the volume @app.local_entrypoint() def main(): train.remote() ``` Run it with `modal run modal_train.py`. The container filesystem is ephemeral, so anything worth keeping goes in the volume or gets pushed out. Set `timeout=` explicitly; that is the only thing standing between a hung run and an open-ended bill. Beam takes the same shape with a `@function` decorator, a `Volume`, and `train.remote()` called from `__main__`. ## Right-size by cost per job $/hr is the wrong number to optimize. A small model half-idles a large card, so a cheaper and slower GPU is often cheaper per epoch. Run the profiler for a few steps on the rented card before committing to a long run: if the verdict is `dataloader` or `host / launch`, a faster GPU buys nothing and more workers or a larger batch buys a lot. See [Training performance](/docs/train/performance). ## Related - [Datasets](/docs/train/datasets) for the layout the staged archive should have, and the doctor command that catches problems before a GPU is billing. - [Multi-GPU training](/docs/train/multi-gpu) for multi-card boxes. --- # Datasets A LibreYOLO dataset is a YAML file naming a root, its splits and its class names. Everything else, including where the label files live, is derived from that file by convention. Verified against LibreYOLO v1.5.0. ## Point train at a dataset `data=` takes a YAML path or the name of a config that ships with the package. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") # A bundled name, a relative path or an absolute path all work. model.train(data="coco8.yaml", epochs=10) ``` **CLI** ```bash libreyolo train model=LibreYOLO9s.pt data=coco8.yaml epochs=10 ``` The name is resolved in a fixed order: an absolute path that exists, then the name as given relative to the working directory, then the same name with `.yaml` appended, then the bundled config directory. When nothing matches, the error names every directory that was searched and lists the bundled configs. ## Bundled configs Thirteen dataset configs ship inside the package, under `libreyolo/config/datasets/`. | Config | Task | Notes | |---|---|---| | `coco8.yaml` | detect | 8 images, downloads from a plain URL | | `coco128.yaml` | detect | 128 images | | `coco1000.yaml` | detect | 800 train, 200 val | | `coco5000.yaml` | detect | 4000 train, 1000 val | | `coco.yaml` | detect | full COCO 2017 | | `coco-val-only.yaml` | detect | val2017 only | | `coco8-pose.yaml` | pose | 8 images, COCO-17 keypoints | | `coco-pose.yaml` | pose | COCO 2017 keypoints | | `ade20k.yaml` | semantic | 150 classes | | `cityscapes.yaml` | semantic | 19 classes, download by hand | | `cocostuff.yaml` | semantic | 182 classes, download by hand | | `gopro.yaml` | restore | deblurring pairs | | `sr8.yaml` | restore | super-resolution pairs | Only `coco8.yaml` and `coco128.yaml` carry a plain download URL. The rest either carry a Python download block, which needs the opt-in described below, or expect the data to already be on disk. ## Where a dataset lives on disk The YAML `path` key names the dataset root. An absolute `path` is used as written. A relative one is looked for under the datasets directory first, then beside the YAML file itself, and a dataset that is about to be downloaded goes under the datasets directory. That directory is `~/datasets`, overridden by the `LIBREYOLO_DATASETS_DIR` environment variable. There is no settings file for it. ## The YAML keys ```yaml path: my-dataset # dataset root train: images/train # required to train val: images/val # required to validate test: images/test # optional nc: 3 # optional; must agree with names names: 0: person 1: helmet 2: vest download: https://example.com/my-dataset.zip # optional ``` `train`, `val` and `test` each accept an image directory, a `.txt` file listing one image path per line, or a list mixing both. Lines in a `.txt` list may be relative, in which case they resolve against the list file's own directory, and lines starting with `#` are skipped. `names` may be a list or an integer-keyed mapping. `nc` is optional; when both are present and disagree, the doctor reports it as an error. ## Directory layout and label files Detection, segmentation, pose and oriented boxes all share one layout. The label path is derived from the image path by rewriting an `images` directory component to `labels` and changing the extension to `.txt`: ```text my-dataset/ images/train/0001.jpg -> labels/train/0001.txt images/val/0002.jpg -> labels/val/0002.txt ``` Only a whole `images` path component is rewritten, so a directory named `images_old` is left alone. A detection row is five fields, all normalized to `[0, 1]` against the original image width and height: ```text ``` A missing or empty label file means the image has no objects, and it trains as background rather than raising. A row with more than five fields is read as a polygon and its box becomes the polygon's extent, so a segmentation export used for detection training loads without complaint. The doctor reports how many rows took that path. ## Other tasks Segmentation keeps the same layout with polygon rows, ` ... `, at least three points. A five-field detection row is accepted and means a rectangular instance. Pose adds `kpt_shape: [K, D]` and an optional `flip_idx` permutation to the YAML. Each row is exactly `5 + K * D` fields: the box, then `K` keypoints of `x y` or `x y v`, with visibility `0`, `1` or `2`. Oriented boxes use exactly nine fields, the class followed by four corner points in normalized coordinates. No angle is stored in the file. Semantic segmentation pairs each image with a single-channel mask of the same resolution, resolved by substituting `masks_dir` (default `masks`) for `images`. Pixel value `255` means ignore. `label_mapping` remaps source ids to train ids at load time. Classification uses an ImageFolder tree instead of label files, with `train/` and `val/` each containing one directory per class. The class-to-index mapping is the sorted folder name order. Restoration pairs a degraded input with a clean target of identical resolution through `input_dir` and `target_dir`. Depth, surface normals and edges each pair an image with a dense map through their own directory key. The full per-task contract, including the depth scale conventions and the panoptic segment-id PNG encoding, is `docs/dataset_schema.md` in the library repository. ## Native COCO JSON A COCO JSON annotation file can be used directly. Add an `annotations` mapping, and the split path becomes the image root: ```yaml path: my-dataset train: images/train val: images/val annotations: train: annotations/train.json val: annotations/val.json ``` When `names` is present, the JSON category names must match it, and `names` defines the label ids the model predicts. Without `names`, COCO category ids are sorted and mapped densely to `0..N-1`. This path expects one image directory per split. A list of paths or a `.txt` image list raises rather than silently loading a different set. ## Autodownload A dataset counts as present when its `train` or `val` path resolves to a non-empty directory or an existing file. When it does not, and the YAML has a `download` key, the value decides what happens next. An `http` or `https` URL is fetched and, if it is a zip, extracted into the dataset root. Anything else is treated as an embedded Python script and runs only when `allow_download_scripts=True`. Without that, the script is skipped with a warning and training continues against whatever is on disk. ```bash libreyolo train model=LibreYOLO9s.pt data=coco.yaml allow_download_scripts=true ``` The flag is a code-execution gate, not a network gate. URL downloads happen either way; it is the `download: |` blocks that need it. The CLI prints a warning when the flag is on, and the doctor never enables it. ## Check the dataset before you train `libreyolo doctor` reads a detection dataset and reports what would go wrong before a GPU is involved. It exits 1 when it finds errors, so it works as a CI gate. **Check a dataset** ```bash libreyolo doctor my-dataset.yaml ``` **Fail a CI job on warnings too** ```bash libreyolo doctor my-dataset.yaml strict=true json=true ``` **Skip the image decode pass** ```bash # Reads labels and YAML only. Corruption, duplicate and split-leakage # checks all need the pixels, so they are skipped. libreyolo doctor my-dataset.yaml fast=true ``` **Python** ```python from libreyolo import doctor report = doctor.diagnose("my-dataset.yaml", imgsz=640) for finding in report.findings: print(finding.severity.value, finding.check_id, finding.message) raise SystemExit(report.exit_code(strict=False)) ``` The checks come in six families: | Family | Looks for | |---|---| | `config` | missing `names`, `nc` that disagrees with `names`, missing or empty splits, duplicate class names | | `files` | images with no label file, labels with no image, missing images listed in a split, stem collisions | | `labels` | malformed rows, class ids outside `[0, nc)`, coordinates outside `[0, 1]`, zero-area boxes, tiny or huge boxes, duplicate boxes, byte-identical label files | | `balance` | classes with zero or few instances, class imbalance ratio, classes present in one split only, background image share | | `images` | undecodable files, EXIF rotation, odd channel layouts, uniform images, exact and near duplicates | | `splits` | the same image appearing in two splits, exactly or near-identically | `--only` and `--skip` take a check id or a family prefix, so `skip=images,labels.tiny_object` is valid. `--fast` drops every check that needs to decode pixels, which is the `images` and `splits` families. Two behaviors are worth knowing. `--strict` makes warnings fail the exit code as well as errors. And the doctor covers detection datasets only: a pose, segment or oriented-box dataset is rejected with a message naming what it detected, rather than being checked against the wrong contract. ## Related - [Hyperparameters](/docs/train/hyperparameters) for the arguments `train()` takes once the data is in place. - [Validation and metrics](/docs/train/validation) for evaluating on the `val` or `test` split. --- # Knowledge distillation Distillation adds a second loss term that pulls the student's intermediate feature maps toward a frozen teacher's. LibreYOLO taps features with forward hooks, so the teacher's own head and loss are never involved. Verified against LibreYOLO v1.5.0. ## Distill from a larger checkpoint Setting `distill_model` turns distillation on. The value is a teacher checkpoint, loaded through the same factory as any other model. **Python** ```python from libreyolo import LibreYOLO # A larger checkpoint of the same family supervises the small one. model = LibreYOLO("LibreYOLO9s.pt") model.train( data="my-dataset.yaml", epochs=100, distill_model="LibreYOLO9c.pt", distill_loss_type="mgd", ) ``` **CLI** ```bash libreyolo train model=LibreYOLO9s.pt data=my-dataset.yaml \ epochs=100 distill_model=LibreYOLO9c.pt distill_loss_type=mgd ``` The teacher runs forward under `no_grad`, and under autocast when AMP is on, so the frozen model does not pay full-precision compute at every step. Forward hooks capture its feature maps at named tap points, the loss compares them with the student's, and the result is added to the training loss and reported as a component named `distill`. ## Distill from a frozen foundation backbone A self-supervised ViT can supervise a single student backbone stage instead. The teacher's features come from its own feature extractor rather than hooks, and the loss handles the mismatch between a patch grid and a convolutional stride. **Python** ```python from libreyolo import LibreYOLO # A frozen self-supervised ViT supervises one backbone stage. model = LibreYOLO("LibreYOLO9s.pt") model.train( data="my-dataset.yaml", epochs=100, distill_model="dinov2", ) ``` **CLI** ```bash libreyolo train model=LibreYOLO9s.pt data=my-dataset.yaml \ epochs=100 distill_model=dinov2 ``` `distill_model` recognizes `dinov2`, which is DINOv2-base, plus `dinov2_vits14`, `dinov2_vitb14`, `dinov2_vitl14`, `dinov2-small`, `dinov2-base`, `dinov2-large`, and any raw hub id starting with `facebook/dinov2`. Anything else is treated as a teacher checkpoint path. This path uses `feat_mse` regardless of `distill_loss_type`, and needs `transformers` installed. A teacher that loads with missing weight keys aborts rather than distilling against a partly random backbone. ## Which families Distillation support is a method on the student model, and there are two of them. `get_distill_config()` provides the multi-scale tap points a detector teacher supervises. YOLOv9, YOLOX and RF-DETR implement it. `get_backbone_distill_config()` provides the single backbone stage a foundation teacher supervises. YOLOv9 implements it, and it is the only family that does. Anything else raises rather than training without the loss: ```text LibreDFINE does not implement get_distill_config(). Distillation is not yet supported for the 'dfine' family. ``` ```text Foundation-model distillation into the 'yolox' family is not supported yet (no get_backbone_distill_config()). ``` ## Tap points The tap points are fixed per family and per role, so teacher and student do not need to be the same architecture; they need matching feature strides. | Family | Role | Tap points | Strides | |---|---|---|---| | YOLOv9 | teacher or student | `neck.elan_up2`, `neck.elan_down1`, `neck.elan_down2` | 8, 16, 32 | | YOLOv9 | foundation student | `backbone.elan3` | 16 | | YOLOX | teacher or student | `backbone.C3_p3`, `backbone.C3_n3`, `backbone.C3_n4` | 8, 16, 32 | | RF-DETR | teacher or student | `model.backbone.0.projector.stages.0` | probed at setup | Mismatched strides raise before training starts: ```text Teacher and student must have matching strides. Teacher: [8, 16, 32], Student: [16] ``` That check is skipped for foundation teachers, whose whole point is that the grids differ. ## The three losses `distill_loss_type` selects the feature loss for a detector teacher. A foundation teacher always uses `feat_mse`. `mgd`, masked generative distillation, masks a fraction of the student's spatial positions and trains a small two-convolution generator to reconstruct the teacher's full feature map from what remains. `distill_mask_ratio` sets the masked fraction, default 0.65. `cwd`, channel-wise distillation, turns each channel's spatial activations into a probability distribution and minimizes the KL divergence channel by channel. `distill_tau` is the softmax temperature, default 1.0. `feat_mse` aligns the student's channels to the teacher's with a 1x1 convolution, resizes the teacher's grid to the student's bilinearly, and takes the mean squared error. `distill_normalize=True` L2-normalizes both feature maps over the channel dimension first, which makes the match angle-only and scale-invariant. It defaults to `False`. `dis` is the global weight applied on top. Left unset, each loss uses its own published default: 2e-5 for MGD, 1.0 for CWD and 1.0 for feature MSE. Those differ by five orders of magnitude, so a weight tuned for one loss type is meaningless for another. **Tuning the loss** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") model.train( data="my-dataset.yaml", distill_model="LibreYOLO9c.pt", distill_loss_type="cwd", dis=1.0, # global distillation weight distill_tau=1.0, # CWD softmax temperature ) ``` `distill_mask_ratio`, `distill_tau` and `distill_normalize` have no CLI flags. They are Python arguments or `cfg=` YAML keys. RF-DETR is also Python-only for distillation as a whole, because its CLI argument mapping does not carry the distillation keys. ## Adapters, checkpoints and multi-GPU Every loss builds small trainable modules that live outside the student: the 1x1 channel adapters, and MGD's generator. They get their own optimizer parameter group at the run's effective learning rate. Those modules are written into the checkpoint under a `distiller` key and restored on resume, so a resumed run does not restart its projectors cold. Under DDP the adapters sit outside the wrapped student, which means the DDP reducer never sees their gradients. The trainer all-reduces them explicitly each step, so every rank trains the same adapters. CUDA graph capture is not available on a distillation run. Passing `cuda_graph=True` logs one line and trains eager. See [Training performance](/docs/train/performance). ## Related - [Layer freezing](/docs/train/layer-freezing) and [LoRA fine-tuning](/docs/train/lora), neither of which is blocked from being combined with distillation. - [Hyperparameters](/docs/train/hyperparameters) for the rest of `train()`. --- # Hyperparameters Every training argument is a field on a TrainConfig dataclass. The base class defines the field and its default; each model family subclasses it and overrides the defaults that its published recipe changes. Verified against LibreYOLO v1.5.0. ## Setting arguments `train()` takes keyword arguments and the CLI takes the same names in `key=value` form. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") results = model.train( data="my-dataset.yaml", epochs=100, batch=16, imgsz=640, lr0=0.01, ) print(results["best_mAP50_95"]) ``` **CLI** ```bash libreyolo train model=LibreYOLO9s.pt data=my-dataset.yaml \ epochs=100 batch=16 imgsz=640 lr0=0.01 ``` Both paths end at the same place. The kwargs are handed to `TrainConfig.from_kwargs()`, which builds the family's config dataclass. ## A typo does not raise `from_kwargs()` drops any key that is not a field on the config and emits a `UserWarning` naming it. Training then starts with the default in place: ```python # UserWarning: Unknown training config keys (ignored): ['learning_rate'] model.train(data="my-dataset.yaml", learning_rate=0.001) ``` Nothing fails, the run completes, and the learning rate was never what the caller asked for. Read the warnings on the first epoch of a new recipe. The CLI is stricter, because it validates flag names before the config is built, so a misspelled CLI flag is rejected outright. ## Defaults are per family `TrainConfig` defines the field and a base default. Each family subclasses it and overrides what its published recipe changes, so there is no single correct answer to "what is the default learning rate". The base defaults are `optimizer="sgd"`, `lr0=0.01`, `momentum=0.937`, `weight_decay=5e-4`, `scheduler="yoloxwarmcos"`, `epochs=300`, `batch=16`, `imgsz=640` and `amp=True`. Three examples of how far a family moves from that: | Field | Base | YOLOv9 | D-FINE | YOLO-NAS | |---|---|---|---|---| | `optimizer` | `sgd` | `sgd` | `adamw` | `adamw` | | `lr0` | `0.01` | `0.01` | `2e-4` | `5e-4` | | `weight_decay` | `5e-4` | `5e-4` | `1e-4` | `1e-5` | | `scheduler` | `yoloxwarmcos` | `linear` | `flat_cosine` | `cos` | | `epochs` | `300` | `300` | `132` | `300` | | `amp` | `True` | `True` | `False` | `False` | D-FINE and DEIM ship with `amp=False` because the D-FINE decoder clamps activations at 65504, the largest finite float16 value. YOLO-NAS and FOMO also default it off. The CLI's `--amp` flag defaults to `True` for every family, so it counts as user-provided and overrides the family default; leave it alone unless you mean to change it. To read a family's real defaults rather than guessing: **Read a family's resolved defaults** ```python from dataclasses import fields from libreyolo import LibreYOLO9 from libreyolo.training.config import TrainConfig family_cfg = LibreYOLO9.TRAIN_CONFIG() base_cfg = TrainConfig() for f in fields(family_cfg): family_value = getattr(family_cfg, f.name) base_value = getattr(base_cfg, f.name, None) if not hasattr(base_cfg, f.name) or family_value != base_value: print(f"{f.name}: {family_value}") ``` **CLI** ```bash # Prints the train, val and predict defaults, including family overrides. libreyolo cfg ``` ## Batch size `batch` is the global batch. Under multi-GPU training each rank loads `batch // world_size`, so the number you pass is the number of images per optimizer step regardless of how many GPUs are involved. See [Multi-GPU training](/docs/train/multi-gpu). `batch=-1` turns on autobatch. The trainer probes the model in training mode with a real backward pass at powers of two, fits a line to the memory curve, and picks the largest power of two strictly below the extrapolated value that fits within 60 percent of total VRAM. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") # batch=-1 probes GPU memory and resolves to a concrete power of two. model.train(data="my-dataset.yaml", batch=-1, imgsz=640) ``` **CLI** ```bash libreyolo train model=LibreYOLO9s.pt data=my-dataset.yaml batch=-1 ``` Probing in training mode with a backward pass is the point: an inference-mode probe misses the retained activations and gradient tensors, which for a deep CNN are several times the inference footprint. RF-DETR lowers the target fraction to 45 percent, because the probe's synthetic backward still underestimates what its criterion and auxiliary decoder layers cost. Autobatch is a CUDA feature. On CPU or MPS it logs one line and keeps the default batch. ## Gradient accumulation `nbs` sets the nominal, or effective, batch size. The trainer accumulates `round(nbs / batch)` micro-batches per optimizer step. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") # 4 micro-batches of 16 per optimizer step, effective batch 64. model.train(data="my-dataset.yaml", batch=16, nbs=64) ``` Left as `None`, the default, accumulation is off and training is unchanged. ## Learning rate and schedule `lr0` is the initial learning rate and `optimizer` accepts `sgd`, `adam` and `adamw`. `momentum` is SGD momentum or Adam's beta1, `weight_decay` is the L2 term, and `nesterov` applies to SGD. The schedule is shaped by `scheduler`, `warmup_epochs`, `warmup_lr_start` and `min_lr_ratio`. `no_aug_epochs` sets how many final epochs run without strong augmentation, and several schedules use it to shape their tail as well, so it is not purely an augmentation knob. What each family does with the augmentation half of it is on [Augmentations](/docs/train/augmentations). Some families add their own learning-rate knobs. `backbone_lr_mult` scales the backbone group against the head, `clip_max_norm` sets gradient clipping, and SegFormer uses `head_lr_mult` to run its decode head at ten times the backbone rate. These live on the family's config subclass, not the base one. ## EMA `ema=True` keeps an exponential moving average of the weights alongside the trained ones. It is on by default everywhere except FOMO. `ema_decay` is the target decay. The decay ramps in rather than starting at its target: the effective value at update `n` is `ema_decay * (1 - exp(-n / tau))` with `tau` defaulting to 2000, so early updates track the model more closely and late updates smooth it. Family defaults range from `0.997` on YOLO-NAS pose through `0.9998` on YOLOX to `0.9999` on YOLOv9 and the DETR line. The EMA weights are what gets validated and what `best.pt` and `last.pt` carry. The raw trained weights are also stored, under the `train_model` key, so a resume continues from the trained trajectory rather than from the average. ## Precision `amp=True` runs the forward pass under CUDA autocast. `amp_dtype` selects `float16` (the default) or `bfloat16`; `fp16` and `bf16` are accepted spellings. Float16 needs dynamic loss scaling and gets a live `GradScaler`. Bfloat16's wider exponent range does not, so its scaler is constructed but disabled, which keeps the optimizer path identical. Asking for bfloat16 on a CUDA device without bfloat16 support raises at setup rather than degrading silently. ## Output, checkpoints and stopping Runs are written to `project/name`. `project` defaults to `runs/train` everywhere, but `name` is one of the per-family overrides: the base default is `exp`, while YOLOv9 uses `yolo9_exp` and D-FINE uses `dfine_exp`. With `exist_ok=False`, the default, an existing directory gets an incremented suffix instead of being overwritten. `save_period` writes an extra `weights/epoch_.pt` every N epochs, on top of `weights/last.pt` after each epoch and `weights/best.pt` whenever the tracked metric improves. `eval_interval` sets how often validation runs, and `patience` stops the run after that many epochs without improvement, with `0` disabling early stopping. `cache` speeds up repeated epochs by holding decoded images in RAM (`True` or `"ram"`) or as `.npy` files beside the sources (`"disk"`). Cached reads are byte-identical to fresh ones. With dataloader workers, `"disk"` is the safer of the two. ## Resume `resume=True` continues an interrupted run. The checkpoint has to be loaded first, because resume reads it from the model, not from a separate argument. **Python** ```python from libreyolo import LibreYOLO # Load the interrupted run's checkpoint, then ask to resume. model = LibreYOLO("runs/train/exp/weights/last.pt") model.train(data="my-dataset.yaml", epochs=100, resume=True) ``` **CLI** ```bash libreyolo train model=runs/train/exp/weights/last.pt \ data=my-dataset.yaml epochs=100 resume=true ``` Resume restores the trained weights, the optimizer state, the EMA weights and update count, the best-metric tracking, the `GradScaler` scale, and the PyTorch, CUDA and NumPy random states. It starts at the checkpoint's epoch plus one and fast-forwards the schedule to that position. Two things it will not do. `resume=True` cannot be combined with `pretrained`, which raises. And when the checkpoint's best-metric key differs from the current run's, best-metric tracking resets to zero with a warning rather than comparing values that do not mean the same thing. ## Recipes in a file `cfg=` loads a YAML mapping of `TrainConfig` field names and merges it under the explicit keyword arguments, so a kwarg always wins over the file. **Python** ```python from libreyolo import LibreYOLO # Keys in the yaml are TrainConfig field names. Explicit kwargs win. model = LibreYOLO("LibreYOLO9s.pt") model.train(data="my-dataset.yaml", cfg="my-recipe.yaml", epochs=50) ``` `size` and `num_classes` are stripped from the file, because the model instance already owns them. There is no `--cfg` flag on the CLI; the file path is a Python argument. ## Related - [Datasets](/docs/train/datasets) for what `data=` accepts. - [Augmentations](/docs/train/augmentations) for the augmentation knobs and which families honor them. - [Layer freezing](/docs/train/layer-freezing) and [LoRA](/docs/train/lora) for training a subset of the weights. - [Validation and metrics](/docs/train/validation) for what the run reports. --- # Layer freezing Freezing holds selected weights fixed while the rest of the model trains. Selectors address a family's own ordered freeze groups or its module names, not raw layer numbers from a YAML graph. Verified against LibreYOLO v1.5.0. ## Freeze something `freeze` is optional and defaults to no freezing. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") # The first 10 groups are the whole YOLOv9 backbone. model.train(data="my-dataset.yaml", epochs=50, freeze=10) ``` **CLI** ```bash libreyolo train model=LibreYOLO9s.pt data=my-dataset.yaml \ epochs=50 freeze=10 ``` **By name** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRFDETRs.pt") model.train(data="my-dataset.yaml", epochs=50, freeze="backbone") ``` **Several selectors** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") model.train(data="my-dataset.yaml", freeze=["backbone", "neck"]) ``` Freezing runs after the model is built and after any head rebuild for a new class count, and before the optimizer is created, so the optimizer only ever receives trainable parameters. ## What a selector can be | Value | Meaning | |---|---| | `None`, `False`, `""`, `"none"` | Train every parameter | | `10` or `"10"` | Freeze the first ten family freeze groups | | `[0, 3, 7]` | Freeze those zero-based groups | | `"backbone"` | Freeze the matching group, module or parameter prefix | | `["backbone", "neck"]` | Freeze each listed selector | | `["backbone", 3]` | Mixed lists work | A string is parsed before it is interpreted, so the CLI and a YAML config accept the same shapes as Python. `freeze="[0, 3, 'head']"` is parsed as a literal list, `freeze="backbone,neck"` splits on the comma, and a bare decimal string becomes a count. `freeze=True` is rejected as ambiguous. Name selectors match a freeze group name, a module name, or a parameter-name prefix, and glob characters `*`, `?` and `[` work. A leading `model.` is treated flexibly, so `backbone` and `model.backbone` both hit whichever spelling the family uses internally. ## Groups are family-defined An integer addresses a family's own ordered list of freeze groups, not a position in a shared graph. LibreYOLO's families are not all one YAML-indexed sequential model, so a raw layer number would mean something different on each of them. YOLOv9 orders its groups from the input side: ten backbone stages, then six neck stages, then the head. That is why `freeze=10` is exactly the backbone. `backbone`, `neck` and `head` are stable name selectors on top of it. RF-DETR's groups are `backbone.encoder`, `backbone.projector`, `decoder`, `queries`, `transformer.encoder_output` and `head`. Names are the better choice here, because transformer components do not map onto a layer count. `backbone` matches both backbone groups by prefix. Families that do not define semantic groups fall back to a conservative default: each direct child of the model that owns at least one parameter, in declaration order. That is usually a short list, so a large integer will not find enough groups: ```text freeze index 10 is out of range for 3 available freeze groups. ``` To see the real list rather than guessing: **List a family's freeze groups in order** ```python from libreyolo import LibreYOLO9 from libreyolo.models.yolo9.trainer import YOLO9Trainer model = LibreYOLO9("LibreYOLO9s.pt", size="s") trainer = YOLO9Trainer(model=model.model, wrapper_model=model, size="s") for index, (name, _module) in enumerate(trainer.get_freeze_groups()): print(index, name) ``` ## Failures are loud Every way of getting this wrong raises rather than training something you did not ask for. A selector that matches nothing raises, naming the selectors that missed: ```text freeze selector(s) matched no parameters: 'backbon' ``` A freeze that would leave nothing trainable raises, both at freeze time and again when the optimizer is built: ```text freeze would leave no trainable parameters. Use a smaller freeze value or target a narrower module. ``` Which is what `freeze="all"` does, since `all` matches every parameter. When freezing succeeds, one line records what happened: ```text Layer freezing: selectors=[10], tensors=124, params=2103776, trainable=1863456/3967232 ``` ## Frozen BatchNorm stops updating A frozen parameter still sits inside a module whose running statistics would keep moving. Every BatchNorm-style module whose parameters land in the frozen set is switched to eval mode, and the trainer re-applies that after each epoch's `model.train()` call, so the statistics stay fixed for the whole run. This is on by default and is what makes freezing a backbone actually freeze it. ## Composing with LoRA `freeze` and `lora=True` work together. On RF-DETR, DEIM and ConvNeXt the adapter parameters are preserved as trainable even when their parent group is frozen, which is the combination you want: a frozen backbone with adapters learning on top of it. See [LoRA fine-tuning](/docs/train/lora). ## Scope This is static freezing decided at startup. Scheduled unfreezing and progressive freezing are not part of the interface. ## Related - [Hyperparameters](/docs/train/hyperparameters) for the rest of `train()`. - [Distillation](/docs/train/distillation) for the other way to move a large model's knowledge into a training run. --- # Experiment loggers Every trainable family emits four training events. The built-in loggers are callback objects listening to those same events, so a backend integration and a custom hook use one interface. Verified against LibreYOLO v1.5.0. ## Turn a logger on `loggers=` takes a registered name, a configured instance, or an iterable mixing both. **By name** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") model.train(data="coco8.yaml", epochs=10, loggers="tensorboard") ``` **Configured instance** ```python from libreyolo import LibreYOLO from libreyolo.training import MLflowLogger model = LibreYOLO("LibreYOLO9s.pt") model.train( data="coco8.yaml", epochs=10, loggers=[MLflowLogger(tracking_uri="sqlite:///mlflow.db"), "tensorboard"], ) ``` Names are case-insensitive. The registered set is `tensorboard`, `mlflow`, `wandb`, `comet`, `clearml`, `neptune`, `dvclive` and `dvc`, the last being an alias for `dvclive`. Anything else raises immediately and lists the valid names. There is no value that enables all of them, and there is no CLI flag: `loggers=` is a Python argument. ## What every backend records All of them write the same metric names, so a dashboard looks the same whichever you pick: | Key | Value | |---|---| | `train/loss` | the epoch's mean training loss | | `train/loss/` | each loss component the family reports | | `lr/` | the learning rate of each optimizer parameter group | | `val/` | each validation metric, with its `metrics/` prefix stripped | | `time/epoch_seconds` | wall clock for the epoch | The step is the 1-based epoch. The fully resolved training configuration is logged as parameters at train start, and the run name defaults to `-`, for example `yolo9s-detect`. At train end the backends that support artifacts upload `results.csv`, `train_config.yaml` and `summary.json` when those exist, plus `weights/best.pt` with `log_checkpoints=True`. TensorBoard uploads nothing, because it has no artifact concept. No logger uploads validation plot images. ## Failure behavior A missing backend package raises at construction, naming the install command, because asking for a logger and silently getting nothing hides a bug. A backend failure during the run does the opposite. The first exception from a handler disables that logger for the rest of the run, logs it, tears the backend run down as failed, and training continues. A tracking server going down does not cost you the training. ## The backends Each needs its own extra. | Name | Extra | Constructor | |---|---|---| | `tensorboard` | `libreyolo[tensorboard]` | `TensorBoardLogger(log_dir=None)` | | `mlflow` | `libreyolo[mlflow]` | `MLflowLogger(tracking_uri, experiment_name, run_name, log_artifacts=True, log_checkpoints=False)` | | `wandb` | `libreyolo[wandb]` | `WandbLogger(project, name, entity, log_checkpoints=False)` | | `comet` | `libreyolo[comet]` | `CometLogger(project_name, workspace, name, api_key, online, log_artifacts=True, log_checkpoints=False)` | | `clearml` | `libreyolo[clearml]` | `ClearMLLogger(project_name="LibreYOLO", task_name, tags, output_uri, log_artifacts=True, log_checkpoints=False)` | | `neptune` | `libreyolo[neptune]` | `NeptuneLogger(project, api_token, name, run_id, tags, mode, capture_console=False, log_artifacts=True, log_checkpoints=False)` | | `dvclive`, `dvc` | `libreyolo[dvclive]` | `DVCLiveLogger(log_dir, resume, report, save_dvc_exp=False, dvcyaml=None, monitor_system=False, log_checkpoints=False)` | Import the classes from `libreyolo.training`. Backend-specific notes worth knowing before the first run: TensorBoard event files default to `/tensorboard`. View with `tensorboard --logdir runs/train`. MLflow 3.x deprecated the local `./mlruns` file store and raises unless `MLFLOW_ALLOW_FILE_STORE=true`. For server-less local tracking, pass a database URI instead, as in the snippet above, and read it with `mlflow ui --backend-store-uri sqlite:///mlflow.db`. Weights & Biases falls back to the `WANDB_PROJECT` environment variable and then to `libreyolo`. Comet falls back to `COMET_PROJECT_NAME` and then to `libreyolo`, and takes credentials from its own configuration; `online=False` gives an offline experiment. ClearML creates a fresh task, reports the config under `TrainConfig`, and disables automatic framework capture so metrics are not reported twice. Neptune uses the current `neptune-scale` client rather than the legacy package, and `mode="offline"` logs locally. DVCLive writes to `/dvclive`. It builds its summary tree from `/`, and cannot hold a float at a path that is also a parent, so `train/loss/box` is written as `train/loss.box` while `train/loss` keeps its name. LibreYOLO also turns off DVCLive's usual defaults of saving a DVC experiment and writing a root `dvc.yaml`, so an opt-in logger creates no version-control state outside the run directory; pass `save_dvc_exp=True` or an explicit `dvcyaml=` to get them back. Neptune is deliberately excluded from `libreyolo[all]`: its stable client requires protobuf below 7 while the TFLite extra requires protobuf 7. Install `libreyolo[neptune]` in an environment without the TFLite extra. ## Writing a callback The same four events drive everything. **A plain function** ```python from libreyolo import LibreYOLO from libreyolo.training import TrainEpochEvent def on_epoch(event: TrainEpochEvent) -> None: print(f"epoch {event.epoch}/{event.total_epochs} loss={event.train_loss:.4f}") model = LibreYOLO("LibreYOLO9s.pt") model.train(data="coco8.yaml", epochs=10, callbacks=on_epoch) ``` **An object with several hooks** ```python from libreyolo import LibreYOLO from libreyolo.training import TrainEndEvent, TrainEpochEvent, TrainStartEvent class RunLog: def on_train_start(self, event: TrainStartEvent) -> None: print(f"{event.model_family}{event.model_size} -> {event.save_dir}") def on_train_epoch_end(self, event: TrainEpochEvent) -> None: if event.is_best: print(f"new best at epoch {event.epoch}: {event.best_metric}") def on_train_end(self, event: TrainEndEvent) -> None: print(f"done in {event.total_seconds:.0f}s") model = LibreYOLO("LibreYOLO9s.pt") model.train(data="coco8.yaml", epochs=10, callbacks=RunLog()) ``` | Event | When | Carries | |---|---|---| | `TrainStartEvent` | after setup, before epoch 1 | `start_epoch`, `total_epochs`, `model_family`, `model_size`, `task`, `save_dir`, `config` | | `TrainEpochEvent` | after each epoch, training and validation | `epoch`, `train_loss`, `train_loss_items`, `lr`, `val_metrics`, `validated`, `is_best`, `current_metric`, `best_metric`, `best_epoch`, `epoch_seconds` | | `TrainEndEvent` | after training completes | `completed_epochs`, `final_loss`, `best_metric`, `best_epoch`, `total_seconds`, `results` | | `TrainExceptionEvent` | if training raises | `epoch`, `exception`, `exception_type`, `exception_message`, `elapsed_seconds` | A plain callable receives `TrainEpochEvent` only. An object may implement any subset of `on_train_start`, `on_train_epoch_end`, `on_train_end` and `on_train_exception`; missing methods are skipped. `TrainStartEvent.config` is the fully resolved configuration, user kwargs merged with family defaults, as a read-only mapping. The events are frozen dataclasses and their mappings are read-only, so a callback cannot change the run by writing to one. An exception raised from `on_train_start`, `on_train_epoch_end` or `on_train_end` propagates and ends the run. Only `on_train_exception` is guarded, so it cannot mask the original failure. Under multi-GPU training, callbacks fire on rank 0 only. With the automatic DDP spawn they also have to be picklable, which means a module-level class or function rather than a closure or a lambda. See [Multi-GPU training](/docs/train/multi-gpu). ## What every run writes anyway Three files land in the run directory with no configuration at all, on every family: | File | Written | Contents | |---|---|---| | `status.json` | atomically, every epoch and on start, end and failure | `state` of `running`, `completed` or `failed`, `current_epoch`, `total_epochs`, `progress`, `eta_seconds`, latest `metrics`, `best_metric`, `best_epoch`, and an `error` object on failure | | `metrics.jsonl` | appended once per epoch | one JSON row per epoch, the same schema as `results.csv` | | `train.log` | live | the run's console output | `status.json` is the cheap read for a script or an agent polling a run, and the atomic write means a reader never sees a half-written file. `results.csv` and `summary.json` are separate and family-gated. They are written for YOLOv9, YOLOv9-E2E, YOLOv9-P2, YOLOv7, YOLO-NAS, RF-DETR, EC and DINOv2, and not for the other families. `results.csv` gets one row per epoch with the loss components, validation metrics and learning rates as columns, and its header widens when a new column appears. On a resume it is trimmed back to the rows before the resumed epoch rather than duplicating them. Alongside those, the trainer always writes `train_config.yaml` at setup and the checkpoints under `weights/`. ## Watch a run live **Watch a run in the browser** ```bash libreyolo monitor # the most recent run under runs/ libreyolo monitor runs/train/exp # a specific run ``` `libreyolo monitor` serves a browser dashboard over the files above using only the standard library: metric charts, the log tail, and any validation images, refreshing while the run is active. It is read-only and never touches the training process, so it attaches to a live run, reopens a finished one, or inspects a crashed one. ## Related - [Validation and metrics](/docs/train/validation) for what the `val/` keys mean and how to add a validation loss. - [Training performance](/docs/train/performance) for the profiler, which is a different tool with a different question. --- # LoRA fine-tuning LoRA freezes the pretrained heavy parts of a model and trains small low-rank adapters beside them, plus the layers that must stay dense. In LibreYOLO the whole public interface is one boolean. Verified against LibreYOLO v1.5.0. ## Install LoRA rides on the optional `peft` dependency. **pip** ```bash pip install "libreyolo[lora]" ``` Without it, `lora=True` raises an `ImportError` naming that command rather than training a full fine-tune by accident. ## Use it **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreRFDETRs.pt") model.train(data="my-dataset.yaml", epochs=50, lora=True) ``` **CLI** ```bash libreyolo train model=LibreRFDETRs.pt data=my-dataset.yaml \ epochs=50 lora=true ``` `lora=True` is the entire interface. Rank, alpha, dropout and target modules are fixed per family to match each upstream reference, and are not user-facing knobs. A family that does not support LoRA raises at setup rather than ignoring the flag: ```text LoRA fine-tuning (lora=True) is not supported for yolo9. LoRA targets transformer components with nn.Linear layers (e.g. RF-DETR, D-FINE, DEIM). ``` The CLI rejects it earlier, before the model is built, using its own allowlist of the same nine families. ## Which families RF-DETR, D-FINE, DEIM, DEIMv2, RT-DETR v1, v2 and v4, EC and ConvNeXt. The gate is the `supports_lora` attribute on each family's trainer class, and the CLI carries a matching allowlist. Task coverage is narrower than family coverage. D-FINE and EC support detection only, and their segment and pose paths raise. RF-DETR's semantic path raises. ConvNeXt is classification. Everything else raises. There is no partial or silent mode. ## What each recipe does The recipes differ because the architectures differ, and a recipe that works on a ViT backbone has nothing to attach to on a convolutional one. RF-DETR uses DoRA, weight-decomposed LoRA, at rank 16 and alpha 16 on the DINOv2 backbone's attention `query`, `key` and `value` projections, matching the RF-DETR reference. The ViT backbone freezes; the projector, decoder and detection head keep training normally. D-FINE, DEIM and RT-DETR v1, v2 and v4 pair a convolutional backbone with a transformer hybrid encoder and a deformable decoder, so the split moves. The convolutional backbone freezes entirely, which also skips its backward pass. The transformer blocks freeze their base weights and train plain LoRA adapters at the same rank 16 and alpha 16 on their linear layers: the feed-forward `linear1` and `linear2`, the gate, and the deformable attention projections. Everything else, the encoder convolution fusion, the input projections, the prediction heads and the query embeddings, keeps training densely. Two details in that recipe are deliberate. Decoder self-attention stays frozen without adapters, because PyTorch's `nn.MultiheadAttention` reads `out_proj.weight` directly and would silently bypass an injected adapter. And it is plain LoRA rather than DoRA, because several decoder linear layers are zero-initialized by design and DoRA's magnitude normalization divides by the weight norm. DEIMv2 takes the same recipe with its SwiGLU feed-forward layers `w12` and `w3` as the targets. Its S, M, L and X sizes also carry a DINOv3 ViT backbone, where the ViT base freezes and its fused attention `qkv` layers get adapters, while the Spatial Tuning Adapter convolution pyramid keeps training as the projector analog. Those `qkv` adapters go in even when the config shipped the ViT frozen, since adapting a frozen backbone is the point. The sub-S sizes use a convolutional backbone and take the plain recipe. EC is a DETR whose backbone is a ViT surrounded by a trainable convolution projector pyramid. The ViT base freezes and its `qkv` layers get adapters, the transformer blocks take the shared recipe, and the projector and heads stay dense. ConvNeXt blocks carry channels-last linear MLPs, `fc1` and `fc2`, and those take plain adapters. The depthwise convolutions, the norms and the layer-scale parameters freeze. The classification head stays dense so custom class counts keep working. The detection and classification heads always stay trainable across every recipe, because a custom class count needs a freshly trained head. ## Checkpoints and export `best.pt` and `last.pt` keep the adapter tensors, so a LoRA run resumes or gets inspected like any other. Loading one of those checkpoints needs the `lora` extra installed, because the loader replays the adapter injection so the keys line up. `export()` merges the adapters into dense weights, so an exported artifact carries no dependency on `peft`. The same merge is available directly for an in-memory model. **Export merges the adapters** ```python from libreyolo import LibreYOLO model = LibreYOLO("runs/train/exp/weights/best.pt") model.export(format="onnx") ``` **Merge in place** ```python from libreyolo import LibreYOLO from libreyolo.training.lora import merge_lora_adapters model = LibreYOLO("runs/train/exp/weights/best.pt") merged = merge_lora_adapters(model.model) print(f"{merged} adapter layers folded into dense weights") ``` After a merge the module tree is fully dense and a second merge is a no-op. ## What it saves, and what it does not LoRA cuts optimizer and gradient memory, and on the families that freeze their backbone outright it also skips that backbone's backward pass. Activation memory is unchanged. Forward activations still have to be retained for whatever remains trainable, and that is usually what sets the peak. For the tightest VRAM budget, lower `batch` or `imgsz` as well. ## Related - [Layer freezing](/docs/train/layer-freezing) for the other way to train a subset of the weights, which works on every family and needs no extra dependency. `freeze` and `lora=True` compose: adapter parameters stay trainable even when their parent backbone group is frozen. - [Hyperparameters](/docs/train/hyperparameters) for `batch`, `imgsz` and the rest of `train()`. --- # Multi-GPU training Multi-GPU training in LibreYOLO is PyTorch DistributedDataParallel: one process per GPU, each holding a full model replica and a shard of every batch, with gradients averaged across ranks at each step. Verified against LibreYOLO v1.5.0. ## Run on two GPUs Pass a device list. Nothing else changes. **Python** ```python from libreyolo import LibreYOLO # The __main__ guard is required: each spawned worker re-imports this # module, and without the guard it would relaunch training recursively. if __name__ == "__main__": model = LibreYOLO("LibreYOLO9s.pt") model.train( data="my-dataset.yaml", epochs=100, batch=32, # global batch: 16 images per GPU on two GPUs device="0,1", ) ``` Given more than one device and no torchrun environment, the model's `train()` saves the weights to a temporary file, resolves autobatch if requested, and spawns one worker process per GPU with `torch.multiprocessing.spawn`. Each worker re-imports the model class, rebuilds it from the saved weights, and runs the ordinary single-device path, because from inside a spawned worker the torchrun environment variables are set. Rank 0's best checkpoint is loaded back into the caller's model instance when the run finishes. `device` accepts `"0,1"`, `[0, 1]`, `0`, `"cuda:0"`, `"cpu"`, `"mps"` and `"auto"`. Only a list of more than one CUDA index triggers the spawn. ## The `__main__` guard is mandatory Spawned workers re-import the module they came from. Without a `if __name__ == "__main__":` guard, that import re-executes the training call and each worker spawns its own workers. The library detects the case and raises rather than letting it recurse: ```text spawn_ddp_train() was called from inside a spawned subprocess. This usually means your script calls model.train(device=...) at the top level without a 'if __name__ == "__main__":' guard. ``` Everything crossing into a worker is pickled, so `callbacks=` has to be picklable. A module-level class works; a closure or a lambda does not, and the error says so and points at the built-in loggers as the alternative. ## batch is the global batch `batch` is the number of images per optimizer step across all GPUs. Each rank's dataloader is built at `batch // world_size` with a `DistributedSampler`, so `batch=32` on two GPUs means 16 images per GPU, not 32. A batch that does not divide evenly by the world size raises rather than quietly training at a different size: ```text batch=6 is the global batch and must be divisible by world_size=4: each rank trains at batch // world_size, so this value would silently train at a different global batch than requested. Use batch=4 or batch=8. ``` Gradients are averaged by DDP itself, so the loss is passed through unscaled. Multiplying it by the world size on top of that would inflate the effective learning rate by roughly the number of GPUs. ## Autobatch under DDP `batch=-1` works, and returns a world-size-divisible global batch. **Python** ```python from libreyolo import LibreYOLO if __name__ == "__main__": model = LibreYOLO("LibreYOLO9s.pt") # Probed once on GPU 0, scaled to a world-size multiple. model.train(data="my-dataset.yaml", batch=-1, device="0,1") ``` On the spawn path the probe runs in the parent process on the first device before any worker exists, so every worker receives a concrete integer and no inter-process coordination is needed. Under torchrun, rank 0 probes and broadcasts the result as a single long tensor. The probe measures one GPU's capacity and multiplies by the world size. When `nbs` is set, the global batch is capped at `nbs` and rounded down to a multiple of the world size, so adding GPUs reduces the number of accumulation steps rather than shrinking the per-GPU batch. The mechanics of the probe itself are on [Hyperparameters](/docs/train/hyperparameters). ## SyncBatchNorm Under DDP each rank's BatchNorm layers see only its own shard. At `batch // world_size` that shard can be small enough for the running statistics to degrade the converged model against a single-GPU run. `sync_bn=True` converts every BatchNorm to SyncBatchNorm so the statistics are computed across the global batch. The conversion only happens when distributed is active, so a single-GPU run is unaffected by the flag either way. It is already on by default for the BatchNorm-heavy convolutional families: YOLOX, YOLOv7, YOLOv9 and its variants, YOLO-NAS, PicoDet, RTMDet and FOMO. Every other family defaults it off. When a model contains BatchNorm, `sync_bn` is off and the per-rank batch is below 16, the trainer warns. **Python** ```python from libreyolo import LibreYOLO if __name__ == "__main__": model = LibreYOLO("LibreRTDETRr18.pt") model.train( data="my-dataset.yaml", batch=32, device="0,1", sync_bn=True, ) ``` There is no CLI flag for `sync_bn`. It is a Python argument. ## Launching with torchrun torchrun works too, and is the right choice when a cluster scheduler already owns process launch. Write the script for a single device and let torchrun set the rank environment. **train.py** ```python from libreyolo import LibreYOLO if __name__ == "__main__": model = LibreYOLO("LibreYOLO9s.pt") model.train(data="my-dataset.yaml", epochs=100, batch=32) ``` **Launch** ```bash torchrun --nproc_per_node=2 train.py ``` Do not combine the two. With the torchrun environment present, `device="0,1"` does not spawn; the trainer takes `cuda:LOCAL_RANK` and torchrun owns the process count. ## Rank behavior Rank 0 owns every side effect. It resolves the run directory and broadcasts the resolved name so all ranks agree, writes checkpoints and artifacts, and fires the user callbacks and loggers. Other ranks train and contribute gradients. Each rank seeds its dataloader and augmentation RNG differently, derived from the configured `seed`, so the ranks do not draw identical augmentations. ## Platform and backend The backend is chosen automatically: NCCL when CUDA and NCCL are both available, Gloo otherwise. NCCL is not built on Windows, so Windows runs get Gloo without any configuration. The process group is initialized with a three hour timeout. ## What does not run under DDP - CUDA graph capture. `cuda_graph=True` logs one line and trains eager. See [Training performance](/docs/train/performance). - The training profiler. `profile=True` is ignored with a warning. Not every family supports the automatic spawn. Twenty-four do, covering the detection, classification, semantic and restoration families that train. A family without it, handed a multi-GPU device, raises an error naming the model API and the torchrun command rather than quietly training on one GPU. ## Related - [Hyperparameters](/docs/train/hyperparameters) for `batch`, `nbs` and resume. - [Experiment loggers](/docs/train/loggers) for the picklability constraint on callbacks. - [Cloud GPUs](/docs/train/cloud-gpus) for renting a multi-GPU box. --- # Training performance Three levers change how fast a training step runs: mixed precision, CUDA graph capture of the network's forward and backward, and whatever the profiler says is actually holding the step up. Verified against LibreYOLO v1.5.0. ## Measure before changing anything The three levers below fix different problems, and applying the wrong one changes nothing. The profiler says which problem you have. **Profile and keep training** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") # Profiles a short window of real steps, prints a verdict, then # continues the run with the hooks removed. model.train(data="my-dataset.yaml", epochs=100, profile=True) ``` **Measure only, then stop** ```bash # Sets no_aug_epochs=0 and runs just enough epochs to fill the window. libreyolo profile run coco128 --weights LibreYOLO9s.pt --size s ``` **Drill into the result** ```bash libreyolo profile summary runs/profile/prof/profile.json libreyolo profile phases runs/profile/prof/profile.json libreyolo profile kernels runs/profile/prof/profile.json --top 10 ``` `profile=True` measures a window of real training steps, five discarded then twenty measured by default, prints a report, writes its artifacts and then keeps training with the hooks dropped. It costs nothing when off, and it is ignored under distributed training. The report ends in one of four verdicts: | Verdict | Meaning | Levers | |---|---|---| | `dataloader` | the GPU waits on input data | more `workers`, `cache="ram"` or `"disk"`, lighter augmentation, larger batch | | `host / launch` | the GPU is fed too slowly, many tiny kernels | larger batch, CUDA graphs, fewer per-step host syncs | | `compute` | the GPU is saturated | AMP or bfloat16, or accept it | | `memory-pressure` | allocator thrash, VRAM at the edge | lower batch; utilization figures here are unreliable | The utilization number is kernel busy time over the unsynchronized step time. The window is deliberately split: the first half runs with no extra synchronization so the verdict reflects real overlap, and only the second half brackets each phase with a sync to attribute GPU time. Synchronizing every phase hands the dataloader workers slack and hides starvation, so the composition numbers are never used to pick the verdict. Four files land in the run directory: `timeline.html`, which opens in a browser by itself, `profile_trace.json` for Perfetto or Nsight, `profile_summary.json`, and `profile.json`, the self-contained one to copy around and feed back to the `libreyolo profile` subcommands. Two things about `profile run` are worth knowing. It sets `no_aug_epochs=0`, because the profiler measures epoch 0 and a short run with the default `no_aug_epochs` would profile the lighter no-augmentation dataloader rather than the one training actually uses. And `--repeat N` reports mean and standard deviation, which matters because a launch-bound step is noisy enough that a single run misleads; it writes per-trial directories `prof_1`, `prof_2` and so on, plus an aggregate `profile_repeat.json`. ## Mixed precision `amp=True` is the default for most families and runs the forward pass under CUDA autocast. `amp_dtype` chooses `float16` or `bfloat16`. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") model.train(data="my-dataset.yaml", amp=True, amp_dtype="bfloat16") ``` **CLI** ```bash libreyolo train model=LibreYOLO9s.pt data=my-dataset.yaml \ amp_dtype=bfloat16 ``` Float16 needs dynamic loss scaling and gets a live gradient scaler; bfloat16's wider exponent range does not, so its scaler is disabled. Four families ship with `amp=False`, D-FINE, DEIM, YOLO-NAS and FOMO, and the DEIM setting carries through to RT-DETRv4 by inheritance. D-FINE states the reason: its decoder clamps activations at 65504, the largest finite float16 value. The argument semantics, including what a bfloat16 request does on hardware without bfloat16 support, are on [Hyperparameters](/docs/train/hyperparameters). ## CUDA graphs `cuda_graph=True` captures the network's training forward and backward into a CUDA graph, removing per-step kernel launch overhead. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") model.train(data="my-dataset.yaml", epochs=100, cuda_graph=True) ``` **CLI** ```bash libreyolo train model=LibreYOLO9s.pt data=my-dataset.yaml \ epochs=100 cuda_graph=true ``` The flag is always safe to pass. A family, task or configuration that cannot be captured logs one line and trains eager, unchanged. Only the network is captured. The loss stays eager by design, because detection losses select with boolean masks, run Hungarian matching and branch on assignment results, none of which a graph can record. The optimizer step, gradient clipping, EMA update and learning-rate schedule stay eager too. That bounds the win by how much of a step is network, and the share varies widely. Measured on an RTX 5070 Ti at 640 px, batch 8: 84 percent of a YOLOv9-t step is network, 44 percent of a YOLOv7-b step, 31 percent of a YOLOX-t step and 26 percent of an RTMDet-t step. The last two spend most of a step inside their label assigners, so capturing the network helps them least. ### What it is worth Conditions for every figure below: RTX 5070 Ti, Windows, AMP, one process per arm from a shared saved state, replaying one real batch so the dataloader is out of the loop, fastest of 24 steps after warm-up. Detection at 640 px, classification at 224 px. Batch size is per row. | Family | Size | Batch | Eager | Graphed | Speedup | |---|---|---:|---:|---:|---:| | FOMO | s | 16 | 7.0 ms | 1.9 ms | 3.63x | | MobileNetV4 | s | 16 | 14.5 ms | 5.3 ms | 2.74x | | EfficientNetV2 | b0 | 16 | 29.0 ms | 11.9 ms | 2.44x | | YOLOv9 | t | 8 | 93.6 ms | 47.0 ms | 1.99x | | NAFNet | s | 8 | 132.5 ms | 105.5 ms | 1.26x | | PicoDet | s | 8 | 145.0 ms | 118.7 ms | 1.22x | | D-FINE | n | 4 | 185.3 ms | 159.2 ms | 1.16x | | RF-DETR | n | 4 | 276.3 ms | 239.8 ms | 1.15x | | YOLOX | t | 8 | 102.2 ms | 90.5 ms | 1.13x | | RTMDet | t | 8 | 149.7 ms | 136.2 ms | 1.10x | | YOLOv7 | b | 4 | 102.5 ms | 98.0 ms | 1.05x | Those isolate the GPU step. A complete fine-tune also pays for the dataloader and for validation. YOLOv9-t on a 406-image detection set, 20 epochs, batch 8, 640 px, 4 dataloader workers, on the same machine: 428.4 s wall clock eager against 367.7 s graphed, a 1.16x gain, with mAP50-95 of 0.6394 in both arms. Three things move these numbers. Small batches are launch-bound and large ones are compute-bound, so RT-DETR-r18 gains 1.19x at batch 2 and 1.04x at batch 8. Launch overhead is highest on Windows, and Linux gains are roughly a third to half of the table. And a dataloader-bound run sees no wall-clock change at all, which is why the profiler comes first. Capture engages the same way at `amp=False`, but fp32 kernels run longer, so a step is less launch-bound and most families gain less. On the same hardware, MobileNetV4-s at batch 16 goes from 2.74x under AMP to 3.61x at fp32, while YOLOv9-t at batch 8 goes from 1.99x to 1.69x and RT-DETR-r18 at batch 4 from 1.12x to 0.99x. ### Where capture applies | Task | Families | |---|---| | detect | yolo9, yolo9_p2, yolo9_e2e, yolox, yolo7, yolonas, picodet, rtmdet, rfdetr, dfine, deim, deimv2, rtdetr, rtdetrv2, rtdetrv4, ec | | classify | resnet, convnext, mobilenetv4, efficientnetv2 | | semantic | segformer, lingbotvision | | point | fomo | | restore | nafnet | Everything else falls back to eager with one log line: other tasks on those families, families not listed, distributed runs and distillation runs. A capture failure at runtime also drops the rest of the run to eager rather than failing. For the encoder-decoder detectors, D-FINE, DEIM, DEIMv2, RT-DETR v1, v2 and v4, and EC, only the backbone and encoder are captured. Their decoder reads the ground truth to build contrastive-denoising queries, and the number of those queries follows the largest ground-truth count in the batch, so its token count changes from batch to batch. ### Shapes A graph is valid for exactly the input shape it was captured with. The trainer counts batch shapes and captures once a shape has repeated three times. Batches at any other shape run eager: multi-scale batches, and the last partial batch of an epoch. This is the trap for the DETR families, which resize every batch by default. With `multi_scale=True` a short run may never see one shape often enough to capture at all. Pass `multi_scale=False` when the speedup is the point. YOLOX changes what the captured region computes partway through a run, turning on its L1 regression branch when mosaic closes at `no_aug_epochs`. The trainer invalidates the capture there and re-captures once the new shape settles. ### Numerics and memory Most families reproduce their eager loss trajectory bit for bit under AMP. FOMO and LingBot-Vision differ in the last bit of float32 from a different summation order. The deformable-attention detectors, D-FINE, DEIM, DEIMv2, RT-DETR, RF-DETR and EC, do not reproduce their own eager runs either, because that backward accumulates with atomics and TF32 convolutions pick a reduction order per launch; the graphed run stays inside that spread. RTMDet differs by roughly 3e-4 relative on two of 139 gradients, because it shares head convolutions across pyramid levels and the two backward paths sum three contributions in a different order. SegFormer has stochastic depth inside the captured region, so a replayed graph draws its own random stream and is statistically equivalent to eager rather than identical; the manager logs that once at capture time. At `amp=False` bit-identical is not available from anything on this hardware, with or without capture. Two identical seeded eager YOLOv9-t runs diverge by 36 percent relative over 20 steps and YOLOX-t by 2.6 percent, because cuDNN picks a nondeterministic weight-gradient algorithm for some fp32 convolution shapes. A captured graph pins static input, output and workspace buffers, so peak VRAM rises by roughly one extra set of activations. Across the families above, peak allocation moved between -5 and +19 percent. The relative cost is largest for the small classification models, whose activations are small to begin with: ResNet-18 at 224 px, batch 16, went from 0.48 GB eager to 0.57 GB graphed. If it pushes a run over the limit, lower the batch or leave the flag off. ## Related - [Hyperparameters](/docs/train/hyperparameters) for `batch`, `nbs`, `cache` and `workers`. - [Multi-GPU training](/docs/train/multi-gpu), where both CUDA graphs and the profiler are unavailable. - [CUDA graphs](/docs/reference/cuda-graphs) for the combined inference and training support matrix, the seam splits and the numerics contract. --- # Validation and metrics Validation runs a model over a dataset split through val() and returns a flat dictionary of metric keys and float values. The keys are literal strings, and which ones you get depends on the task, not the family. Verified against LibreYOLO v1.5.0. ## Run a validation `val()` takes the dataset and returns the metrics. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") metrics = model.val(data="coco8.yaml") print(metrics["metrics/mAP50-95"]) print(metrics["metrics/mAP50"]) print(metrics["speed/total_ms"]) ``` **CLI** ```bash libreyolo val model=LibreYOLO9s.pt data=coco8.yaml ``` **On another split** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") metrics = model.val(data="coco8.yaml", split="train", batch=4) print(metrics) ``` The return value is a plain `dict[str, float]`. Every key is literal, so read it by name rather than by position. The main arguments are `data`, `split`, `batch`, `imgsz`, `conf`, `iou`, `workers`, `device`, `augment`, `save_json` and `verbose`. `conf` defaults to `0.001` and `iou` to `0.6`, both far looser than prediction defaults, because a mAP sweep needs the low-confidence tail. `imgsz` defaults to the model's own input size rather than a fixed number. `split` accepts `val`, `test` or `train` and nothing else. Any other field of the validation config passes through as a keyword argument, including `save_dir`, `max_det`, `eval_max_det`, `half`, `amp_dtype`, `cache` and `save_plots`. ## Metric keys per task Detection returns the COCO family of numbers: ```text metrics/mAP50-95 metrics/mAP50 metrics/mAP75 metrics/mAP_small metrics/mAP_medium metrics/mAP_large metrics/AR1 metrics/AR10 metrics/AR100 metrics/AR_max_det metrics/AR_small metrics/AR_medium metrics/AR_large metrics/precision metrics/recall metrics/precision(B) metrics/recall(B) metrics/mAP50(B) metrics/mAP50-95(B) ``` Two of those are traps. `metrics/precision` and `metrics/recall` are aliases held for backward compatibility: they carry the mAP 50-95 and AR@100 values, not a precision and recall pair. Use the named keys. Instance segmentation returns the mAP and AR figures above as mask numbers under the unsuffixed keys, with the box versions under a `(B)` suffix and the mask versions repeated under `(M)`. Precision and recall exist only in suffixed form for this task, as `metrics/precision(B)`/`metrics/recall(B)` and `metrics/precision(M)`/`metrics/recall(M)`, and both pairs carry the same alias values as detect's: the `(B)` pair is box mAP50-95 and box AR@100, the `(M)` pair is mask mAP50-95 and mask AR@100. | Task | Keys | |---|---| | detect | `metrics/mAP50-95`, `metrics/mAP50`, `metrics/mAP75`, plus the size and recall breakdowns above | | segment | mask versions of the detect keys above (unsuffixed keys are mask); `precision`/`recall` exist only as `(B)`/`(M)`, both aliased the same way | | pose | `metrics/keypoints_mAP50-95`, `metrics/keypoints_mAP50`, `metrics/keypoints_mAP75`, `metrics/keypoints_mAP_M`, `metrics/keypoints_mAP_L`, and the matching `keypoints_AR` keys | | obb | `metrics/mAP50-95`, `metrics/mAP50`, `metrics/mAP75`, `metrics/precision`, `metrics/recall`, plus `(OBB)`-suffixed copies | | classify | `metrics/accuracy_top1`, `metrics/accuracy_top5` | | semantic | `metrics/mIoU`, `metrics/pixel_accuracy` | | panoptic | `metrics/PQ`, `metrics/SQ`, `metrics/RQ`, `metrics/PQ_things`, `metrics/PQ_stuff`, `metrics/categories` | | depth | `metrics/abs_rel`, `metrics/rmse`, `metrics/delta1`, `metrics/delta2`, `metrics/delta3` | | normal | `metrics/mean_angular_error`, `metrics/median_angular_error`, `metrics/within_11_25`, `metrics/within_22_5`, `metrics/within_30` | | edge | `metrics/ODS`, `metrics/OIS`, `metrics/best_threshold` | | restore | `metrics/PSNR`, `metrics/SSIM` | | matte | `metrics/MAE`, `metrics/Smeasure` | | ocr | `metrics/det_precision`, `metrics/det_recall`, `metrics/det_hmean`, `metrics/e2e_precision`, `metrics/e2e_recall`, `metrics/e2e_f1`, `metrics/rec_1-NED` | | point | `metrics/precision`, `metrics/recall`, `metrics/f1`, `metrics/MLE`, `metrics/MAE`, `metrics/RMSE`, plus a mAP sweep key | OBB's `metrics/precision` and `metrics/recall` are not aliases: they are the real precision and recall at IoU 0.50, taken at the loosest operating point (every prediction that survives `conf`, default `0.001`). The `(OBB)`-suffixed copies repeat the same four values under a task-specific name, the same convention as `(B)` and `(M)` above. `accuracy_top5` is really top-`min(5, num_classes)`, so on a three-class dataset it is top-3, which every sample satisfies and which therefore reads 1.0. The point task's sweep key is built from the distance thresholds, so with the defaults it reads `metrics/mAP@[0.01:0.10]` and the single-threshold key reads `metrics/mAP@0.01`. Passing `dist_thresholds` changes both strings. Most tasks also return a `fitness` key, the single number best-checkpoint selection uses by default. Detection, segmentation and OBB do not carry one; their families are selected on `metrics/mAP50-95`, which their dicts do return. Pose returns neither `fitness` nor `metrics/mAP50-95`; its trainers set `best_metric_key` to `metrics/keypoints_mAP50-95` instead. ## Speed keys Every validator adds timing: ```text speed/preprocess_ms speed/inference_ms speed/postprocess_ms speed/total_ms speed/total_s speed/images_seen ``` These are per-image milliseconds averaged over the run. They describe the machine and settings you ran on, so a figure taken from them is only meaningful reported with its hardware, batch size and precision. ## Evaluation backend Detection and segmentation metrics are computed through a COCO evaluator, and `faster_coco_eval=True`, the default, selects the C++ backend when the `faster-coco-eval` package is installed. When it is not, the run falls back to pycocotools with one warning per process: ```text faster_coco_eval requested but not installed; falling back to pycocotools. Install with: pip install faster-coco-eval ``` Which backend actually ran is recorded on the model as `last_eval_backend`, and the CLI reports it in its output for detection-style tasks. Set `LIBREYOLO_FASTER_COCO_EVAL` to override the config value from the environment. `iou_thresholds` is honored only on the OBB path. The COCO path evaluates through its own fixed 0.50 to 0.95 sweep and ignores the value. ## Validation loss By default validation reports accuracy only. `val_loss=True` also computes the family's training objective on validation batches. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") model.train(data="coco8.yaml", epochs=10, val_loss=True) ``` It emits `metrics/loss` plus one `metrics/loss/` per term, weighted exactly as training weights them, so the components sum to the total. Through a logger they appear as `val/loss` and `val/loss/`, and `libreyolo monitor` overlays `metrics/loss` with `train/loss`. The components are the family's own: | Task | Families | Components | |---|---|---| | detect | `yolo9`, `yolo9_p2`, `yolo9_e2e` | `box`, `cls`, `dfl` | | detect | `yolonas` | `cls`, `iou`, `dfl` | | detect | `rfdetr` | `ce`, `bbox`, `giou` | | detect | `rtdetr`, `rtdetrv2` | `vfl`, `bbox`, `giou` | | detect | `dfine` | `vfl`, `bbox`, `giou`, `fgl`, `ddf` | | detect | `domedetr` | `vfl`, `bbox`, `giou`, `fgl`, `ddf`, `defe_density`, `defe_reg` | | detect | `deim`, `deimv2`, `rtdetrv4`, `ec` | `mal`, `bbox`, `giou`, `fgl`, `ddf` | | detect | `rtmdet` | `cls`, `bbox` | | detect | `picodet` | `cls`, `bbox`, `dfl` | | detect | `yolox` | `iou`, `obj`, `cls`, `l1` | | detect | `yolo7` | `iou`, `obj`, `cls` | | point | `fomo` | `ce` | | classify | `resnet`, `convnext`, `mobilenetv4`, `efficientnetv2` | `ce` | | semantic | `segformer`, `lingbotvision`, `dinov2` | `sem` | | restore | `nafnet` | `restore` | It is off by default because target assignment adds time and memory to validation. The validator reuses the model output already produced for the accuracy metric rather than running a second forward pass, it runs under `no_grad` on the evaluation or EMA model, and under multi-GPU training it is computed locally on rank 0 with no collectives. Best-checkpoint selection stays on the accuracy metric. Three things it deliberately does not do. It never includes contrastive-denoising terms, because those need the ground truth at forward time and validation forwards without it. It reports the evaluation-mode model, so where a family's train and eval forwards genuinely differ, in BatchNorm statistics or stochastic depth, the number reflects eval mode; that is the intended comparison. And a task a family has not implemented it for raises a configuration error at setup rather than quietly skipping: ```text val_loss=True currently supports RF-DETR detection only; segment, pose, OBB, classify, and semantic tasks are not supported ``` FOMO is the exception that changes nothing: its validator always computed this loss, and `val_loss=True` only affects which keys it is published under. Augmented validation and validation loss cannot be combined, and asking for both raises. ## Files a validation writes `val()` always writes `config.yaml` into its save directory, defaulting to `runs/val/__` when `save_dir` is not given. **Write COCO-format predictions** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") model.val(data="coco8.yaml", save_json=True, save_dir="runs/val/exp") ``` `save_json=True` writes `predictions.json` for detection, and `predictions_bbox.json` plus `predictions_masks.json` for segmentation. OBB does not support it and says so. `save_plots=True` writes into a `plots/` subdirectory. Detection gets `box_metrics.png`, per-class AP and recall charts, precision-recall and confidence curves, a confusion matrix, and annotated sample images when OpenCV is installed. Segmentation adds the mask-side copies of each, and pose gets its own metric and curve set. The other validators do not implement plots; classification, semantic, panoptic, depth, normal, edge, restore, matte, OCR, OBB and point all write nothing there. A plotting failure warns and never aborts the run. ## Validation during training Training validates every `eval_interval` epochs against the dataset's `val` split, and the metrics it produces are what drives `best.pt` selection, the `patience` early stop, and the `val/` keys in every logger. The validation runs on the EMA weights when EMA is on. See [Hyperparameters](/docs/train/hyperparameters) for `eval_interval`, `patience` and `save_plots`, and [Experiment loggers](/docs/train/loggers) for where the numbers go. ## Related - [Datasets](/docs/train/datasets) for the split keys and formats validators read. --- # Troubleshooting Errors grouped by the message you see. Two entries at the end cover the opposite problem: code that runs, returns something plausible, and is wrong. Verified against LibreYOLO v1.5.0. Errors are grouped by the text you see. If your message is not here, the [FAQ](/docs/faq) answers the questions that are not failures, and `libreyolo models` reports what your install can actually load. ## ModuleNotFoundError naming a package you never imported Some families need an optional extra. The message names the missing package rather than the extra, so the fix is not always obvious from the traceback. Run `libreyolo models`. Any family whose dependency is missing is printed with the exact pip command that enables it, so you do not have to map package back to extra yourself. `libreyolo models --json` prints the same as an object. The [install page](/docs/install) lists every extra and what it covers. ## ONNX inference requires onnxruntime ``` ImportError: ONNX inference requires onnxruntime. Install with: pip install onnxruntime ``` The base package does not depend on a runtime, because which one you want depends on your hardware. Install `onnxruntime` for CPU or `onnxruntime-gpu` for CUDA. Both provide the same `onnxruntime` module, so install one, not both. ## ONNX model not found ``` FileNotFoundError: ONNX model not found: ``` The path is resolved relative to the working directory, not the script. This also appears when an export silently wrote somewhere else: `export()` returns the path it wrote, so capture the return value rather than assuming a name. ## NotImplementedError from train() Not every family trains. Some are ported for prediction, validation and export only, and their `train()` raises rather than pretending to run. The [FAQ entry](/docs/faq) explains the reasoning. To check a specific family before writing a training script, its model page states whether it trains. ## NotImplementedError from export() A family can support a task and still not export it. EoMT is the case people hit: `export()` accepts the semantic task and raises for `segment` and `panoptic`, because the query-mask runtime contract those need is not defined. ``` NotImplementedError: LibreEoMT instance and panoptic export need query-mask runtime contracts. ``` Every family's page carries an export matrix showing which task and format combinations are validated. ## CUDA out of memory Reduce `batch` first, then `imgsz`. Both change memory roughly with their size, but batch is the one you can drop without changing what the model sees. If it fails at validation rather than training, validation runs its own batch size, so lower that too. On Windows, a display GPU has a second failure mode that looks like a random CUDA error rather than an out-of-memory: the driver resets a GPU that stops responding for longer than the timeout, killing whatever was running. Long kernels on the card driving your monitor can trip it. ## Weights will not download Weights fetch from Hugging Face on first use and cache locally. The [FAQ](/docs/faq) covers where the cache lives and how to run fully offline. If a download 404s, check the filename you passed. The URL is derived from it, including the task suffix, so a name that does not match a published checkpoint produces a URL that does not exist. The checkpoint table on each model page lists the exact published filenames. ## Training hangs or restarts on Windows Windows has no `fork`, so dataloader workers start by re-importing your script. Without a `if __name__ == "__main__":` guard, each worker re-runs your training call, which either deadlocks or spawns processes without end. ```python def main(): ... # build the model and call train() if __name__ == "__main__": main() ``` Setting `workers=0` also avoids it, at a throughput cost. The guard is the better fix. ## Two failures that do not raise The rest of this page is about errors. These two are worse, because the code runs and hands back something that looks right. ### Indexing a single result `predict()` returns one `Results` for one image, and a list for several. Indexing the single-image return selects a *detection*, not an image: ```python result = model.predict("image.jpg") # a Results result.boxes # every detection, correct result[0].boxes # ONE detection, silently ``` Nothing raises, because indexing a `Results` is a valid operation that returns a subset. Code written against the list form quietly reports one box per image. Index only what you know is a list. ### Reading metrics as attributes `val()` returns a plain dictionary keyed by metric name, not an object with attribute access: ```python metrics = model.val(data="coco8.yaml") metrics["metrics/mAP50-95"] # correct metrics.box.map # AttributeError ``` The keys are namespaced with `metrics/` and `speed/`. Print the dictionary once to see what your task produced, since the set differs by task. ## Checking a dataset before you train Most training failures are dataset problems. `libreyolo doctor data.yaml` runs health checks over a detection dataset and reports findings by severity, which is faster than reading a traceback from epoch one. ```python from libreyolo import doctor report = doctor.diagnose("data.yaml", imgsz=640) if report.errors: ... ``` See the [doctor command](/docs/cli/doctor) for the check catalog. --- # Upgrading to 1.5.0 Nothing was removed from the public model API: every class and function that worked in 1.4.0 still imports. Four arguments changed shape, and three defaults move numbers you may be comparing against. Verified against LibreYOLO v1.5.0. This page is about upgrading LibreYOLO itself. If you are looking for how to load a checkpoint from an upstream project, that is [import existing weights](/docs/migrate), a different subject. The full entry for the release is the [changelog](/docs/changelog). What follows is only the part that asks something of you. ## Code changes you must make ### `allow_experimental=True` no longer exists The acknowledgement gate is gone, along with the `ddp_aware(experimental_key=...)` mechanism behind it. EC, RTMDet, PicoDet and FOMO training and export previously required the argument, so any script that trains one of those families is affected. ```python # 1.4.0 model.train(data="data.yaml", epochs=100, allow_experimental=True) # 1.5.0: delete the argument model.train(data="data.yaml", epochs=100) ``` There is no deprecation shim. A call that still passes it raises `TypeError`. `BaseModel.EXPERIMENTAL_WEIGHT_FILENAMES` was removed with it. The `get_download_notice()` hook survives, and is still overridden by MiDaS, SegFormer and YOLO9-P2. Support levels are still published, they are just no longer an argument: see [stability tiers](/docs/reference/stability-tiers). ### The export tier `"experimental"` no longer exists ```python from libreyolo.export.support import Tier # 1.4.0: Literal["validated", "experimental", "blocked"] # 1.5.0: Literal["validated", "available", "blocked"] ``` Code branching on the tier string should read `"available"` where it read `"experimental"`. `BaseExporter` no longer emits a `RuntimeWarning` for those formats. The per-format state is listed in the [export matrix](/docs/reference/export-matrix). ### `pretrained=False` with `resume` is now rejected The combination previously proceeded incoherently. It now raises: ``` ValueError: pretrained=False cannot be combined with resume. ``` Pick one. `pretrained=False` starts from a fresh seeded initialization, which in 1.5.0 works for every trainable family rather than three of them, and `resume` continues an interrupted run from its checkpoint. Both are documented under [training](/docs/train). ### CLI `--imgsz` is a string, not an int Narrower than it sounds. Both of these are unaffected: ```bash libreyolo predict --model yolo9-t --source img.jpg --imgsz 640 # still fine ``` ```python model.predict("img.jpg", imgsz=640) # still fine ``` Only code that calls the [CLI](/docs/cli) command functions directly from Python needs to change, because `predict`, `train` and `val` widened `--imgsz` from `int` to `str` so it can accept rectangular sizes: ```python from libreyolo.cli.commands.predict import predict_cmd predict_cmd(..., imgsz=640) # 1.4.0 predict_cmd(..., imgsz="640") # 1.5.0, and "480x640" now works too ``` `train`'s default is now the string `"640"`. `export --imgsz` was already a string, and `profile` is unchanged. ## Numbers that change Three changes move metrics at default settings. If you track results across versions, read these before you compare a 1.5.0 run against a 1.4.0 one. ### faster-coco-eval is the default COCO metrics backend `val()` and per-epoch training validation now compute COCO metrics with the faster-coco-eval C++ backend instead of pycocotools. The switch was decided on measured parity across all 100 RF100-VL test splits: 1381 of 1400 metric values bit-identical, maximum deviation 2.22e-16, headline deltas exactly 0, at 15.6x faster overall and 56x on detection-dense datasets. Your numbers should not move. They are produced by a different implementation all the same, which is the reason this is on the list. pycocotools stays the automatic fallback when faster-coco-eval is not installed. To force it: ```bash libreyolo val --model yolo9-t --data coco.yaml --no-faster-coco-eval ``` ```python model.val(data="coco.yaml", faster_coco_eval=False) ``` `LIBREYOLO_FASTER_COCO_EVAL=0` does the same thing globally. The backend actually used is logged at INFO, exposed as `model.last_eval_backend` after `val()`, and included as `eval_backend` in the [CLI](/docs/cli/val) JSON payload. Install the fast path with `pip install libreyolo[fast-eval]`. ### YOLOX checkpoints trained before 1.5.0 need an eps override This is the trap in the release. Read it if you have fine-tuned [YOLOX](/docs/models/yolox). YOLOX specifies BatchNorm `eps=1e-3` and `momentum=0.03`. Until 1.5.0 those values were applied as a post-hoc fixup that did not survive the class-count rebuild `train()` performs when your dataset's `nc` differs from the checkpoint's. Such a fine-tune trained and reported in-training validation at torch's default `eps=1e-5`, then reloaded for inference at `1e-3`: the same tensors under different normalization. Regular-conv sizes barely move. Depthwise `n` moves a lot, because its per-channel `running_var` is small enough for eps to dominate. On RF100-VL `ball`, the same nano checkpoint scores **0.566** mAP50-95 evaluated at its trained eps and **0.151** after a stock reload. A checkpoint trained before 1.5.0 carries eps=1e-5 semantics. To report faithful numbers for it, either evaluate with BN eps overridden to 1e-5: ```python import torch from libreyolo import LibreYOLOX model = LibreYOLOX("my-yolox-finetune.pt") for module in model.model.modules(): if isinstance(module, torch.nn.BatchNorm2d): module.eps = 1e-5 model.val(data="data.yaml") ``` or fold `sqrt((var + 1e-3) / (var + 1e-5))` into the BN weights once and save the result. Checkpoints trained on 1.5.0 and later need neither. ### D-FINE multi-scale training uses the upstream per-size recipe `base_size_repeat` was hardcoded to 3 for every size. It now resolves per size as upstream specifies: **n** trains at fixed size with multi-scale off, **s** 20, **m** 6, **l** 4, **x** 3. Only x matched before, so n, s, m and l see a different scale distribution and converge to different metrics. To restore the old behavior, set it explicitly: ```python from libreyolo.training.config import DFINEConfig config = DFINEConfig(base_size_repeat=3) ``` DEIM still uses the hardcoded 3. Family details are on [D-FINE](/docs/models/d-fine). ## Worth knowing, no action needed - **Rectangular `imgsz` results changed because they were wrong before.** Box coordinates, RTMDet mask resizing, YOLO-NAS rescaling and validator ground-truth scaling now use per-axis height and width instead of one scalar. Square `imgsz` is bit-unchanged. Rectangular inference or validation run on 1.4.0 was mis-scaled. YOLO-NAS now rejects rectangular `imgsz` outright rather than silently producing wrong output. - **Metrics dictionaries gained keys.** `max_det`, `ar_max_det` and `AR_max_det` from the COCO evaluator, and `metrics/loss` plus `metrics/loss/ce` from FOMO. Values at defaults are unchanged, but anything iterating metric keys, including custom [loggers](/docs/train/loggers) and CSV headers, sees new columns. - **Seeded YOLO9 runs that trigger a head rebuild** start from a different initialization, because the seed is now applied before the rebuild rather than after. A seeded 1.4.0 fine-tune onto a different class count is not reproducible bit-for-bit on 1.5.0. - **`libreyolo[hub-kernels]` on CUDA now actually engages the native MS-deform-attn kernel.** 1.4.0 gated it behind a condition RF-DETR never took, so the kernel never ran. Predictions can shift at float tolerance for RF-DETR and the other deformable-attention families. Stock installs are unaffected, and `LIBREYOLO_HUB_KERNELS=0` disables it. - **`libreyolo predict` drops unsupported options instead of raising.** The CLI filters kwargs against the model's `__call__` signature, so an option a family does not accept is ignored rather than raising `TypeError`. A typo in a flag name is now silently ignored. - **Live sources change the JSON output shape.** Webcams, RTSP streams and screen capture implicitly enable streaming, which emits one record per frame rather than one for the call. These [sources](/docs/predict/sources) are new in 1.5.0, so no 1.4.0 script is affected. - **Re-exporting `rfdetr-pose` or `yolonas-pose` to ONNX yields different output names.** 1.4.0 misread their multi-tensor pose heads as segmentation through an output-count heuristic. Existing `.onnx` files on disk are untouched. - **On a torch-free install**, results hold numpy arrays rather than `torch.Tensor`, so `.boxes.data` returns a different type and NMS tie-breaking may differ from torchvision. With torch installed, behavior is byte-for-byte unchanged. See [lightweight install](/docs/lightweight-install). - **Config objects validate more at construction.** `TrainConfig` gained a `__post_init__` where it had none, so a config that was already invalid now raises immediately instead of failing deep into a run. `ValidationConfig` serialization gained an `edge_thresholds` key, which breaks a strict `ValidationConfig(**dump)` round-trip from a 1.4.0 dump. - **Weight filenames for task-suffixed families resolve differently.** `segformer-b0` now resolves to `LibreSegformerb0-sem.pt`. This fixes auto-download 404s, and breaks any script that hardcoded the old unsuffixed filename. - **The pytest marker `experimental_backend` is now `extended_backend`.** Only relevant if you run the test suite with `-m`. ## Checkpoints and datasets Checkpoints written by 1.4.0 load unchanged. The [schema](/docs/reference/checkpoint-schema) gained `imgsz_h` and `imgsz_w` for rectangular models, and still writes the scalar `imgsz = max(h, w)` for older readers. [ExecuTorch](/docs/export/executorch) and [MNN](/docs/export/mnn) exports now require a sidecar, `.pte.json` and `.mnn.json` respectively, and HRNet exports carry `pose_input: "person_crop"`. Dataset formats are unchanged. --- # Versions These pages describe one release of LibreYOLO on unversioned URLs. Documentation for earlier releases stays online at its own versioned path. Verified against LibreYOLO v1.5.0. ## The release these pages describe These pages describe LibreYOLO 1.5.0. The version sits at the top of the docs sidebar, and every page ends with a line naming the release it was verified against. These URLs carry no version number and do not move. `/docs/models/rf-detr` is the current release's page for that family, today and after the next release. Pages are revised in place when behavior changes, so the verified-against line at the foot of a page is what tells you how current the text is. ## Check what you have installed `libreyolo version` prints the version together with the Python, torch and CUDA versions it is running against, which is usually what you want when something behaves differently from the docs. `libreyolo --version` prints the bare number and exits. **CLI** ```bash libreyolo version ``` **Python** ```python import libreyolo print(libreyolo.__version__) ``` ## Documentation for earlier releases Each earlier release keeps the single-page documentation it shipped with, at its own path. | Release | Documentation | | --- | --- | | 1.5.0 | [/docs](/docs), these pages, released 2026-08-09 | | 1.4.0 | [/docs/v1.4.0](/docs/v1.4.0) | | 1.3.1 | [/docs/v1.3.1](/docs/v1.3.1) | | 1.3.0 | [/docs/v1.3.0](/docs/v1.3.0) | | 1.2.0 | [/docs/v1.2.0](/docs/v1.2.0) | | 1.1.0 | [/docs/v1.1.0](/docs/v1.1.0) | Those pages are a record of what the library did at that release. They are not revised, so an argument that has since been renamed or a default that has since moved is still described there the old way. If you are pinned to one of those versions, its page is the accurate reference and this tree is not. ## Pinning a release `pip install libreyolo` installs the latest release published to PyPI. Pin the version when a result has to be reproducible: ```bash pip install "libreyolo==1.4.0" ``` Installing from source, a plain clone checks out `release`, the stable branch whose code matches the published release. The `dev` branch carries work that has not been released yet, including anything listed under [changelog](/docs/changelog) as landing since the last version. ```bash git clone https://github.com/LibreYOLO/libreyolo.git cd libreyolo pip install -e . ``` ## What changes between releases Checkpoints move forward, not backward. A checkpoint written by a newer release can fail to load in an older one: the 1.4.0 notes record that checkpoints using the task strings introduced in that release, or carrying finalized quantization state, are not loadable by 1.3.1. Keep the version that wrote a checkpoint alongside the checkpoint itself. Defaults move too, and the changelog records each one with its reason. In 1.4.0 PicoDet's `lr0` went from 0.1 to 0.01, because the old default destroyed COCO-pretrained weights, and DEIM's went from 4e-4 to 1e-4. A training script that leaned on a default therefore reproduces differently across versions. Pass the values you care about explicitly and a run stays comparable. [Changelog](/docs/changelog) summarizes what landed in recent releases, and [`CHANGELOG.md`](https://github.com/LibreYOLO/libreyolo/blob/dev/CHANGELOG.md) in the repository carries the full entries. --- # Checkpoints and weights A LibreYOLO checkpoint is a torch.save dictionary holding a state dict plus the metadata needed to identify it. This page covers where those files come from, where they land, and how they are loaded. Verified against LibreYOLO v1.5.0. ## Where a checkpoint is looked for A model reference with no directory component, such as `LibreYOLO9t.pt`, is resolved against `weights/` relative to the current working directory. If `weights/LibreYOLO9t.pt` exists it is used; if a file of that name exists in the working directory itself it is used instead; otherwise `weights/LibreYOLO9t.pt` becomes the download target. A reference that does contain a directory, absolute or relative, is taken literally. That is the form to use when weights live somewhere central and nothing should be fetched. **Auto-download** ```python from libreyolo import LibreYOLO, SAMPLE_IMAGE # A bare filename resolves to weights/LibreYOLO9t.pt and is # downloaded there if it is not already present. model = LibreYOLO("LibreYOLO9t.pt") print(model(SAMPLE_IMAGE).boxes) ``` **Explicit path** ```python from libreyolo import LibreYOLO # A path with a directory component is used exactly as written and # is never fetched from the network. model = LibreYOLO("/opt/models/LibreYOLO9t.pt") print(model.family, model.size, model.task) ``` ## Auto-download When the resolved path does not exist, LibreYOLO parses the filename to recover the family, the size and the task, and asks the matching family for a download URL. Most families build it from the LibreYOLO organization on Hugging Face, where each checkpoint has its own repository named after the file: ```text https://huggingface.co/LibreYOLO//resolve/main/.pt ``` A dataset-variant suffix stays part of the repository name, so a checkpoint trained on something other than the family default resolves to its own repository rather than overwriting the default one. The transfer itself is defensive, because a truncated weight file fails later with an unhelpful error. Downloads are streamed to a `.part` file and moved into place atomically only when complete, so an interrupted process can never leave a half-written checkpoint at the final path. An interrupted transfer resumes from its byte offset using an HTTP validator, and restarts from zero if the server indicates the object changed. Failures are retried three times with exponential backoff. Concurrent processes targeting the same path take a lock file, so two training runs starting together download once. Where a family fetches from a third-party host rather than the LibreYOLO organization, it can pin a checksum and refuse the file on mismatch. If `HF_TOKEN` is set, or a token is cached at `~/.cache/huggingface/token`, it is attached as a bearer token. It is attached only to `huggingface.co` URLs, so a family that downloads from another host never receives it. Not every family auto-downloads. Some deliberately return no URL because the released weights may not be redistributed, and the error then explains what to supply instead. Others print a license notice before the transfer starts. That notice is the runtime signal that a checkpoint's terms are narrower than the code's, and it is worth reading rather than scrolling past. ## The Hugging Face organization Published weights live at [huggingface.co/LibreYOLO](https://huggingface.co/LibreYOLO), one repository per checkpoint. Each repository carries a license, and the license is not uniform across a family: a family whose code is MIT can have some weights that are not. The repository is authoritative. Every model page lists that family's published checkpoints and their licenses under its Checkpoints and Licensing sections. ## Working offline Nothing about the library requires network access once the files are local. Two approaches work: Pre-populate a `weights/` directory next to wherever the job runs. Fetching the checkpoints once on a connected machine, then copying the directory, is enough; the resolution step above finds them and never reaches the network. Or pass an absolute path to a shared location. A reference with a directory component is used as given, so a read-only mount of curated weights is a valid setup. If the process cannot write next to a checkpoint it needs to convert, conversion falls back to a private temporary directory instead of failing. Datasets follow a separate rule: they resolve under `~/datasets`, or under the directory named by `LIBREYOLO_DATASETS_DIR` when that variable is set. ## Loading safety Checkpoints are pickles, and a pickle can execute arbitrary code when it is opened. LibreYOLO treats every weight file as untrusted and loads it with PyTorch's `weights_only=True` path, which restricts the unpickler to tensors and a small set of safe types. This applies to the file you pass, not only to files LibreYOLO downloaded. On a PyTorch build too old to support that argument, the load is refused rather than performed unsafely. Some upstream training checkpoints embed objects the restricted unpickler rejects, such as a configuration object from the framework they were trained with. Those objects are metadata that LibreYOLO does not need, so during conversion each blocked class is replaced by an inert stand-in that satisfies the unpickler without running anything, and only tensors survive into the converted file. Sensitive module names are refused outright rather than stubbed, and the retry loop is bounded so a file engineered to introduce an endless series of blocked classes fails closed. See [import existing weights](/docs/migrate) for the rest of that path. ## Checkpoint metadata A LibreYOLO checkpoint is a dictionary whose `model` key holds the PyTorch state dict. Nine keys are required by schema v1.0, and together they let the factory identify a file without parsing its name or guessing from tensor shapes. | Key | Meaning | |---|---| | `model` | The PyTorch state dict | | `schema_version` | The metadata contract version. v1.0 uses the string `1.0` | | `libreyolo_version` | The LibreYOLO version that produced the file | | `model_family` | A registered family identifier, such as `yolo9` | | `size` | The variant within that family, such as `t` or `r18` | | `task` | One canonical task name | | `nc` | A positive class count | | `names` | A mapping of class index to label, covering `0` to `nc - 1` | | `imgsz` | A positive input resolution | Tasks with extra structure record it alongside those keys. Pose checkpoints add `num_keypoints` and `keypoint_dim`, and may add per-keypoint OKS sigmas. OCR checkpoints embed the full CTC charset so the file is self-contained. Restore checkpoints may record the degradation type and an upscale factor. Trainer checkpoints add resume state such as `epoch`, the optimizer state and the EMA weights; published inference weights should not carry that. A file that satisfies all nine keys loads through the metadata path. A file that does not is either converted, if a family recognizes its layout, or loaded through the compatibility path with a warning naming what is missing. ## Inspecting a checkpoint **CLI** ```bash # Reads the metadata without constructing a model, and reports # whether it satisfies the schema. libreyolo metadata path=weights/LibreYOLO9t.pt ``` **JSON** ```bash libreyolo metadata path=weights/LibreYOLO9t.pt --json ``` **Python** ```python from libreyolo.utils.serialization import ( load_untrusted_torch_file, validate_checkpoint_metadata, ) loaded = load_untrusted_torch_file("weights/LibreYOLO9t.pt") # Returns a list of problems. Empty means the file satisfies v1.0. print(validate_checkpoint_metadata(loaded)) print(loaded["model_family"], loaded["size"], loaded["task"], loaded["nc"]) ``` `libreyolo metadata` never constructs a model, so it works on a file whose family is not installed and on a file you are not sure about.