Two companion guides go deeper on specialized topics. The LibreVLM guide covers the vision-language tier (Qwen3-VL, Florence-2), which generates text that LibreYOLO parses into boxes. That is a different thing from open-vocabulary detection, new in v1.3.1, which uses purpose-built detectors conditioned on text and is documented on this page. The experimental tasks guide covers additional experimental workflows, including LoRA / DoRA fine-tuning.

1
from libreyolo import LibreYOLO, SAMPLE_IMAGE
2
3
# Default: YOLO9 detection
4
model = LibreYOLO("LibreYOLO9c.pt")
5
result = model(SAMPLE_IMAGE, conf=0.25, save=True)
6
7
print(f"Detected {len(result)} objects")
8
print(result.boxes.xyxy)
9
print(result.saved_path)

Breaking changes in v1.3.0

• DAMO-YOLO removed with no alias: LibreDAMOYOLO raises AttributeError, and DAMO-YOLO checkpoints are rejected on load.
• YOLO9 is detect-only. The -seg, -pose, -cls, -obb, and -sem YOLO9 variants were removed.
• RF-DETR lost classify, semantic, and depth. Its tasks are now detect, segment, pose, and obb. Classification and semantic moved to the new LibreDINOv2 family; depth moved to the new LibreDepthAnythingV2 family.
• TFLite export needs Python 3.12+ (onnx2tf wheel constraint). The onnx2tf floor was raised to >=2.4.3 and the old onnx2tf runtime patches were removed.

Use this matrix as the quick v1.3.1 support map. ✓ marks a validated path, exp is experimental, prev is a research preview, and empty cells are not currently supported. Only YOLO9 and RF-DETR detection (plus RF-DETR segmentation) are heavily tested; everything else, including the new classification, semantic, depth and point families, is experimental.

Depth Anything V2 has no export path. TFLite export is experimental and limited to YOLO9 detection and RF-DETR detect / segment / pose. The classification families (MobileNetV4, ConvNeXt, EfficientNetV2) support ONNX export. CoreML exports produce .mlpackage bundles and require libreyolo[coreml]: macOS only, no INT8, and no embedded NMS for RF-DETR, D-FINE, DEIM, DEIMv2, or EC.

Requirements

• Python 3.10+
• PyTorch 1.13+ and torchvision 0.11+

From PyPI

1
pip install libreyolo

v1.3.1 is the current release on PyPI, and it is what these docs describe. Everything on this page works from the published package: you do not need a source install.

From source

1
git clone https://github.com/LibreYOLO/libreyolo.git
2
cd libreyolo
3
pip install -e .

Optional dependencies

1
# ONNX export and inference
2
pip install libreyolo[onnx]
3
# or: pip install onnx onnxsim onnxruntime
4
5
# RT-DETR compatibility extra (currently no extra packages)
6
pip install libreyolo[rtdetr]
7
8
# RF-DETR support
9
pip install libreyolo[rfdetr]
10
# or: pip install transformers
11
12
# TensorRT export and inference (NVIDIA GPU)
13
pip install libreyolo[tensorrt]
14
# Installs TensorRT CUDA 12 Python packages on Linux/Windows.
15
# Host driver/CUDA compatibility still matters.
16
17
# OpenVINO export and inference (Intel CPU/GPU/VPU)
18
pip install libreyolo[openvino]
19
# INT8 export also needs: pip install nncf
20
21
# NCNN export and inference
22
pip install libreyolo[ncnn]
23
# or: pip install pnnx ncnn
24
25
# ByteTrack API compatibility extra
26
pip install libreyolo[tracking]
27
# Tracking dependencies are part of the base install in v1.3.1.
28
29
# CoreML export and inference (macOS only for runtime)
30
pip install libreyolo[coreml]
31
# or: pip install coremltools
32
33
# L2CS gaze optional auto-download helper
34
pip install libreyolo[gaze]
35
36
# Promptable segmentation (LibreSAM: SAM-1, SAM-2, MobileSAM)
37
pip install libreyolo[sam]
38
39
# Open-vocabulary detection (Grounding DINO, OWLv2)
40
pip install libreyolo[openvocab]
41
42
# LibreLabel AI assist (SAM click-to-mask)
43
pip install libreyolo[label]
44
45
# Zero-shot classification (CLIP)
46
pip install libreyolo[clip]
47
48
# Install every optional LibreYOLO extra
49
pip install libreyolo[all]

If using uv, the most reliable path is an isolated venv per extra:

1
# ONNX environment
2
uv venv .venv-onnx
3
uv pip install --python .venv-onnx/bin/python -e '.[onnx]'
4
5
# RT-DETR environment
6
uv venv .venv-rtdetr
7
uv pip install --python .venv-rtdetr/bin/python -e '.[rtdetr]'
8
9
# Repeat with .[rfdetr], .[openvino], .[ncnn], .[coreml], .[gaze], .[tracking], or .[tensorrt] as needed

This avoids mutating the project environment and keeps optional dependencies isolated. Vendor-specific extras such as TensorRT, OpenVINO, NCNN, and CoreML may still require platform-specific native packages.

For the most tested path, pick single-GPU YOLO9 detection, RF-DETR detection, or RF-DETR segmentation. They load through the same factory, accept the same inputs, and return the same Results object, so you can swap between them without changing surrounding code.

YOLO9 - CNN flagship

1
from libreyolo import LibreYOLO, SAMPLE_IMAGE
2
3
# Use the official checkpoint name and let the factory resolve the details
4
model = LibreYOLO("LibreYOLO9c.pt")
5
6
# Run on a single image (SAMPLE_IMAGE ships with the package)
7
result = model(SAMPLE_IMAGE)
8
9
print(f"Found {len(result)} objects")
10
print(result.boxes.xyxy)   # bounding boxes (N, 4)
11
print(result.boxes.conf)   # confidence scores (N,)
12
print(result.boxes.cls)    # class IDs (N,)

RF-DETR - transformer flagship

1
from libreyolo import LibreYOLO, SAMPLE_IMAGE
2
3
# Same factory, same call shape - just point at an RF-DETR checkpoint
4
model = LibreYOLO("LibreRFDETRs.pt")
5
result = model(SAMPLE_IMAGE)
6
7
print(f"Found {len(result)} objects")
8
print(result.boxes.xyxy)

Save annotated output

1
result = model(SAMPLE_IMAGE, save=True)
2
print(result.saved_path)   # e.g. runs/detect/predict/parkour.jpg

Process a directory

1
results = model("images/", save=True, batch=4)
2
for r in results:
3
    print(f"{r.path}: {len(r)} detections")

LibreYOLO v1.3.1 ships two validated flagship families plus a broader catalogue of supported and freshly added models. Every model loads through the same LibreYOLO() factory, but only the validated paths below should be treated as heavily tested.

YOLO9 is detection-only in v1.3.1. The non-detect flagship variants (including the old -seg checkpoints) were removed; for segmentation use RF-DETR or EdgeCrafter below.

1
from libreyolo import LibreYOLO
2
3
model = LibreYOLO("LibreYOLO9c.pt")   # detection

LibreYOLO ships the Apache-clean RF-DETR detect sizes N/S/M/L on the Hugging Face org. The XL/2XL tiers are intentionally not shipped.

Heavily tested Segmentation: LibreRFDETRn-seg.pt, LibreRFDETRs-seg.pt, LibreRFDETRm-seg.pt, LibreRFDETRl-seg.pt. Larger -seg sizes (x, xx) pull upstream RF-DETR seg-XL / seg-2XL weights under a non-commercial license and are not hosted on the LibreYOLO org. See the Segmentation section.

Research preview Pose: LibreRFDETRx-pose.pt (ported from RF-DETR v1.8.0 GroupPose; only size x at 576 ships). OBB: LibreRFDETRn-obb.pt, LibreRFDETRs-obb.pt, LibreRFDETRm-obb.pt, LibreRFDETRl-obb.pt (oriented boxes, uses detection input sizes). Treat both as research previews, not validated paths.

1
from libreyolo import LibreYOLO
2
3
model = LibreYOLO("LibreRFDETRs.pt")           # detect (validated)
4
# model = LibreYOLO("LibreRFDETRs-seg.pt")     # segment (validated)
5
# model = LibreYOLO("LibreRFDETRx-pose.pt")    # pose  (research preview)
6
# model = LibreYOLO("LibreRFDETRn-obb.pt")     # obb   (research preview)

Additional detection families

Detection-capable families that share the same factory and API surface as the validated paths. These are experimental in v1.3.1. Each checkpoint name links to its model card on the LibreYOLO org; pass any name to LibreYOLO() and the factory fetches it on first use.

Hosting note: YOLO-NAS checkpoints (plain text above) are hosted on Deci's CDN under their proprietary weights license, not on the LibreYOLO Hugging Face org. The factory still downloads them automatically on first use. DAMO-YOLO was removed in v1.3.0 and is no longer loadable.

New model families in v1.3.0

v1.3.0 adds classification, dense semantic segmentation, monocular depth and point-localization families. They load through the same factory but are newly added and experimental. DINOv2 needs pip install libreyolo[rfdetr] (transformers).

• MobileNetV4 is the commercially clean classification path: Apache-2.0 ImageNet-1k weights (s/m/l at 224/224/256), with predict, top-1/top-5 validation, fine-tune training and ONNX export.
• ConvNeXt (V1 Tiny/Small/Base, 224) and EfficientNetV2 (b0-b3, 224-300) are additional Apache-2.0 ImageNet-1k classifiers (the accuracy tier).
• DINOv2 is a DINOv2 backbone with a task head: dense semantic segmentation at 518 (default) and a classification linear probe at 224. It is not the RF-DETR detector. Classification was moved here from RF-DETR in v1.3.0.
• Depth Anything V2 does monocular depth (sizes s/b/l/g, all at 518). ViT-S weights are Apache-2.0; ViT-B/L/G are CC-BY-NC-4.0 (non-commercial). Inference and zero-shot validation only: not trainable and with no export.
• FOMO is a point-localizer emitting (x, y, class, confidence) per object. Pretrained weights are not redistributed: pass a local checkpoint or train from scratch.

Promptable and VLM tiers: LibreSAM (promptable segmentation, libreyolo[sam]) and the LibreVLM tier of vision-language detectors (libreyolo[vlm]) are separate categories that load upstream Hugging Face snapshots and are not routed through the LibreYOLO() detector factory. Their weights inherit each upstream model's license.

Specialized models

L2CS architecture sizes include r18, r34, r50, r101, and r152, but the upstream-published Gaze360 checkpoint is ResNet-50. Install libreyolo[gaze] for the optional download helper, or pass a local checkpoint path for other sizes. L2CS weights are not hosted by LibreYOLO (the Gaze360 dataset license forbids redistribution).

Factory function

Use the LibreYOLO() factory for every model and runtime. Give it an official checkpoint name or exported artifact path, then let it choose the right model family, task, class count, and runtime:

1
from libreyolo import LibreYOLO
2
3
# Default: YOLO9 detection
4
model = LibreYOLO("LibreYOLO9c.pt")
5
6
# Flagship transformer: RF-DETR
7
model = LibreYOLO("LibreRFDETRs.pt")
8
model = LibreYOLO("LibreRFDETRs-seg.pt")        # validated segmentation
9
10
# New in v1.3.0
11
model = LibreYOLO("LibreMobileNetV4s-cls.pt")   # classification (Apache, ImageNet-1k)
12
model = LibreYOLO("LibreDINOv2n.pt")            # semantic segmentation
13
model = LibreYOLO("LibreDepthAnythingV2s-depth.pt")  # monocular depth
14
model = LibreYOLO("LibreFOMOs-point.pt")        # point localization (local weights)
15
16
# Exported deployment formats
17
model = LibreYOLO("model.onnx")                 # ONNX Runtime
18
model = LibreYOLO("model.engine")               # TensorRT
19
model = LibreYOLO("model.mlpackage")            # CoreML (macOS)
20
model = LibreYOLO("model_openvino/")            # OpenVINO (directory)
21
model = LibreYOLO("model_ncnn/")                # NCNN (directory)

For recognized official checkpoint filenames, LibreYOLO can auto-download missing weights. For custom filenames, point at an explicit local path. Keep new projects on YOLO9 detection or RF-DETR detection / segmentation; other families, tasks, and the new families are experimental in v1.3.1.

LibreYOLO uses a uniform filename convention so the factory can detect family, size, and task from the checkpoint name alone:

1
Libre<FAMILY><size>[-<task>].pt

Task suffixes

Detection is implicit (no suffix), following the common YOLO convention. The factory accepts aliases at the API boundary ("detection", "seg","keypoints", "cls", etc.); only the canonical names above appear in filenames. A task is available only when it is in that family's supported-task set.

Resolution precedence

When you load a model, the task is resolved in this order:

1
explicit task=  →  checkpoint["task"]  →  filename suffix  →  family default

1
from libreyolo import LibreYOLO
2
3
# 1. Filename suffix decides → segment
4
model = LibreYOLO("LibreRFDETRs-seg.pt")
5
6
# 2. Override regardless of filename
7
model = LibreYOLO("custom_weights.pt", task="segment")
8
9
# 3. Detection is implicit
10
model = LibreYOLO("LibreYOLO9c.pt")  # task="detect"

Per-family task support

Three tiers sit outside the LibreYOLO() factory and are imported directly instead: LibreSAM (promptable segmentation), LibreOpenVocab (open-vocabulary detection), and LibreVLM. They are not checkpoint families, so LibreYOLO("sam_b") and friends will not resolve.

Legacy YOLO baselines

v1.3.1 adds the historical Darknet lineage so you can reproduce old baselines against modern ones with one API. These are inference-only: none of them can be trained in LibreYOLO, and they are not the path to pick for new work. Reach for YOLO9 or RF-DETR instead.

All four are COCO-80. LibreYOLO7 is ported from the MIT-licensed MultimediaTechLab/YOLO, deliberately not from the GPL-3.0 reference implementation, so it is safe to use commercially.

YOLO9-P2, for small objects

LibreYOLO9P2 adds a stride-4 detection scale to YOLO9. That extra high-resolution head is what makes it worth the cost when your objects are tiny in frame, which is the classic aerial and drone-footage problem. It trains and exports like YOLO9.

One published checkpoint ships: LibreYOLO9P2s-visdrone.pt, trained on VisDrone. There is no COCO-pretrained P2 checkpoint. Note the licence carefully: the VisDrone weights are CC BY-NC-SA 3.0, so they are non-commercial. Train your own P2 weights on a permissive dataset if you need commercial use.

Two rough edges in v1.3.1. TFLite export is not available for P2 (though it is available for the legacy families above), and the CLI cannot resolve the variant filename, so load the VisDrone checkpoint from Python.

Examples

1
# Detection (implicit)
2
LibreYOLO9c.pt
3
LibreRFDETRs.pt
4
LibreRTDETRr50.pt
5
6
# Instance segmentation (-seg)
7
LibreRFDETRs-seg.pt
8
LibreECm-seg.pt
9
10
# Semantic segmentation (-sem)
11
LibreDINOv2n.pt          # semantic is DINOv2's default; -sem optional
12
13
# Pose (-pose)
14
LibreYOLONASn-pose.pt
15
LibreECs-pose.pt
16
LibreRFDETRx-pose.pt     # preview; size x only
17
18
# Oriented boxes (-obb)
19
LibreRFDETRn-obb.pt      # preview
20
21
# Classification (-cls)
22
LibreMobileNetV4s-cls.pt
23
LibreConvNeXtt-cls.pt
24
LibreEfficientNetV2b0-cls.pt
25
LibreDINOv2n-cls.pt      # DINOv2 linear probe
26
27
# Depth (-depth)
28
LibreDepthAnythingV2s-depth.pt
29
30
# Point (-point)
31
LibreFOMOs-point.pt
32
33
# Gaze (-gaze optional; only task for L2CS)
34
LibreL2CSr50.pt

Deprecated aliases

LibreYOLORTDETR and LibreYOLORFDETR are old names for LibreRTDETR and LibreRFDETR respectively. They still resolve with a DeprecationWarning - update imports when convenient.

The single-GPU prediction path is heavily tested for YOLO9 detection, RF-DETR detection, and RF-DETR segmentation. Other families and tasks use the same API but are experimental in v1.3.1.

Basic prediction

1
result = model("image.jpg")

All prediction parameters

1
result = model(
2
    "image.jpg",
3
    conf=0.25,            # confidence threshold (default: 0.25)
4
    iou=0.45,             # NMS IoU threshold (default: 0.45)
5
    imgsz=640,            # input size override (default: model's native)
6
    device="auto",        # "auto", "cpu", "mps", "0", "cuda:0", ...
7
    classes=[0, 2, 5],    # filter to specific class IDs (default: all)
8
    max_det=300,          # max detections per image (default: 300)
9
    augment=False,        # test-time augmentation where implemented
10
    save=True,            # save annotated image (default: False)
11
    batch=4,              # directory batch size
12
    stream=False,         # video only: yield frame results instead of a list
13
    vid_stride=1,         # video only: process every N-th frame
14
    show=False,           # video only: display annotated frames
15
    tiling=False,         # large-image tiled detection
16
    overlap_ratio=0.2,    # tile overlap ratio
17
    output_path="out/",   # where to save (default: runs/detect/predict*/)
18
    color_format="auto",  # "auto", "rgb", or "bgr"
19
    output_file_format="png",  # output format: "jpg", "png", "webp"
20
)

model.predict(...) is an alias for model(...).

Supported input formats

LibreYOLO accepts images in any of these formats:

1
# File path (string or pathlib.Path)
2
result = model("photo.jpg")
3
result = model(Path("photo.jpg"))
4
5
# URL
6
result = model("https://example.com/image.jpg")
7
result = model("s3://bucket/image.jpg")
8
result = model("gs://bucket/image.jpg")
9
10
# PIL Image
11
from PIL import Image
12
img = Image.open("photo.jpg")
13
result = model(img)
14
15
# NumPy array (HWC or CHW, RGB or BGR, uint8 or float32)
16
import numpy as np
17
arr = np.random.randint(0, 255, (480, 640, 3), dtype=np.uint8)
18
result = model(arr)
19
20
# OpenCV (BGR) - specify color_format
21
import cv2
22
frame = cv2.imread("photo.jpg")
23
result = model(frame, color_format="bgr")
24
25
# PyTorch tensor (CHW or NCHW)
26
import torch
27
tensor = torch.randn(3, 640, 640)
28
result = model(tensor)
29
30
# Raw bytes
31
with open("photo.jpg", "rb") as f:
32
    result = model(f.read())
33
34
# BytesIO
35
from io import BytesIO
36
result = model(BytesIO(open("photo.jpg", "rb").read()))
37
38
# Directory of images
39
results = model("images/", batch=4)

Working with results

Every prediction returns a Results object (or a list of them for directories):

1
result = model("image.jpg")
2
3
# Number of detections
4
len(result)  # e.g., 5
5
6
# Bounding boxes in xyxy format (x1, y1, x2, y2)
7
result.boxes.xyxy        # tensor of shape (N, 4)
8
9
# Bounding boxes in xywh format (center_x, center_y, width, height)
10
result.boxes.xywh        # tensor of shape (N, 4)
11
12
# Confidence scores
13
result.boxes.conf        # tensor of shape (N,)
14
15
# Class IDs
16
result.boxes.cls         # tensor of shape (N,)
17
18
# Combined data: [x1, y1, x2, y2, conf, cls]
19
# Tracking adds a track_id column before conf/cls.
20
result.boxes.data        # shape (N, 6), or (N, 7) when tracked
21
22
# Metadata
23
result.orig_shape        # (height, width) of original image
24
result.path              # source file path (or None)
25
result.names             # {0: "person", 1: "bicycle", ...}
26
27
# Move to CPU / convert to numpy
28
result_cpu = result.cpu()
29
boxes_np = result.boxes.numpy()

Class filtering

Filter detections to specific class IDs:

1
# Only detect people (class 0) and cars (class 2)
2
result = model("image.jpg", classes=[0, 2])

Batched in-memory inference

New in v1.3.0: model.predict() accepts a list or tuple of in-memory images (NumPy arrays, PIL images, or tensors) and runs them as a true stacked-forward batch. Set batch > 1 to actually batch the forward pass on families that support it; a list of results is returned, one per input.

1
import numpy as np
2
from libreyolo import LibreYOLO
3
4
model = LibreYOLO("LibreYOLO9c.pt")
5
6
frames = [
7
    np.random.randint(0, 255, (480, 640, 3), dtype=np.uint8),
8
    np.random.randint(0, 255, (480, 640, 3), dtype=np.uint8),
9
    np.random.randint(0, 255, (480, 640, 3), dtype=np.uint8),
10
]
11
12
results = model(frames, batch=4)   # list/tuple -> true batched inference
13
for r in results:
14
    print(len(r), r.boxes.xyxy.shape)

Model info

model.info() (new in v1.3.0) returns a JSON-friendly dict of family, size, task, parameter counts, input size, and class names, and logs a human-readable summary when verbose=True.

1
meta = model.info(detailed=False, verbose=True)
2
# meta -> {"family": ..., "size": ..., "task": ..., "params": ..., "imgsz": ..., "names": {...}, ...}

For images much larger than the model's input size (e.g., satellite imagery, drone footage), tiled inference splits the image into overlapping tiles, runs detection on each, and merges results.

Tiling is detection-only in v1.3.1. It rejects segmentation masks, and it cannot be combined with augment=True.

1
result = model(
2
    "large_aerial_image.jpg",
3
    tiling=True,
4
    overlap_ratio=0.2,   # 20% overlap between tiles (default)
5
    save=True,
6
)
7
8
# Extra metadata on tiled results
9
result.tiled           # True
10
result.num_tiles       # number of tiles used
11
result.saved_path      # output directory when save=True
12
result.tiles_path      # directory containing per-tile crops
13
result.grid_path       # grid visualization image

When save=True with tiling, LibreYOLO saves:

• final_image.jpg - full image with all merged detections drawn
• grid_visualization.jpg - image showing tile grid overlay
• tiles/ - individual tile crops
• metadata.json - tiling parameters and detection counts

If the image is already smaller than the model's input size, tiling is skipped automatically.

Pass any video file to a flagship and LibreYOLO auto-detects the format from the extension. Supported: .mp4, .avi, .mov, .mkv, .webm, .gif, and other common containers.

Save annotated video

1
from libreyolo import LibreYOLO
2
3
model = LibreYOLO("LibreYOLO9c.pt")
4
results = model("clip.mp4", save=True)
5
# Saved under runs/detect/predict*/clip.mp4

Stream results (memory-flat)

For long videos, pass stream=True to get a generator. Each iteration yields the Results for one frame - no full list buffered in RAM.

1
for result in model("long_clip.mp4", stream=True):
2
    print(f"frame {result.frame_idx}: {len(result)} detections")

Frame subsampling

1
# Process every 2nd frame (halves compute and saved fps)
2
results = model("clip.mp4", vid_stride=2, save=True)

Live preview

1
# Display annotated frames in an OpenCV window while processing
2
results = model("clip.mp4", show=True)

VideoSource / VideoWriter for custom pipelines

When you need full control of decoding and encoding - custom frame transforms, mixing tracker output, writing to a non-default codec - use the building blocks directly:

1
from libreyolo import LibreYOLO
2
from libreyolo.utils.video import VideoSource, VideoWriter
3
4
model = LibreYOLO("LibreYOLO9c.pt")
5
6
with VideoSource("clip.mp4", vid_stride=1) as src, \
7
     VideoWriter("out.mp4", fps=src.fps, width=src.width, height=src.height) as out:
8
    for frame_bgr, frame_idx in src:
9
        result = model(frame_bgr, color_format="bgr")
10
        # ... draw, transform, etc.
11
        out.write_frame(frame_bgr)

LibreYOLO ships two motion trackers that consume Results from any detector and add persistent track IDs: ByteTrack (default) and OC-SORT (new in v1.3.0), which is more robust to occlusion and non-linear motion. Tracking is most tested with single-GPU YOLO9 detection and RF-DETR detection; other detection families are experimental in v1.3.1.

Install

1
pip install libreyolo[tracking]   # compatibility extra; tracking deps ship in base dev install

Video tracking helper

1
from libreyolo import LibreYOLO
2
3
model = LibreYOLO("LibreYOLO9c.pt")
4
5
for result in model.track(
6
    "clip.mp4",
7
    track_conf=0.25,
8
    iou=0.45,
9
    save=True,             # writes runs/track/<video_stem>.mp4 by default
10
    vid_stride=1,
11
):
12
    print(result.frame_idx, result.track_id)

model.track() is a generator for video files. It runs detection frame by frame, uses the lower ByteTrack confidence internally for recovery, and yields Results with result.track_id and result.boxes.id populated.

Basic loop

1
from libreyolo import LibreYOLO, ByteTracker
2
from libreyolo.utils.video import VideoSource
3
4
model = LibreYOLO("LibreYOLO9c.pt")
5
tracker = ByteTracker()
6
7
with VideoSource("clip.mp4") as src:
8
    for frame_bgr, frame_idx in src:
9
        result = model(frame_bgr, color_format="bgr", conf=0.1)
10
        tracked = tracker.update(result)
11
12
        for i in range(len(tracked.boxes)):
13
            track_id = int(tracked.boxes.id[i])
14
            xyxy = tracked.boxes.xyxy[i].tolist()
15
            cls = int(tracked.boxes.cls[i])
16
            print(f"frame {frame_idx} - id {track_id} cls {cls} {xyxy}")

After tracker.update(), result.boxes.id holds the track IDs and result.boxes.is_track is True.

TrackConfig knobs

1
from libreyolo import ByteTracker, TrackConfig
2
3
cfg = TrackConfig(
4
    track_high_thresh=0.25,           # first-stage match threshold
5
    track_low_thresh=0.1,             # second-stage (low-conf recovery)
6
    new_track_thresh=0.25,            # minimum conf to start a new track
7
    match_thresh=0.8,                 # IoU cost cutoff (stage 1)
8
    match_thresh_low=0.5,             # IoU cost cutoff (stage 2)
9
    match_thresh_unconfirmed=0.7,     # IoU cost cutoff for unconfirmed tracks
10
    track_buffer=30,                  # frames to keep lost tracks before removal
11
    frame_rate=30,                    # scales track_buffer
12
    fuse_score=True,                  # multiply IoU by detection score
13
    minimum_consecutive_frames=1,     # frames to confirm a new track
14
)
15
tracker = ByteTracker(config=cfg)

Reset between clips

1
tracker.reset()   # clears tracked / lost / removed lists and the ID counter

OC-SORT (occlusion-robust)

Select OC-SORT with tracker="ocsort" on model.track(). ByteTrack stays the default. With OC-SORT, track_conf maps to the tracker's det_thresh (for ByteTrack it maps to track_high_thresh).

1
from libreyolo import LibreYOLO
2
3
model = LibreYOLO("LibreYOLO9c.pt")
4
5
for result in model.track(
6
    "clip.mp4",
7
    tracker="ocsort",      # "bytetrack" (default) or "ocsort"
8
    track_conf=0.25,       # maps to OC-SORT det_thresh
9
    iou=0.45,
10
    save=True,
11
):
12
    print(result.frame_idx, result.track_id)

Pass an OCSortConfig for full control. Supplying a config instance selects the tracker by type, so the tracker= string is then ignored.

1
from libreyolo import LibreYOLO, OCSortConfig
2
3
cfg = OCSortConfig(
4
    det_thresh=0.25,     # boxes above this drive association and spawn new tracks
5
    max_age=30,          # frames a track survives without an observation
6
    min_hits=3,          # consecutive hits before a track is reported
7
    iou_threshold=0.3,   # minimum IoU for a valid association
8
    delta_t=3,           # frame span used to estimate velocity direction
9
    inertia=0.2,         # weight of the velocity-direction (momentum) term
10
    use_byte=False,      # enable the BYTE low-score recovery pass
11
)
12
13
model = LibreYOLO("LibreYOLO9c.pt")
14
for result in model.track("clip.mp4", tracker_config=cfg, save=True):
15
    print(result.frame_idx, result.track_id)

LibreEnsemble runs two or more detection models and fuses their detections into one ordinary Results. Fusion happens at the detection level, never at the tensor level, so every member keeps its own input size, normalization and NMS. That is what lets you mix a grid detector with a DETR, or a .pt checkpoint with an exported backend, in the same ensemble.

Class spaces do not have to match. Members are unified by class name: identical name maps pass straight through, otherwise LibreYOLO builds the union and remaps each member into it. Boxes are only fused with boxes of the same unified class, and a class that only one member knows passes through unfused.

Fuse two detectors

1
from libreyolo import LibreEnsemble
2
3
# Weighted Boxes Fusion (the default), keep only boxes BOTH models found
4
ens = LibreEnsemble(["LibreYOLO9s.pt", "LibreRFDETRs.pt"], min_votes=2)
5
6
result = ens("image.jpg", conf=0.25)
7
print(result.boxes.xyxy)
8
print(result.names)     # the unified (union) class map
9
print(result.speed)     # per-member timings plus fusion

Trust weights and per-member settings

weights expresses how much you trust each member (set it proportional to each model's validation mAP). conf, iou and device accept either one value for everyone or one value per member.

1
ens = LibreEnsemble(
2
    ["LibreYOLO9s.pt", "LibreRFDETRs.pt"],
3
    weights=[1.0, 1.4],     # pull fused coordinates and scores toward member 2
4
    fusion="wbf",           # "wbf" | "wbf_seeded" | "nms" | your own callable
5
    fusion_iou=0.55,        # IoU used to CLUSTER boxes for fusion, not member NMS
6
    min_votes=1,            # keep boxes confirmed by at least N members
7
)
8
9
result = ens("image.jpg", conf=[0.25, 0.4])   # per-member confidence

Bring an outside detector

ExternalDetector wraps any callable that returns boxes, so a model that is not a LibreYOLO model can still join the ensemble. The function receives a PIL image and must return boxes in original-image pixels.

1
from libreyolo import LibreEnsemble, ExternalDetector
2
3
def my_detector(image):
4
    # -> (boxes_xyxy, scores, labels) in ORIGINAL-image pixels
5
    return boxes, scores, labels
6
7
member = ExternalDetector(my_detector, names={0: "person"})
8
ens = LibreEnsemble(["LibreYOLO9s.pt", member])

Limits

• Detection members only. Any member whose task is not detect raises. Segmentation and pose models cannot be ensembled.
• At least two members are required.
• min_votes above 1 requires a voting fusion. It raises with fusion="nms"; use wbf or wbf_seeded.
• Images and image directories only. Video sources and stream=True raise: run the members individually for video.
• ens.val() and ens.export() both raise. Validate and export the members individually.
• batch is accepted for API parity but images are still processed one at a time.

RF-DETR segmentation is the segmentation path in v1.3.1 and is the heavily tested option. EdgeCrafter (-seg) also exposes a segmentation head but is experimental. YOLO9 no longer ships a segmentation head: it is detect-only as of v1.3.1.

Run segmentation

1
from libreyolo import LibreYOLO
2
3
# RF-DETR segmentation, the heavily tested segmentation path
4
model = LibreYOLO("LibreRFDETRs-seg.pt")
5
result = model("photo.jpg")
6
7
# EdgeCrafter segmentation is also available but experimental
8
# model = LibreYOLO("LibreECs-seg.pt")
9
10
# Segmentation returns boxes + masks
11
print(result.boxes.xyxy)        # bounding boxes (N, 4)
12
print(result.boxes.cls)         # class IDs (N,)
13
print(result.masks.data.shape)  # (N, H, W) tensor of binary masks

Mask representations

1
# Raw bitmasks
2
result.masks.data        # tensor (N, H, W) - original image resolution
3
4
# Polygon contours (one ndarray of (M, 2) per instance)
5
result.masks.xy          # absolute pixel coords
6
result.masks.xyn         # normalized to [0, 1]
7
8
# Move / convert like Boxes
9
result.masks.cpu()
10
result.masks.numpy()

Save annotated output

save=True draws boxes and translucent mask overlays automatically.

1
model("photo.jpg", save=True)

Training segmentation

RF-DETR segmentation uses the RF-DETR COCO-format training pipeline and is part of the heavily tested single-GPU scope. EdgeCrafter segmentation training is available but experimental. YOLO9 segmentation training was removed in v1.3.0.

Semantic segmentation labels every pixel with a class. It is a different task from instance segmentation: there are no object instances and no boxes, just one dense class map. Pass task="semantic" (aliases: semseg, sem), and read the result from result.semantic_mask. On a semantic model result.boxes and result.masks are both None.

Models

The three behave quite differently, so pick deliberately. LibrePIDNet is a fast real-time CNN carrying Cityscapes road-scene classes. LibreEoMT carries ADE20K's 150 general scene classes. Both ship pretrained weights and cannot be trained inside LibreYOLO: fine-tune them upstream and convert the result.

LibreDINOv2 is the mirror image, and the distinction matters: it is the fine-tuning family. There is no published LibreDINOv2 semantic checkpoint. You construct it from the pretrained DINOv2 backbone with a fresh dense head and train it on your own masks, so it is the family to reach for when your classes are not Cityscapes or ADE20K.

Run semantic segmentation

1
from libreyolo import LibreYOLO
2
3
model = LibreYOLO("LibrePIDNets-sem.pt")   # Cityscapes, 19 classes
4
result = model.predict("street.jpg")
5
6
sm = result.semantic_mask     # SemanticMask
7
print(sm.data.shape)          # (H, W) int class ids, on the ORIGINAL image canvas
8
print(sm.classes)             # sorted class ids present, 255 (ignore) excluded
9
print(model.names[13])        # 'car'
10
11
car = sm.class_mask(13)       # (H, W) bool mask for one class
12
13
result.plot().save("out.png") # class map overlaid on the image
14
15
print(result.boxes, result.masks)   # None None: semantic has no instances

SemanticMask API

1
sm = result.semantic_mask
2
3
sm.data               # (H, W) integer class ids at original resolution
4
sm.orig_shape         # (H, W)
5
sm.classes            # list[int] of ids present, excluding the ignore index
6
sm.class_mask(cid)    # (H, W) bool
7
SemanticMask.IGNORE_INDEX   # 255: the void label, never counted as a class
8
9
sm.cpu(); sm.numpy()

Validate

Validation reports mean IoU and pixel accuracy. Classes never seen in either the prediction or the ground truth are excluded from the mean rather than scored as zero. fitness is an alias of mIoU, so it is what drives best-checkpoint selection during training.

1
metrics = model.val(data="cityscapes.yaml")
2
print(metrics["metrics/mIoU"])
3
print(metrics["metrics/pixel_accuracy"])

1
libreyolo val model=LibrePIDNets-sem.pt data=cityscapes.yaml split=val

Train (LibreDINOv2)

Masks are single-channel lossless images whose pixel value is the class id, paired to each image by filename stem. 255 means ignore and is excluded from both loss and metrics.

1
dataset/
2
    images/train/*.jpg
3
    images/val/*.jpg
4
    masks/train/*.png      # same stem as the image; pixel value = class id
5
    masks/val/*.png

1
from libreyolo import LibreDINOv2
2
3
# model_path=None -> pretrained DINOv2 backbone + a fresh dense head
4
model = LibreDINOv2(model_path=None, size="s", task="semantic", nb_classes=19)
5
model.train(data="cityscapes.yaml", epochs=100, batch_size=4, lr=1e-4)

In the dataset YAML, masks_dir names the mask directory (default masks). If you omit it, LibreYOLO rasterizes masks from YOLO polygon labels at load time and appends a background class. label_mapping remaps source pixel values to training ids, and anything unmapped becomes ignore.

Limits

• No export, in any format. Semantic export is blocked at the framework level, not just per model: ONNX, TensorRT, OpenVINO, NCNN, CoreML and TFLite all raise. A semantic export contract (dense logits plus backend argmax) has not landed yet.
• Only LibreDINOv2 trains. LibrePIDNet.train() and LibreEoMT.train() raise.
• EoMT is size l only and locked to imgsz=512 (its checkpoint uses fixed position embeddings), and it cannot batch: val(batch=N) warns and still runs one image at a time.
• imgsz divisibility differs per family: PIDNet needs a multiple of 8, EoMT of 16, DINOv2 of 14. Violations raise.
• No tracking and no test-time augmentation for semantic models.
• Cityscapes, ADE20K and COCO-Stuff all require a manual download. LibreYOLO ships the dataset YAMLs, not the data.
• Raw upstream checkpoints are rejected. Convert with weights/convert_pidnet_weights.py or weights/convert_eomt_weights.py.

LibreSAM is a separate tier from the detector factory, because a promptable segmenter has a different contract: it runs a heavy image encoder once, then answers cheap spatial prompts (a click, a box) with a mask. There is no fixed class list. Install it with pip install "libreyolo[sam]".

Two things surprise people. First, LibreSAM is a factory function, not a class, and it is deliberately kept outside the LibreYOLO() loader, so LibreYOLO("sam_b") does not work. Import it directly. Second, the whole tier is Python-only: there is no CLI path to it.

Models

Those short aliases only work through the LibreSAM() factory. The concrete classes take canonical sizes, so LibreSAM1("base") is right and LibreSAM1("sam_b") raises.

Prompt with a click or a box

1
from libreyolo import LibreSAM
2
3
model = LibreSAM("base")          # SAM-1 ViT-B
4
5
# a single click
6
r = model.predict("img.jpg", points=[640, 360], labels=[1])
7
print(r.masks.data.shape)   # (1, H, W) bool, at the original resolution
8
print(r.boxes.xyxy)         # a tight box derived from the mask
9
print(r.boxes.conf)         # SAM's predicted mask quality, NOT a detection score
10
11
# a box prompt
12
r = model.predict("img.jpg", bboxes=[100, 100, 500, 500])
13
14
# segment everything (a coarse grid); lower the grid on CPU, it is slow
15
r = model.predict("img.jpg", points_per_side=16)

Encode once, prompt many times

This is the pattern that makes interactive use fast. The expensive encoder runs once per image and every later prompt reuses the embedding.

1
model.set_image("img.jpg")                        # heavy encoder runs ONCE
2
a = model.predict(points=[500, 375], labels=[1])  # cheap: decoder only
3
b = model.predict(bboxes=[100, 100, 200, 200])    # cheap: reuses the embedding
4
model.reset_image()

How prompts are shaped

Nesting depth carries meaning, and this is the single easiest thing to get wrong. Points are plain [x, y] pixels. A label of 1 means include, 0 means exclude.

1
# refine ONE object with a positive and a negative click
2
r = model.predict(
3
    "img.jpg",
4
    points=[[[500, 375], [620, 400]]],   # one object, two points
5
    labels=[1, 0],                        # include, then exclude
6
)
7
8
# all three whole-vs-part candidate masks for an ambiguous click
9
r = model.predict("img.jpg", points=[640, 360], labels=[1], multimask=True)

Limits

• SAM-2 is images only. There is no video segmentation and no memory propagation across frames in v1.3.1: track() raises. Call predict() per frame.
• No training, no validation, no export, for any SAM family. All three raise, including export(format="onnx").
• Mask prompts (masks=) are not supported and raise. Use points or boxes.
• conf here filters on SAM's predicted mask quality, not on detection confidence. Detector intuition does not transfer.
• Everything runs in fp32, even on CUDA. This is deliberate: half precision rounds prompt coordinates by several pixels at SAM's 1024px working size, which silently moves where you clicked.
• Segment-everything is a simplified grid, not the reference automatic mask generator. It under-segments crowded scenes.
• Weights download into ./weights/ relative to your working directory, so running from elsewhere re-downloads.

Give the model a list of class names as text and get real detection boxes back. No training, no labelled data. Change the list and you change what it detects. Install with pip install "libreyolo[openvocab]".

This is not the same as the LibreVLM tier, and the difference matters. These are purpose-built detectors conditioned on text: the detector head returns boxes with real model scores. A VLM instead generates text that LibreYOLO parses into boxes. The rule of thumb: boxes for named classes, use open-vocab; describe or instruct, use a VLM. The other practical difference is licensing: every weight in this tier is Apache-2.0, while the VLM tier contains non-commercial models.

Models

Detect anything you can name

The vocabulary is set on the model, with set_classes(), and it is sticky across later calls. There is no prompts= or text= argument on predict().

1
from libreyolo import LibreOpenVocab
2
3
model = LibreOpenVocab("grounding-dino")
4
model.set_classes(["person", "dog", "skateboard"])   # sticky vocabulary
5
6
result = model.predict("street.jpg", conf=0.25, text_threshold=0.25)
7
print(result.boxes.xyxy, result.boxes.conf)
8
print(result.names)     # {0: 'person', 1: 'dog', 2: 'skateboard'}
9
10
result = model.predict("another.jpg")   # same vocabulary, still set
11
12
# or set it at construction
13
model = LibreOpenVocab("owlv2", names=["forklift", "pallet"])