这是上一个版本 v1.2.0 的文档。最新文档为 v1.3.0,可在版本菜单中找到。

简介

v1.2.0 验证范围

经过深度测试的路径是 YOLO9 和 RF-DETR 的检测、训练与推理,包括 RF-DETR 分割。

其他模型系列、任务和多 GPU 工作流均可用,但属实验性。

LibreYOLO 是一个采用 MIT 许可证的目标检测工具包。v1.2.0 提供了广泛的模型目录,但经过验证的支持范围有意保持精简:

  • YOLO9 检测 - CNN 路径。
  • RF-DETR 检测 - Transformer 路径。
  • RF-DETR 分割 - 经过深度测试的分割路径。

我们推荐将这些路径作为新项目的默认选择,因为它们在检测、训练和推理方面经过了最充分的测试。其他受支持的系列和任务通过同一个统一的 LibreYOLO() 工厂函数工作,但在 v1.2.0 中仍属实验性。如有特定理由再使用它们。

python
1from libreyolo import LibreYOLO, SAMPLE_IMAGE
2
3# Default: YOLO9 detection
4model = LibreYOLO("LibreYOLO9c.pt")
5result = model(SAMPLE_IMAGE, conf=0.25, save=True)
6
7print(f"Detected {len(result)} objects")
8print(result.boxes.xyxy)
9print(result.saved_path)

核心特性

  • 针对 YOLO9 检测、RF-DETR 检测和 RF-DETR 分割的深度测试与推荐默认配置
  • 统一的 LibreYOLO() 工厂函数,用于加载检查点、导出产物和运行时
  • 通过一致的 API 支持检测、分割、姿态和视线任务
  • 支持图像、目录和视频推理(大尺寸帧可选用分块推理)
  • 通过 ByteTrack 内置多目标跟踪
  • 支持 ONNX、TorchScript、TensorRT、OpenVINO、NCNN 和 CoreML 导出(含内嵌元数据),并提供匹配的运行时后端
  • 兼容 COCO 的验证与 mAP 指标,并提供分割和姿态验证器
  • Ultralytics 风格的 libreyolo 命令行工具,用于 predict / train / val / export
  • 接受任意图像格式:文件路径、URL、PIL、NumPy、PyTorch 张量、原始字节

兼容性

可将此矩阵用作 v1.2.0 的快速支持速查表。对勾表示该路径在经过验证的文档范围内受支持,exp 表示该路径存在但属实验性,空白单元格表示当前不受支持或不应依赖。

模型系列v1.2.0 状态推理训练检测分割姿态视线ONNXTorchScriptTensorRTOpenVINONCNNCoreML
YOLO9已验证 detect,单 GPUexpNot currently supportedNot currently supported
RF-DETR已验证 detect + segment,单 GPUNot currently supportedNot currently supportedexpNot currently supportedexp
YOLOX实验性expexpexpNot currently supportedNot currently supportedNot currently supportedexpexpexpexpexpexp
YOLO9-E2E实验性expexpexpNot currently supportedNot currently supportedNot currently supportedexpexpexpexpexpNot currently supported
YOLO-NAS实验性expexpexpNot currently supportedexpNot currently supportedexpexpexpexpexpNot currently supported
D-FINE实验性expexpexpNot currently supportedNot currently supportedNot currently supportedexpexpexpexpNot currently supportedNot currently supported
DEIM实验性expexpexpNot currently supportedNot currently supportedNot currently supportedexpexpexpexpNot currently supportedNot currently supported
DEIMv2实验性expexpexpNot currently supportedNot currently supportedNot currently supportedexpexpexpexpNot currently supportedNot currently supported
RT-DETR实验性expexpexpNot currently supportedNot currently supportedNot currently supportedexpexpexpexpNot currently supportedexp
PicoDet实验性expexpexpNot currently supportedNot currently supportedNot currently supportedexpexpexpexpexpNot currently supported
EC实验性expexpexpexpexpNot currently supportedexpexpexpexpNot currently supportedNot currently supported
RT-DETRv2实验性expexpexpNot currently supportedNot currently supportedNot currently supportedexpexpexpexpexpNot currently supported
RT-DETRv4实验性expexpexpNot currently supportedNot currently supportedNot currently supportedexpexpexpexpexpNot currently supported
DAMO-YOLO实验性expexpexpNot currently supportedNot currently supportedNot currently supportedexpexpexpexpexpNot currently supported
RTMDet实验性expexpexpNot currently supportedNot currently supportedNot currently supportedexpexpexpexpexpNot currently supported
L2CS实验性,仅推理expNot currently supportedNot currently supportedNot currently supportedNot currently supportedexpNot currently supportedNot currently supportedNot currently supportedNot currently supportedNot currently supportedNot currently supported

CoreML 导出会生成 .mlpackage 包,并需要 libreyolo[coreml]。CoreML 推理仅限 macOS,不支持 INT8,且 RF-DETR、D-FINE、DEIM、DEIMv2 或 EC 不提供内嵌的 CoreML NMS。

安装

环境要求

  • Python 3.10+
  • PyTorch 1.13+ 与 torchvision 0.11+

从 PyPI 安装

bash
1pip install libreyolo

本文档对应即将发布的 v1.2.0 开发分支。在 v1.2.0 发布到 PyPI 之前,请使用源码安装以获得本页所述功能。

从源码安装

bash
1git clone https://github.com/LibreYOLO/libreyolo.git
2cd libreyolo
3git checkout dev
4pip install -e .

可选依赖

bash
1# ONNX export and inference
2pip install libreyolo[onnx]
3# or: pip install onnx onnxsim onnxruntime
4
5# RT-DETR compatibility extra (currently no extra packages)
6pip install libreyolo[rtdetr]
7
8# RF-DETR support
9pip install libreyolo[rfdetr]
10# or: pip install transformers
11
12# TensorRT export and inference (NVIDIA GPU)
13pip install libreyolo[tensorrt]
14# Installs TensorRT CUDA 12 Python packages on Linux/Windows.
15# Host driver/CUDA compatibility still matters.
16
17# OpenVINO export and inference (Intel CPU/GPU/VPU)
18pip install libreyolo[openvino]
19# INT8 export also needs: pip install nncf
20
21# NCNN export and inference
22pip install libreyolo[ncnn]
23# or: pip install pnnx ncnn
24
25# ByteTrack API compatibility extra
26pip install libreyolo[tracking]
27# Tracking dependencies are part of the base install in v1.2.0 dev.
28
29# CoreML export and inference (macOS only for runtime)
30pip install libreyolo[coreml]
31# or: pip install coremltools
32
33# L2CS gaze optional auto-download helper
34pip install libreyolo[gaze]
35# Optional parity with the upstream RetinaFace-based L2CS pipeline
36pip install libreyolo[gaze-retinaface]
37
38# Install every optional LibreYOLO extra
39pip install libreyolo[all]

如果使用 uv,最可靠的方式是为每个 extra 创建独立的虚拟环境:

bash
1# ONNX environment
2uv venv .venv-onnx
3uv pip install --python .venv-onnx/bin/python -e '.[onnx]'
4
5# RT-DETR environment
6uv venv .venv-rtdetr
7uv pip install --python .venv-rtdetr/bin/python -e '.[rtdetr]'
8
9# Repeat with .[rfdetr], .[openvino], .[ncnn], .[coreml], .[gaze], .[tracking], or .[tensorrt] as needed

这可避免改动项目环境,并使可选依赖保持隔离。TensorRT、OpenVINO、NCNN 和 CoreML 等特定厂商的 extra 仍可能需要平台专属的原生包。

快速开始

若要使用经过最充分测试的路径,请选择单 GPU 的 YOLO9 检测、RF-DETR 检测或 RF-DETR 分割。它们通过同一个工厂函数加载,接受相同的输入,并返回相同的 Results 对象,因此你可以在它们之间切换而无需改动周边代码。

YOLO9 - CNN 旗舰

python
1from libreyolo import LibreYOLO, SAMPLE_IMAGE
2
3# Use the official checkpoint name and let the factory resolve the details
4model = LibreYOLO("LibreYOLO9c.pt")
5
6# Run on a single image (SAMPLE_IMAGE ships with the package)
7result = model(SAMPLE_IMAGE)
8
9print(f"Found {len(result)} objects")
10print(result.boxes.xyxy) # bounding boxes (N, 4)
11print(result.boxes.conf) # confidence scores (N,)
12print(result.boxes.cls) # class IDs (N,)

RF-DETR - Transformer 旗舰

python
1from libreyolo import LibreYOLO, SAMPLE_IMAGE
2
3# Same factory, same call shape - just point at an RF-DETR checkpoint
4model = LibreYOLO("LibreRFDETRs.pt")
5result = model(SAMPLE_IMAGE)
6
7print(f"Found {len(result)} objects")
8print(result.boxes.xyxy)

保存标注后的输出

python
1result = model(SAMPLE_IMAGE, save=True)
2print(result.saved_path) # e.g. runs/detect/predict/parkour.jpg

处理目录

python
1results = model("images/", save=True, batch=4)
2for r in results:
3 print(f"{r.path}: {len(r)} detections")

可用模型

推荐的已验证路径:YOLO9 检测或 RF-DETR 检测 / 分割

这些模型的检测、训练与推理获得了最充分的测试。在 v1.2.0 中,请将其他系列、任务和多 GPU 工作流视为实验性。

LibreYOLO 在 v1.2.0 中提供一小部分经过验证的范围,外加更广泛的受支持模型目录。每个模型都通过同一个 LibreYOLO() 工厂函数加载,但只有下方经过验证的路径才应视为经过深度测试。

YOLO9 - CNN 旗舰

推荐
默认:LibreYOLO9c.pt深度测试:检测、训练与推理实验性:分割、多 GPU
尺寸代号输入尺寸适用场景检测检查点
Tiny"t"640快速推理LibreYOLO9t.pt
Small"s"640均衡LibreYOLO9s.pt
Medium"m"640更高精度LibreYOLO9m.pt
Compact"c"640最佳精度LibreYOLO9c.pt

实验性 分割检查点: LibreYOLO9t-seg.pt, LibreYOLO9s-seg.pt, LibreYOLO9m-seg.pt, LibreYOLO9c-seg.pt。参见 分割 章节。

python
1from libreyolo import LibreYOLO
2
3model = LibreYOLO("LibreYOLO9c.pt")
4# Experimental segmentation variant
5# model = LibreYOLO("LibreYOLO9c-seg.pt")

RF-DETR - Transformer 旗舰

推荐
推荐的 Transformer 路径深度测试:检测、分割、训练与推理实验性:多 GPU
尺寸代号输入尺寸适用场景检测检查点
Nano"n"384边缘设备LibreRFDETRn.pt
Small"s"512均衡LibreRFDETRs.pt
Medium"m"576更高精度LibreRFDETRm.pt
Large"l"704最高精度LibreRFDETRl.pt

深度测试 分割检查点: LibreRFDETRn-seg.pt, LibreRFDETRs-seg.pt, LibreRFDETRm-seg.pt, LibreRFDETRl-seg.pt, LibreRFDETRx-seg.pt, LibreRFDETRxx-seg.pt。参见 分割 章节。

python
1from libreyolo import LibreYOLO
2
3model = LibreYOLO("LibreRFDETRs.pt")
4# Segmentation variants exist for every RF-DETR size
5# model = LibreYOLO("LibreRFDETRs-seg.pt")

其他受支持的系列

具备检测能力的系列,与经过验证的路径共享同一个工厂函数和 API。它们在 v1.2.0 中属实验性。每个检查点名称都链接到 LibreYOLO 组织 在 Hugging Face 上的模型卡;将任意名称传给 LibreYOLO(),工厂函数会在首次使用时自动获取。

系列状态任务检查点
YOLOX实验性detectLibreYOLOXn.pt, LibreYOLOXt.pt, LibreYOLOXs.pt, LibreYOLOXm.pt, LibreYOLOXl.pt, LibreYOLOXx.pt
YOLO9-E2E实验性detectLibreYOLO9E2Et.pt, LibreYOLO9E2Es.pt, LibreYOLO9E2Em.pt, LibreYOLO9E2Ec.pt
YOLO-NAS实验性detect, poseLibreYOLONASs.pt, LibreYOLONASm.pt, LibreYOLONASl.pt, LibreYOLONASn-pose.pt, LibreYOLONASs-pose.pt, LibreYOLONASm-pose.pt, LibreYOLONASl-pose.pt
D-FINE实验性detectLibreDFINEn.pt, LibreDFINEs.pt, LibreDFINEm.pt, LibreDFINEl.pt, LibreDFINEx.pt
DEIM实验性detectLibreDEIMn.pt, LibreDEIMs.pt, LibreDEIMm.pt, LibreDEIMl.pt, LibreDEIMx.pt
DEIMv2实验性detectLibreDEIMv2atto.pt, LibreDEIMv2femto.pt, LibreDEIMv2pico.pt, LibreDEIMv2n.pt, LibreDEIMv2s.pt, LibreDEIMv2m.pt, LibreDEIMv2l.pt, LibreDEIMv2x.pt
RT-DETR实验性detectLibreRTDETRr18.pt, LibreRTDETRr34.pt, LibreRTDETRr50.pt, LibreRTDETRr50m.pt, LibreRTDETRr101.pt, LibreRTDETRl.pt, LibreRTDETRx.pt
RT-DETRv2实验性detectLibreRTDETRv2r18.pt, LibreRTDETRv2r34.pt, LibreRTDETRv2r50.pt, LibreRTDETRv2r50m.pt, LibreRTDETRv2r101.pt
RT-DETRv4实验性detectLibreRTDETRv4s.pt, LibreRTDETRv4m.pt, LibreRTDETRv4l.pt, LibreRTDETRv4x.pt
PicoDet实验性detectLibrePICODETs.pt, LibrePICODETm.pt, LibrePICODETl.pt
EdgeCrafter实验性detect, pose, segmentLibreECs.pt, LibreECm.pt, LibreECl.pt, LibreECx.pt, LibreECs-pose.pt, LibreECm-pose.pt, LibreECl-pose.pt, LibreECx-pose.pt, LibreECs-seg.pt, LibreECm-seg.pt, LibreECl-seg.pt, LibreECx-seg.pt
DAMO-YOLO实验性detectLibreDAMOYOLOns.pt, LibreDAMOYOLOnm.pt, LibreDAMOYOLOnl.pt, LibreDAMOYOLOt.pt, LibreDAMOYOLOs.pt, LibreDAMOYOLOm.pt, LibreDAMOYOLOl.pt
RTMDet实验性detectLibreRTMDett.pt, LibreRTMDets.pt, LibreRTMDetm.pt, LibreRTMDetl.pt, LibreRTMDetx.pt

托管说明: YOLO-NAS 检查点(上方以纯文本显示)托管在 Deci 的 CDN 上,遵循其专有权重许可证,而非 LibreYOLO 的 Hugging Face 组织。工厂函数仍会在首次使用时自动下载它们。

专用模型

系列状态任务检查点
L2CS实验性gaze(仅推理) - 参见 视线估计LibreL2CSr50.pt

L2CS 架构尺寸包括 r18、r34、r50、r101 和 r152,但上游发布的 Gaze360 检查点是 ResNet-50。安装 libreyolo[gaze] 以使用可选的 Google Drive 辅助下载,或为其他尺寸传入本地检查点路径。

工厂函数

对所有模型和运行时都使用 LibreYOLO() 工厂函数。传入官方检查点名称或导出产物路径,让它自动选择正确的模型系列、任务、类别数和运行时:

python
1from libreyolo import LibreYOLO
2
3# Default: YOLO9 detection
4model = LibreYOLO("LibreYOLO9c.pt")
5
6# Flagship: RF-DETR
7model = LibreYOLO("LibreRFDETRs.pt")
8
9# Segmentation checkpoints use the same factory path
10model = LibreYOLO("LibreRFDETRs-seg.pt") # validated segmentation
11model = LibreYOLO("LibreYOLO9c-seg.pt") # experimental segmentation
12
13# Exported deployment formats
14model = LibreYOLO("model.onnx") # ONNX Runtime
15model = LibreYOLO("model.engine") # TensorRT
16model = LibreYOLO("model.mlpackage") # CoreML (macOS)
17model = LibreYOLO("model_openvino/") # OpenVINO (directory)
18model = LibreYOLO("model_ncnn/") # NCNN (directory)

对于可识别的官方检查点文件名,LibreYOLO 可自动下载缺失的权重。对于自定义文件名,请指向明确的本地路径。实验性系列仍通过同一个工厂函数加载,但新项目请坚持使用 YOLO9 检测或 RF-DETR 检测/分割。如有特定理由再使用它们。

任务与文件名

LibreYOLO 使用统一的文件名约定,使工厂函数仅凭检查点名称即可识别系列、尺寸和任务:

text
1Libre<FAMILY><size>[-<task>].pt

任务后缀

任务规范名称文件名后缀
检测"detect"(无 - 隐式)
实例分割"segment"-seg
姿态估计"pose"-pose
分类"classify"-cls
视线估计"gaze"-gaze

工厂函数在 API 边界接受别名("detection""seg""keypoints" 等) - 但文件名中只出现规范名称。

解析优先级

加载模型时,任务按以下顺序解析:

text
1explicit task= → checkpoint["task"] → filename suffix → family default
python
1from libreyolo import LibreYOLO
2
3# 1. Filename suffix decides → segment
4model = LibreYOLO("LibreRFDETRs-seg.pt")
5
6# 2. Override regardless of filename
7model = LibreYOLO("custom_weights.pt", task="segment")
8
9# 3. Detection is implicit
10model = LibreYOLO("LibreYOLO9c.pt") # task="detect"

各系列任务支持

系列v1.2.0 状态默认支持的任务
YOLO9detect 单 GPU 深度测试;segment 与多 GPU 实验性detectdetect, segment
RF-DETRdetect 与 segment 单 GPU 深度测试;多 GPU 实验性detectdetect, segment
YOLOX实验性detectdetect
YOLO9-E2E实验性detectdetect
YOLO-NAS实验性detectdetect, pose
D-FINE / DEIM / DEIMv2实验性detectdetect
RT-DETR / RT-DETRv2 / RT-DETRv4实验性detectdetect
PicoDet实验性detectdetect
EdgeCrafter (EC)实验性detectdetect, pose, segment
DAMO-YOLO / RTMDet实验性detectdetect
L2CS实验性gazegaze(仅推理)

示例

text
1# Detection (implicit)
2LibreYOLO9c.pt
3LibreRFDETRs.pt
4LibreRTDETRr50.pt
5
6# Segmentation
7LibreYOLO9c-seg.pt
8LibreRFDETRs-seg.pt
9LibreECm-seg.pt
10
11# Pose
12LibreYOLONASn-pose.pt
13LibreECs-pose.pt
14
15# Gaze
16LibreL2CSr50.pt # gaze is L2CS's only task - suffix optional

已弃用的别名

LibreYOLORTDETRLibreYOLORFDETR 分别是 LibreRTDETRLibreRFDETR 的旧名称。它们仍可解析,但会触发 DeprecationWarning - 请在方便时更新导入。

预测

单 GPU 预测路径针对 YOLO9 检测、RF-DETR 检测和 RF-DETR 分割经过深度测试。其他系列和任务使用相同的 API,但在 v1.2.0 中属实验性。

基本预测

python
1result = model("image.jpg")

全部预测参数

python
1result = model(
2 "image.jpg",
3 conf=0.25, # confidence threshold (default: 0.25)
4 iou=0.45, # NMS IoU threshold (default: 0.45)
5 imgsz=640, # input size override (default: model's native)
6 device="auto", # "auto", "cpu", "mps", "0", "cuda:0", ...
7 classes=[0, 2, 5], # filter to specific class IDs (default: all)
8 max_det=300, # max detections per image (default: 300)
9 augment=False, # test-time augmentation where implemented
10 save=True, # save annotated image (default: False)
11 batch=4, # directory batch size
12 stream=False, # video only: yield frame results instead of a list
13 vid_stride=1, # video only: process every N-th frame
14 show=False, # video only: display annotated frames
15 tiling=False, # large-image tiled detection
16 overlap_ratio=0.2, # tile overlap ratio
17 output_path="out/", # where to save (default: runs/detect/predict*/)
18 color_format="auto", # "auto", "rgb", or "bgr"
19 output_file_format="png", # output format: "jpg", "png", "webp"
20)

model.predict(...)model(...) 的别名。

支持的输入格式

LibreYOLO 接受以下任意格式的图像:

python
1# File path (string or pathlib.Path)
2result = model("photo.jpg")
3result = model(Path("photo.jpg"))
4
5# URL
6result = model("https://example.com/image.jpg")
7result = model("s3://bucket/image.jpg")
8result = model("gs://bucket/image.jpg")
9
10# PIL Image
11from PIL import Image
12img = Image.open("photo.jpg")
13result = model(img)
14
15# NumPy array (HWC or CHW, RGB or BGR, uint8 or float32)
16import numpy as np
17arr = np.random.randint(0, 255, (480, 640, 3), dtype=np.uint8)
18result = model(arr)
19
20# OpenCV (BGR) - specify color_format
21import cv2
22frame = cv2.imread("photo.jpg")
23result = model(frame, color_format="bgr")
24
25# PyTorch tensor (CHW or NCHW)
26import torch
27tensor = torch.randn(3, 640, 640)
28result = model(tensor)
29
30# Raw bytes
31with open("photo.jpg", "rb") as f:
32 result = model(f.read())
33
34# BytesIO
35from io import BytesIO
36result = model(BytesIO(open("photo.jpg", "rb").read()))
37
38# Directory of images
39results = model("images/", batch=4)

处理结果

每次预测都会返回一个 Results 对象(处理目录时返回其列表):

python
1result = model("image.jpg")
2
3# Number of detections
4len(result) # e.g., 5
5
6# Bounding boxes in xyxy format (x1, y1, x2, y2)
7result.boxes.xyxy # tensor of shape (N, 4)
8
9# Bounding boxes in xywh format (center_x, center_y, width, height)
10result.boxes.xywh # tensor of shape (N, 4)
11
12# Confidence scores
13result.boxes.conf # tensor of shape (N,)
14
15# Class IDs
16result.boxes.cls # tensor of shape (N,)
17
18# Combined data: [x1, y1, x2, y2, conf, cls]
19# Tracking adds a track_id column before conf/cls.
20result.boxes.data # shape (N, 6), or (N, 7) when tracked
21
22# Metadata
23result.orig_shape # (height, width) of original image
24result.path # source file path (or None)
25result.names # {0: "person", 1: "bicycle", ...}
26
27# Move to CPU / convert to numpy
28result_cpu = result.cpu()
29boxes_np = result.boxes.numpy()

类别过滤

将检测结果过滤为特定类别 ID:

python
1# Only detect people (class 0) and cars (class 2)
2result = model("image.jpg", classes=[0, 2])

分块推理

对于远大于模型输入尺寸的图像(如卫星影像、无人机航拍),分块推理会将图像切分为相互重叠的小块,分别在每块上运行检测,再合并结果。

在 v1.2.0 dev 中分块仅支持检测。它不接受分割掩码,且无法与 augment=True 组合使用。

python
1result = model(
2 "large_aerial_image.jpg",
3 tiling=True,
4 overlap_ratio=0.2, # 20% overlap between tiles (default)
5 save=True,
6)
7
8# Extra metadata on tiled results
9result.tiled # True
10result.num_tiles # number of tiles used
11result.saved_path # output directory when save=True
12result.tiles_path # directory containing per-tile crops
13result.grid_path # grid visualization image

在分块时设置 save=True,LibreYOLO 会保存:

  • final_image.jpg - 绘制了全部合并检测结果的完整图像
  • grid_visualization.jpg - 显示分块网格叠加的图像
  • tiles/ - 各个分块裁剪图
  • metadata.json - 分块参数和检测数量

如果图像本身已小于模型的输入尺寸,则会自动跳过分块。

视频推理

将任意视频文件传给旗舰模型,LibreYOLO 会根据扩展名自动识别格式。支持:.mp4.avi.mov.mkv.webm.gif 以及其他常见容器格式。

保存标注后的视频

python
1from libreyolo import LibreYOLO
2
3model = LibreYOLO("LibreYOLO9c.pt")
4results = model("clip.mp4", save=True)
5# Saved under runs/detect/predict*/clip.mp4

流式结果(内存恒定)

对于长视频,传入 stream=True 可获得一个生成器。每次迭代产出一帧的 Results - 不会在内存中缓存完整列表。

python
1for result in model("long_clip.mp4", stream=True):
2 print(f"frame {result.frame_idx}: {len(result)} detections")

帧抽样

python
1# Process every 2nd frame (halves compute and saved fps)
2results = model("clip.mp4", vid_stride=2, save=True)

实时预览

python
1# Display annotated frames in an OpenCV window while processing
2results = model("clip.mp4", show=True)

用于自定义流水线的 VideoSource / VideoWriter

当你需要完全掌控解码和编码时 - 自定义帧变换、混入跟踪器输出、写入非默认编解码器 - 可直接使用这些构建模块:

python
1from libreyolo import LibreYOLO
2from libreyolo.utils.video import VideoSource, VideoWriter
3
4model = LibreYOLO("LibreYOLO9c.pt")
5
6with VideoSource("clip.mp4", vid_stride=1) as src, \
7 VideoWriter("out.mp4", fps=src.fps, width=src.width, height=src.height) as out:
8 for frame_bgr, frame_idx in src:
9 result = model(frame_bgr, color_format="bgr")
10 # ... draw, transform, etc.
11 out.write_frame(frame_bgr)

跟踪

LibreYOLO 内置 ByteTrack 多目标跟踪器,可消费任意检测器输出的 Results 并添加持久的跟踪 ID。它在单 GPU 的 YOLO9 检测和 RF-DETR 检测上测试最充分;其他检测系列在 v1.2.0 中属实验性。

安装

bash
1pip install libreyolo[tracking] # compatibility extra; tracking deps ship in base dev install

视频跟踪辅助函数

python
1from libreyolo import LibreYOLO
2
3model = LibreYOLO("LibreYOLO9c.pt")
4
5for result in model.track(
6 "clip.mp4",
7 track_conf=0.25,
8 iou=0.45,
9 save=True, # writes runs/track/<video_stem>.mp4 by default
10 vid_stride=1,
11):
12 print(result.frame_idx, result.track_id)

model.track() 是用于视频文件的生成器。它逐帧运行检测,内部使用较低的 ByteTrack 置信度进行恢复,并产出已填充 result.track_idresult.boxes.idResults

基本循环

python
1from libreyolo import LibreYOLO, ByteTracker
2from libreyolo.utils.video import VideoSource
3
4model = LibreYOLO("LibreYOLO9c.pt")
5tracker = ByteTracker()
6
7with VideoSource("clip.mp4") as src:
8 for frame_bgr, frame_idx in src:
9 result = model(frame_bgr, color_format="bgr", conf=0.1)
10 tracked = tracker.update(result)
11
12 for i in range(len(tracked.boxes)):
13 track_id = int(tracked.boxes.id[i])
14 xyxy = tracked.boxes.xyxy[i].tolist()
15 cls = int(tracked.boxes.cls[i])
16 print(f"frame {frame_idx} - id {track_id} cls {cls} {xyxy}")

调用 tracker.update() 后,result.boxes.id 保存跟踪 ID,且 result.boxes.is_trackTrue

TrackConfig 可调参数

python
1from libreyolo import ByteTracker, TrackConfig
2
3cfg = TrackConfig(
4 track_high_thresh=0.25, # first-stage match threshold
5 track_low_thresh=0.1, # second-stage (low-conf recovery)
6 new_track_thresh=0.25, # minimum conf to start a new track
7 match_thresh=0.8, # IoU cost cutoff (stage 1)
8 match_thresh_low=0.5, # IoU cost cutoff (stage 2)
9 match_thresh_unconfirmed=0.7, # IoU cost cutoff for unconfirmed tracks
10 track_buffer=30, # frames to keep lost tracks before removal
11 frame_rate=30, # scales track_buffer
12 fuse_score=True, # multiply IoU by detection score
13 minimum_consecutive_frames=1, # frames to confirm a new track
14)
15tracker = ByteTracker(config=cfg)

片段之间重置

python
1tracker.reset() # clears tracked / lost / removed lists and the ID counter

分割

v1.2.0 验证范围

经过深度测试的路径是 YOLO9 和 RF-DETR 的检测、训练与推理,包括 RF-DETR 分割。

其他模型系列、任务和多 GPU 工作流均可用,但属实验性。

RF-DETR 分割是 v1.2.0 中经过深度测试的分割路径。YOLO9 分割和 EdgeCrafter 分割可通过相同的 -seg 后缀使用,但属实验性。

运行分割

python
1from libreyolo import LibreYOLO
2
3# RF-DETR segmentation, heavily tested on single GPU
4model = LibreYOLO("LibreRFDETRs-seg.pt")
5result = model("photo.jpg")
6
7# YOLO9 segmentation is available but experimental in v1.2.0
8# model = LibreYOLO("LibreYOLO9c-seg.pt")
9
10# Segmentation returns boxes + masks
11print(result.boxes.xyxy) # bounding boxes (N, 4)
12print(result.boxes.cls) # class IDs (N,)
13print(result.masks.data.shape) # (N, H, W) tensor of binary masks

掩码表示

python
1# Raw bitmasks
2result.masks.data # tensor (N, H, W) - original image resolution
3
4# Polygon contours (one ndarray of (M, 2) per instance)
5result.masks.xy # absolute pixel coords
6result.masks.xyn # normalized to [0, 1]
7
8# Move / convert like Boxes
9result.masks.cpu()
10result.masks.numpy()

保存标注后的输出

save=True 会自动绘制边界框和半透明掩码叠加层。

python
1model("photo.jpg", save=True)

训练分割模型

RF-DETR 分割使用 RF-DETR 的 COCO 格式训练流水线,属于经过深度测试的单 GPU 范围。YOLO9 分割和 EdgeCrafter 分割训练可用,但在 v1.2.0 中属实验性。

姿态估计

姿态(人体关键点)估计在 YOLO-NAS (-pose)EdgeCrafter (-pose) 上受支持。每个姿态模型均为单类别("person"),包含 17 个 COCO 关键点。

运行姿态估计

python
1from libreyolo import LibreYOLO
2
3# YOLO-NAS pose
4model = LibreYOLO("LibreYOLONASs-pose.pt")
5result = model("people.jpg")
6
7# EdgeCrafter pose
8# model = LibreYOLO("LibreECs-pose.pt")
9
10# Per-person bbox + 17 keypoints
11print(result.boxes.xyxy) # person boxes (N, 4)
12print(result.keypoints.xy.shape) # (N, 17, 2) pixel coordinates

关键点 API

python
1result.keypoints.xy # (N, K, 2) absolute pixel coords
2result.keypoints.xyn # (N, K, 2) normalized to [0, 1]
3result.keypoints.conf # (N, K) per-keypoint confidence (None if model doesn't emit it)
4result.keypoints.has_visible # (N, K) bool - conf > 0
5
6result.keypoints.cpu()
7result.keypoints.numpy()

保存标注后的输出

python
1model("people.jpg", save=True) # draws boxes + skeleton

YOLO-NAS 支持姿态训练;EdgeCrafter 姿态目前仅支持推理。YOLO9 和 RF-DETR 尚未提供姿态检查点。

视线估计

视线方向估计由 LibreL2CS 系列提供,它是 L2CS-Net 的移植版,采用 ResNet 主干和两个角度分箱分类头。这是一个两阶段模型:上游的人脸检测器先定位人脸,然后视线头以弧度预测每张人脸的俯仰角(pitch)和偏航角(yaw)。它仅支持推理,且在 v1.2.0 中属实验性。

安装

bash
1pip install libreyolo[gaze] # optional Google Drive helper for Gaze360 weights

已发布的 L2CS ResNet-50 权重在 Gaze360 上训练,LibreYOLO 不进行镜像。若没有可选辅助工具,请传入本地检查点路径,或按照 LibreL2CS 打印的手动下载说明操作。

两阶段推理

python
1from libreyolo import LibreYOLO
2from libreyolo.models.l2cs.face import resolve_face_detector
3
4# Gaze head
5gaze = LibreYOLO("LibreL2CSr50.pt")
6
7# Wire any LibreYOLO detector trained on faces
8face = LibreYOLO("path/to/face-detector.pt")
9gaze.face_detector = resolve_face_detector(face)
10
11result = gaze("portrait.jpg")
12print(result.boxes.xyxy) # face boxes
13print(result.gaze.data) # (N, 2) tensor - pitch, yaw in radians

解码角度

python
1import math
2
3for i in range(len(result.gaze)):
4 pitch_rad, yaw_rad = result.gaze.data[i].tolist()
5 pitch_deg = pitch_rad * 180.0 / math.pi
6 yaw_deg = yaw_rad * 180.0 / math.pi
7 print(f"face {i}: pitch={pitch_deg:.1f} deg, yaw={yaw_deg:.1f} deg")

通过命令行:libreyolo predict model=LibreL2CSr50.pt source=portrait.jpg --face-detector path/to/face.pt

训练

v1.2.0 验证范围

经过深度测试的路径是 YOLO9 和 RF-DETR 的检测、训练与推理,包括 RF-DETR 分割。

其他模型系列、任务和多 GPU 工作流均可用,但属实验性。

经过深度测试的训练路径是单 GPU 的 YOLO9 检测、RF-DETR 检测和 RF-DETR 分割。其他模型系列的训练器、YOLO9 分割训练和多 GPU 工作流可用,但在 v1.2.0 中属实验性。

YOLO9 - CNN 旗舰训练

python
1from libreyolo import LibreYOLO
2
3# Fine-tune from a pretrained checkpoint (recommended)
4model = LibreYOLO("LibreYOLO9c.pt")
5
6results = model.train(
7 data="coco128.yaml", # path to data.yaml (required)
8
9 # Schedule
10 epochs=300, # default: 300
11 batch=16,
12 imgsz=640,
13
14 # Optimizer
15 lr0=0.01, # initial learning rate
16 optimizer="SGD", # "SGD", "Adam", "AdamW"
17
18 # System
19 device="0", # "" | "cpu" | "cuda" | "0" | "0,1"
20 workers=8,
21 seed=0,
22
23 # Output
24 project="runs/train",
25 name="yolo9_exp",
26 exist_ok=False,
27
28 # Training features
29 amp=True, # automatic mixed precision
30 patience=50, # early stopping patience
31 resume=False, # resume from loaded checkpoint
32)
33
34print(f"Best mAP50-95: {results['best_mAP50_95']:.3f}")
35print(f"Best checkpoint: {results['best_checkpoint']}")

训练完成后,模型实例会自动以最佳权重重新加载,因此你可以立即调用 model(...)。YOLO9 分割训练可通过 LibreYOLO("LibreYOLO9c-seg.pt") 使用,但在 v1.2.0 中属实验性。

RF-DETR - Transformer 旗舰训练

python
1from libreyolo import LibreYOLO
2
3model = LibreYOLO("LibreRFDETRs.pt")
4
5results = model.train(
6 data="path/to/data.yaml",
7 epochs=100,
8 batch_size=4, # NOTE: RF-DETR uses batch_size, not batch
9 lr=1e-4,
10 output_dir="runs/train/rfdetr_exp",
11)

RF-DETR 有自己的训练签名(batch_sizelroutput_dir),但它使用 LibreYOLO 的数据集配置加载器。为检测或分割传入 data.yaml;该配置可引用 COCO/Roboflow 风格的标注布局。

训练结果字典

python
1{
2 "final_loss": 2.31,
3 "best_mAP50": 0.682,
4 "best_mAP50_95": 0.451,
5 "best_epoch": 87,
6 "save_dir": "runs/train/yolo9_exp",
7 "best_checkpoint": "runs/train/yolo9_exp/weights/best.pt",
8 "last_checkpoint": "runs/train/yolo9_exp/weights/last.pt",
9}

恢复训练

python
1# Load the checkpoint with the factory, then resume
2model = LibreYOLO("runs/train/yolo9_exp/weights/last.pt")
3results = model.train(data="coco128.yaml", resume=True)

自定义数据集 YAML 格式

data.yaml
1path: /path/to/dataset
2train: images/train
3val: images/val
4test: images/test # optional
5
6nc: 3
7names: ["cat", "dog", "bird"]

其他训练路径

其他系列也有训练器钩子,但它们不是 v1.2.0 中的推荐路径。新工作请坚持使用 YOLO9 检测或 RF-DETR 检测/分割;仅在兼容、复现基准或定向研究时使用实验性训练器。DAMO-YOLO、PicoDet、RTMDet 和 EC 的训练需要显式的 allow_experimental=True 确认。

通过 YAML 配置训练

每个 model.train(...) 都接受 cfg="train.yaml",以从文件加载全部参数。显式关键字参数仍优先于 yaml 中的值,因此你可以用 yaml 作为基线,并在每次运行时覆盖单个字段。

python
1model = LibreYOLO("LibreYOLO9c.pt")
2results = model.train(cfg="configs/yolo9_finetune.yaml")
3# Override individual fields:
4# results = model.train(cfg="configs/yolo9_finetune.yaml", epochs=50)

梯度累积

传入 nbs(名义批量大小)以启用梯度累积。训练器每 nbs / batch 次前向传播执行一次优化器步进,使你能在较小的硬件上以配方的参考批量大小进行训练。

python
1# Effective batch 64 on a single GPU that only fits batch=8
2model.train(data="coco128.yaml", batch=8, nbs=64)

分布式训练(DDP,实验性)

YOLO9 和 RF-DETR 通过 PyTorch DistributedDataParallel 支持多 GPU 训练,但多 GPU 不在 v1.2.0 的深度测试范围内。使用 torchrun 启动训练脚本:

bash
1# 4-GPU node
2torchrun --nproc_per_node=4 train_yolo9.py
3
4# Multi-node - see PyTorch's torchrun docs for --nnodes / --rdzv_endpoint
train_yolo9.py
1from libreyolo import LibreYOLO
2
3model = LibreYOLO("LibreYOLO9c.pt")
4# Pass device="" (auto-detect) and let torchrun set the rank
5model.train(data="coco128.yaml", epochs=300, batch=16)

验证

在验证集上运行 COCO 标准评估。经过深度测试的验证路径是单 GPU 的 YOLO9 检测、RF-DETR 检测和 RF-DETR 分割。

python
1results = model.val(
2 data="coco128.yaml", # dataset config
3 batch=16,
4 imgsz=640,
5 conf=0.001, # low conf for mAP calculation
6 iou=0.6, # NMS IoU threshold
7 split="val", # "val", "test", or "train"
8 save_json=False, # save predictions as COCO JSON
9 verbose=True, # print per-class metrics
10)
11
12print(f"mAP50: {results['metrics/mAP50']:.3f}")
13print(f"mAP50-95: {results['metrics/mAP50-95']:.3f}")

验证结果字典

默认情况下,LibreYOLO 使用 COCO 评估,返回精确率、召回率、AP/AR 指标以及每张图像的耗时:

python
1{
2 "metrics/mAP50-95": 0.489, # COCO primary metric (AP@[.5:.95])
3 "metrics/mAP50": 0.721, # AP@0.5 (PASCAL VOC style)
4 "metrics/mAP75": 0.534, # AP@0.75 (strict)
5 "metrics/precision": 0.68,
6 "metrics/recall": 0.61,
7 "metrics/precision(B)": 0.68, # bbox aliases
8 "metrics/recall(B)": 0.61,
9 "metrics/mAP50(B)": 0.721,
10 "metrics/mAP50-95(B)": 0.489,
11 "metrics/mAP_small": 0.291,
12 "metrics/mAP_medium": 0.532,
13 "metrics/mAP_large": 0.648,
14 "metrics/AR1": 0.362, # Average Recall (max 1 det)
15 "metrics/AR10": 0.571,
16 "metrics/AR100": 0.601,
17 "metrics/AR_small": 0.387,
18 "metrics/AR_medium": 0.641,
19 "metrics/AR_large": 0.739,
20 "speed/preprocess_ms": 1.2,
21 "speed/inference_ms": 6.8,
22 "speed/postprocess_ms": 0.9,
23 "speed/total_ms": 8.9,
24 "speed/total_s": 12.3,
25 "speed/images_seen": 1382,
26}

分割验证返回带 (M) 后缀的掩码指标,以及带 (B) 后缀的边界框指标。姿态验证通过 PoseValidator 返回 COCO 关键点指标。

导出

将 PyTorch 模型导出为 ONNX、TorchScript、TensorRT、OpenVINO、NCNN 或 CoreML 以便部署。经过深度测试的导出与运行时后端路径是单 GPU 的 YOLO9 检测、RF-DETR 检测和 RF-DETR 分割。其他系列和任务属实验性。

快速导出

python
1# ONNX (default)
2model.export()
3
4# TorchScript
5model.export(format="torchscript")
6
7# TensorRT (requires NVIDIA GPU + TensorRT)
8model.export(format="tensorrt")
9
10# OpenVINO (optimized for Intel hardware)
11model.export(format="openvino")
12
13# NCNN (via PNNX)
14model.export(format="ncnn")
15
16# CoreML (.mlpackage, macOS runtime)
17model.export(format="coreml")

全部导出参数

python
1path = model.export(
2 format="onnx", # "onnx", "torchscript", "tensorrt", "openvino", "ncnn", or "coreml"
3 output_path="model.onnx", # output file (auto-generated if None)
4 imgsz=640, # input resolution (default: model's native)
5 opset=None, # ONNX opset (auto: 13, or 17 for wrappers that need it)
6 simplify=True, # run onnxsim graph simplification
7 dynamic=True, # enable dynamic batch axis
8 half=False, # export in FP16
9 batch=1, # batch size for static graph
10 device=None, # device to trace on (default: model's current device)
11 int8=False, # INT8 quantization (TensorRT / OpenVINO only)
12 data=None, # calibration dataset for INT8
13 fraction=1.0, # fraction of calibration data to use
14 allow_download_scripts=False, # allow data.yaml download hooks during calibration
15 workspace=4.0, # TensorRT workspace size (GB)
16 min_batch=1, # TensorRT dynamic profile minimum batch
17 opt_batch=1, # TensorRT dynamic profile optimal batch
18 max_batch=8, # TensorRT dynamic profile maximum batch
19 hardware_compatibility="none", # TensorRT compatibility mode
20 gpu_device=0, # GPU device index for TensorRT
21 trt_config=None, # optional TensorRT YAML config path
22 compute_units="all", # CoreML routing: all, cpu_only, cpu_and_gpu, cpu_and_ne
23 nms=False, # CoreML embedded NMS where supported
24 iou=0.45, # CoreML embedded NMS IoU threshold
25 conf=0.25, # CoreML embedded NMS confidence threshold
26 verbose=False, # verbose logging
27)

OpenVINO INT8 导出还需要 nncf。NCNN 导出会写出一个目录,包含 model.ncnn.parammodel.ncnn.binmetadata.yaml。CoreML 导出会写出 .mlpackage 包,需要 coremltools,且不支持 INT8。

ONNX 元数据

导出的 ONNX 文件包含内嵌元数据:

示例值
libreyolo_version"1.2.0"
model_family"yolox"
model_size"s"
nb_classes"80"
names'{"0": "person", "1": "bicycle", ...}'
imgsz"640"
dynamic"True"
half"False"

使用 LibreYOLO("model.onnx") 加载导出文件时,会自动读回此元数据。

TorchScript 推理

通过相同的运行时后端预测 API 运行导出的 .torchscript 模型。

python
1from libreyolo import LibreYOLO
2
3model = LibreYOLO("model.torchscript")
4
5result = model("image.jpg", conf=0.25, iou=0.45, save=True)
6print(result.boxes.xyxy)

ONNX 推理

使用 ONNX Runtime 而非 PyTorch 运行推理。适用于没有 PyTorch 的部署环境。

python
1from libreyolo import LibreYOLO
2
3model = LibreYOLO("model.onnx")
4
5result = model("image.jpg", conf=0.25, iou=0.45, save=True)
6print(result.boxes.xyxy)

自动元数据

如果 ONNX 文件由 LibreYOLO 导出,类别名称和类别数会自动从内嵌元数据中读取:

python
1# Export with metadata
2model.export(format="onnx", output_path="model.onnx")
3
4# Load - names and nb_classes auto-populated
5onnx_model = LibreYOLO("model.onnx")
6print(onnx_model.names) # {0: "person", 1: "bicycle", ...}
7print(onnx_model.nb_classes) # 80

对于没有元数据的 ONNX 文件(如由其他工具导出),请手动指定 nb_classes

python
1model = LibreYOLO("external_model.onnx", nb_classes=20)

设备选择

python
1# Auto-detect (CUDA if available, else CPU)
2model = LibreYOLO("model.onnx", device="auto")
3
4# Force CPU
5model = LibreYOLO("model.onnx", device="cpu")
6
7# Force CUDA
8model = LibreYOLO("model.onnx", device="cuda")

预测参数

通过 LibreYOLO() 加载的运行时产物支持共享的运行时预测 API:

python
1result = model(
2 "image.jpg",
3 conf=0.25,
4 iou=0.45,
5 imgsz=640,
6 classes=[0, 2],
7 max_det=300,
8 save=True,
9 output_path="output/annotated.jpg", # final file path when save=True
10 color_format="auto",
11)

运行时后端不暴露仅限 PyTorch 的选项,如 tilingoverlap_ratiooutput_file_format

运行时后端的保存方式与 PyTorch 封装略有不同:如果设置 output_path,请传入最终文件路径,而非目录。如果省略它,当前后端默认保存在 runs/detections/ 下。

TensorRT 推理

使用 TensorRT 在 NVIDIA GPU 上以最大吞吐量运行推理。需要 CUDA 以及 TensorRT 的 Python 绑定。

python
1from libreyolo import LibreYOLO
2
3model = LibreYOLO("model.engine")
4
5result = model("image.jpg", conf=0.25, iou=0.45, save=True)
6print(result.boxes.xyxy)

通过 LibreYOLO() 加载的 TensorRT 产物支持与 ONNX 和 OpenVINO 相同的核心运行时预测 API,包括 save=True 时同样仅接受文件路径的 output_path 行为。

OpenVINO 推理

使用 OpenVINO 运行推理,针对 Intel CPU、GPU 和 VPU 进行了优化。

python
1from libreyolo import LibreYOLO
2
3model = LibreYOLO("model_openvino/")
4
5result = model("image.jpg", conf=0.25, iou=0.45, save=True)
6print(result.boxes.xyxy)

通过 LibreYOLO() 加载的 OpenVINO 目录会在存在时读取 metadata.yaml,并支持相同的核心运行时预测 API。

NCNN 推理

使用 NCNN 运行推理,以在 CPU 或支持 Vulkan 的 GPU 目标上实现轻量化部署。

python
1from libreyolo import LibreYOLO
2
3model = LibreYOLO("model_ncnn/")
4
5result = model("image.jpg", conf=0.25, iou=0.45, save=True)
6print(result.boxes.xyxy)

NCNN 导出目录包含 model.ncnn.parammodel.ncnn.bin,通常还有 metadata.yaml

CoreML 推理

在 macOS 上通过 CoreML 运行导出的 .mlpackage。CoreML 使用 compute_units 而非 PyTorch 设备字符串来路由执行。

python
1from libreyolo import LibreYOLO
2
3model = LibreYOLO("model.mlpackage", compute_units="all")
4
5result = model("image.jpg", conf=0.25, iou=0.45, save=True)
6print(result.boxes.xyxy)

支持的 compute_units 取值为 allcpu_onlycpu_and_gpucpu_and_ne

命令行工具

安装 LibreYOLO 会在你的 PATH 中注册一个 libreyolo 命令(入口点在 pyproject.toml 中)。该命令行镜像 Python API,并遵循 Ultralytics 风格的 key=value 语法。

子命令

命令用途
predict对图像、目录或视频运行推理
train在数据集上训练模型
val在数据集上评估模型
export导出为 ONNX / TorchScript / TensorRT / OpenVINO / NCNN / CoreML
checks打印 Python、torch、CUDA、GPU 及可选包信息
models列出已注册的模型系列和 CLI 快捷名称
formats列出支持的导出格式
cfg打印默认的训练配置 YAML
info加载模型并打印解析出的系列、尺寸、任务、设备和类别
metadata检视 .pt 文件中的原始检查点元数据
version打印 LibreYOLO + Python + torch 版本

模型名称快捷方式

命令行接受短名称(yolo9-c),它们会解析为权重文件名(LibreYOLO9c.pt) - 可通过 libreyolo models 查看。你也可以传入任意明确的检查点路径。

常用选项

命令重要选项
predictconf, iou, imgsz, classes, max_det, half, batch, tiling, overlap_ratio, output_file_format, project, name, exist_ok, face_detector
trainepochs, batch, imgsz, lr0, optimizer, scheduler, workers, seed, resume, amp, allow_download_scripts, dry_run
valsplit, batch, imgsz, conf, iou, max_det, half, data_dir, use_coco_eval, project, name, exist_ok, save_json, allow_download_scripts
exportformat, imgsz, batch, half, int8, dynamic, simplify, opset, data, fraction, device, allow_download_scripts, verbose

预测

bash
1# Flagship: YOLO9
2libreyolo predict model=yolo9-c source=image.jpg conf=0.25 save=true
3
4# Flagship: RF-DETR
5libreyolo predict model=rfdetr-s source=image.jpg save=true
6
7# Video - saved under runs/detect/predict*/
8libreyolo predict model=yolo9-c source=clip.mp4 save=true
9
10# Tiled inference for very large images
11libreyolo predict model=yolo9-c source=aerial.jpg tiling=true save=true
12
13# Gaze (requires a face detector)
14libreyolo predict model=LibreL2CSr50.pt source=portrait.jpg \
15 --face-detector path/to/face.pt save=true

训练

bash
1libreyolo train model=yolo9-c data=coco128.yaml epochs=300 batch=16 device=0
2
3# Dry-run prints the resolved config without launching training
4libreyolo train model=yolo9-c data=coco128.yaml --dry-run

验证

bash
1libreyolo val model=runs/train/exp/weights/best.pt data=coco128.yaml split=val

导出

bash
1libreyolo export model=runs/train/exp/weights/best.pt format=onnx dynamic=true
2libreyolo export model=best.pt format=tensorrt half=true
3libreyolo export model=best.pt format=openvino int8=true data=coco128.yaml
4libreyolo export model=best.pt format=coreml

机器可读输出

每个命令都接受 --json(结构化的标准输出,便于管道传入脚本或智能体)和 --quiet(抑制 stderr 进度行)。核心的 predicttrainvalexport 命令还接受 --help-json,可将其参数 schema 以 JSON 形式导出。

bash
1libreyolo predict model=yolo9-c source=img.jpg --json | jq .
2
3libreyolo train --help-json > train_schema.json

API 参考

LibreYOLO(工厂函数)

python
1LibreYOLO(
2 model_path: str,
3 *,
4 device: str = "auto",
5 task: str | None = None, # override only when a custom artifact is ambiguous
6 nb_classes: int | None = None, # mainly for external exported artifacts
7 compute_units: str = "all", # CoreML only: all, cpu_only, cpu_and_gpu, cpu_and_ne
8) -> model wrapper or runtime backend

优先使用官方检查点文件名和导出产物路径,再让工厂函数解析细节。它可处理 PyTorch 检查点、.onnx.torchscript.engine.tensorrt.mlpackage、包含 model.xml 的 OpenVINO 目录,以及包含 model.ncnn.parammodel.ncnn.bin 的 NCNN 目录。task 参数用于含糊的自定义产物;否则解析来自检查点元数据、文件名后缀和系列默认值。

预测(PyTorch 模型封装)

python
1model(
2 source, # image input (see supported formats)
3 *,
4 conf: float = 0.25,
5 iou: float = 0.45,
6 imgsz: int = None,
7 device: str = "auto",
8 classes: list[int] = None,
9 max_det: int = 300,
10 augment: bool = False,
11 save: bool = False,
12 batch: int = 1,
13 stream: bool = False,
14 vid_stride: int = 1,
15 show: bool = False,
16 output_path: str = None,
17 color_format: str = "auto",
18 tiling: bool = False,
19 overlap_ratio: float = 0.2,
20 output_file_format: str = None,
21) -> Results | list[Results] | Generator[Results, None, None]

预测(运行时后端)

python
1backend(
2 source,
3 *,
4 conf: float = 0.25,
5 iou: float = 0.45,
6 imgsz: int = None,
7 classes: list[int] = None,
8 max_det: int = 300,
9 save: bool = False,
10 batch: int = 1,
11 output_path: str = None, # final file path when save=True
12 color_format: str = "auto",
13) -> Results | list[Results]

如果运行时后端省略 output_path,当前默认保存位置为 runs/detections/

Results

python
1result = Results(
2 boxes: Boxes | None,
3 orig_shape: tuple[int, int], # (height, width)
4 path: str | None,
5 names: dict[int, str],
6 masks: Masks | None = None,
7 keypoints: Keypoints | None = None,
8 probs: Probs | None = None,
9 obb: OBB | None = None,
10 gaze: Gaze | None = None,
11 speed: dict[str, float] | None = None,
12 track_id = None,
13 frame_idx: int | None = None,
14)
15
16len(result) # number of detections
17result.cpu() # copy with tensors on CPU
18result.cuda() # copy with tensors on CUDA
19result.numpy() # copy with numpy arrays
20result.summary() # list[dict] with boxes, masks, gaze, and track_id when present
21result.to_json() # JSON string from summary()

Boxes

python
1boxes = Boxes(boxes, conf, cls)
2
3boxes.xyxy # (N, 4) tensor - x1, y1, x2, y2
4boxes.xywh # (N, 4) tensor - cx, cy, w, h
5boxes.conf # (N,) tensor - confidence scores
6boxes.cls # (N,) tensor - class IDs
7boxes.id # (N,) track IDs when tracking, else None
8boxes.is_track # True when track IDs are attached
9boxes.data # (N, 6) [xyxy, conf, cls], or (N, 7) with track IDs
10
11len(boxes) # number of boxes
12boxes.cpu() # copy on CPU
13boxes.numpy() # copy as numpy arrays

任务负载

python
1result.masks.data # segmentation masks, (N, H, W)
2result.masks.xy # list of mask contours in pixel coordinates
3result.masks.xyn # normalized mask contours
4
5result.keypoints.xy # pose keypoint coordinates
6result.keypoints.xyn # normalized keypoint coordinates
7result.keypoints.conf # keypoint confidence when present
8
9result.gaze.data # (N, 2): pitch, yaw in radians
10result.gaze.pitch_deg # pitch in degrees
11result.gaze.yaw_deg # yaw in degrees
12result.gaze.direction_3d # approximate 3D direction vectors

model.export()

python
1model.export(
2 format: str = "onnx", # "onnx", "torchscript", "tensorrt", "openvino", "ncnn", or "coreml"
3 *,
4 output_path: str | None = None,
5 imgsz: int | None = None,
6 opset: int | None = None, # auto: 13, or 17 for wrappers that need it
7 simplify: bool = True,
8 dynamic: bool = True,
9 half: bool = False,
10 batch: int = 1,
11 device: str | None = None,
12 int8: bool = False,
13 data: str | None = None, # calibration data for INT8
14 fraction: float = 1.0, # fraction of calibration data
15 allow_download_scripts: bool = False,
16 workspace: float = 4.0, # TensorRT workspace (GB)
17 min_batch: int = 1, # TensorRT dynamic profile minimum batch
18 opt_batch: int = 1, # TensorRT dynamic profile optimal batch
19 max_batch: int = 8, # TensorRT dynamic profile maximum batch
20 hardware_compatibility: str = "none",
21 gpu_device: int = 0,
22 trt_config = None, # optional TensorRT YAML config path
23 compute_units: str = "all", # CoreML only
24 nms: bool = False, # CoreML embedded NMS where supported
25 iou: float = 0.45, # CoreML embedded NMS IoU threshold
26 conf: float = 0.25, # CoreML embedded NMS confidence threshold
27 verbose: bool = False,
28) -> str # path to exported file or directory

model.val()

python
1model.val(
2 data: str = None, # path to data.yaml
3 batch: int = 16,
4 imgsz: int = None,
5 conf: float = 0.001,
6 iou: float = 0.6,
7 workers: int = 4,
8 allow_download_scripts: bool = False,
9 device: str = None,
10 split: str = "val", # "val", "test", or "train"
11 augment: bool = False,
12 save_json: bool = False,
13 verbose: bool = True,
14) -> dict

返回(COCO 评估,默认):

python
1{
2 "metrics/mAP50-95": float, # COCO primary metric
3 "metrics/mAP50": float,
4 "metrics/mAP75": float,
5 "metrics/mAP_small": float,
6 "metrics/mAP_medium": float,
7 "metrics/mAP_large": float,
8 "metrics/AR1": float,
9 "metrics/AR10": float,
10 "metrics/AR100": float,
11 "metrics/AR_small": float,
12 "metrics/AR_medium": float,
13 "metrics/AR_large": float,
14}

model.train() (YOLO9)

python
1model.train(
2 data: str, # path to data.yaml (required)
3 *,
4 epochs: int = 300,
5 batch: int = 16,
6 imgsz: int = 640,
7 lr0: float = 0.01,
8 optimizer: str = "SGD",
9 device: str = "",
10 workers: int = 8,
11 seed: int = 0,
12 project: str = "runs/train",
13 name: str = "yolo9_exp",
14 exist_ok: bool = False,
15 resume: bool = False,
16 amp: bool = True,
17 patience: int = 50,
18 allow_download_scripts: bool = False,
19 callbacks = None,
20) -> dict

返回标准的 LibreYOLO 训练字典,包含 final_lossbest_mAP50best_mAP50_95best_epochsave_dirbest_checkpointlast_checkpoint

model.train() (RF-DETR)

python
1model.train(
2 data: str, # path to data.yaml
3 epochs: int = 100,
4 batch_size: int = 4,
5 lr: float = 1e-4,
6 output_dir: str = "runs/train",
7 resume: str = None,
8 **kwargs, # additional RF-DETR training args
9) -> dict

YOLO-NAS、D-FINE、DEIM、DEIMv2、EC、PicoDet、DAMO-YOLO、RT-DETRv2/v4 和 RTMDet 还提供额外的实验性训练器。它们遵循相同的 model.train(data="...yaml", ...) 形式,但其默认值和实验性门控因系列而异。

运行时产物加载

通过 LibreYOLO() 加载导出产物,方式与加载 PyTorch 检查点相同。工厂函数会根据路径选择 ONNX Runtime、TorchScript、TensorRT、OpenVINO、NCNN 或 CoreML:

python
1from libreyolo import LibreYOLO
2
3model = LibreYOLO("model.onnx")
4model = LibreYOLO("model.torchscript")
5model = LibreYOLO("model.engine")
6model = LibreYOLO("model_openvino/")
7model = LibreYOLO("model_ncnn/")
8model = LibreYOLO("model.mlpackage", compute_units="all")

高级集成可以访问更底层的运行时模块,但普通应用代码应坚持使用工厂函数路径。

ValidationConfig

python
1from libreyolo import ValidationConfig
2
3config = ValidationConfig(
4 data="coco128.yaml",
5 data_dir=None, # override dataset root directory
6 split="val", # "val", "test", or "train"
7 batch_size=16,
8 imgsz=640,
9 conf_thres=0.001,
10 iou_thres=0.6,
11 max_det=300,
12 iou_thresholds=( # mAP IoU sweep
13 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, 0.95,
14 ),
15 device="auto",
16 save_dir=None,
17 save_json=False,
18 verbose=True,
19 num_workers=4,
20 half=False,
21 augment=False, # test-time augmentation (TTA)
22 allow_download_scripts=False,
23 # Pose-only fields (PoseValidator)
24 keypoints_json=None,
25 images_dir=None,
26 oks_sigmas=None,
27)
28
29# Load/save YAML
30config = ValidationConfig.from_yaml("config.yaml")
31config.to_yaml("config.yaml")

架构指南

本节面向希望了解代码库内部实现的贡献者。

基类设计

PyTorch 模型系列继承自 libreyolo/models/base/model.py 中的 BaseModel。子类需实现以下抽象方法:

方法用途
_init_model()构建并返回 nn.Module
_get_available_layers()返回层名称到模块的映射
_get_preprocess_numpy()返回用于导出/校准的 NumPy 预处理器
_preprocess()图像到张量的转换
_forward()模型前向传播
_postprocess()原始输出到检测字典

BaseModel 提供共享的封装行为:预测、导出、验证、尺寸/名称元数据和训练辅助函数。实际的单图、批量和分块推理流程位于 libreyolo/models/base/inference.py,而部署运行时位于 libreyolo/backends/ 下。

包结构

text
1libreyolo/
2 __init__.py # Public API exports + deprecated-alias resolver
3 tasks.py # Task types, suffix conventions, resolution rules
4 assets/parkour.jpg # SAMPLE_IMAGE
5 models/
6 __init__.py # LibreYOLO() factory + model registry bootstrap
7 base/
8 model.py # BaseModel - shared wrapper behaviour
9 inference.py # Shared prediction pipeline (image/dir/video/tiled)
10 yolox/ # LibreYOLOX (detect)
11 yolo9/ # LibreYOLO9 (detect, segment)
12 yolo9_e2e/ # LibreYOLO9E2E (detect)
13 yolonas/ # LibreYOLONAS (detect, pose)
14 dfine/ # LibreDFINE (detect)
15 deim/ # LibreDEIM (detect)
16 deimv2/ # LibreDEIMv2 (detect)
17 rtdetr/ # LibreRTDETR (detect)
18 rtdetrv2/ # LibreRTDETRv2 (detect)
19 rtdetrv4/ # LibreRTDETRv4 (detect)
20 rfdetr/ # LibreRFDETR (detect, segment) - lazy-loaded
21 ec/ # LibreEC / EdgeCrafter (detect, pose, segment)
22 picodet/ # LibrePICODET (detect)
23 damoyolo/ # LibreDAMOYOLO (detect)
24 rtmdet/ # LibreRTMDet (detect)
25 l2cs/ # LibreL2CS (gaze, inference-only)
26 backends/
27 base.py
28 onnx.py # ONNX Runtime loader
29 torchscript.py # TorchScript loader
30 tensorrt.py # TensorRT loader
31 openvino.py # OpenVINO loader
32 ncnn.py # NCNN loader
33 coreml.py # CoreML loader
34 export/
35 exporter.py # BaseExporter and format registry
36 onnx.py / torchscript.py / tensorrt.py / openvino.py / ncnn.py / coreml.py
37 config.py / calibration.py
38 training/
39 trainer.py # Shared trainer scaffolding
40 config.py # TrainConfig dataclass (single source of truth)
41 augment.py / callbacks.py / distributed.py / ema.py / scheduler.py
42 artifacts.py / train_config.yaml
43 # Per-family trainers live in models/<family>/trainer.py
44 validation/
45 config.py # ValidationConfig
46 base.py / preprocessors.py
47 detection_validator.py # DetectionValidator, SegmentationValidator
48 pose_validator.py # PoseValidator
49 coco_evaluator.py # COCOEvaluator
50 tracking/
51 tracker.py # ByteTracker
52 config.py # TrackConfig
53 kalman_filter.py / matching.py / strack.py
54 cli/
55 __init__.py # libreyolo entrypoint (Typer app)
56 commands/ # predict / train / val / export / special
57 aliases.py / config.py / parsing.py / output.py / errors.py
58 utils/
59 results.py # Results, Boxes, Masks, Keypoints, Probs, OBB, Gaze
60 image_loader.py # Unified image loading
61 video.py # VideoSource, VideoWriter, video inference loop
62 general.py # Path helpers, NMS, tiling utilities
63 download.py / drawing.py / logging.py / predict_args.py
64 serialization.py / box_ops.py
65 data/
66 dataset.py / pose_dataset.py / utils.py / yolo_coco_api.py
67 config/
68 datasets/ # Built-in dataset YAML configs (coco8, coco128, coco5000, coco, etc.)
69 export/ # TensorRT default YAML

添加新的模型系列

  1. 1创建 libreyolo/models/newmodel/model.py,其中包含一个继承 BaseModel 的类
  2. 2按需设置 FAMILYFILENAME_PREFIXINPUT_SIZESSUPPORTED_TASKSDEFAULT_TASK
  3. 3实现注册表钩子,如 can_load()detect_size()detect_nb_classes()detect_size_from_filename()
  4. 4实现该系列所需的模型初始化、预处理、前向、后处理、训练和验证钩子
  5. 5libreyolo/models/newmodel/ 下创建配套的网络和工具
  6. 6将该导入添加到 libreyolo/models/__init__.py;导入运行时会进行子类注册
  7. 7libreyolo/__init__.py 导出该类
  8. 8(可选)如果验证预处理与标准路径不同,覆盖 val_preprocessor_class

导出架构

用户代码应通过 model.export(...) 导出。在内部,libreyolo/export/exporter.py 中的 BaseExporter 持有格式注册表,具体的导出器通过子类注册来注册自身。

python
1from libreyolo import LibreYOLO
2
3model = LibreYOLO("LibreYOLO9c.pt")
4model.export(format="onnx")

要添加新的导出格式,请实现一个具有唯一 format_name 的新 BaseExporter 子类,并从 libreyolo/export/exporter.py 导入它,以填充注册表。

数据集格式

训练和验证使用通过 data.yaml 加载的数据集配置。检测、分割、姿态和 RF-DETR 训练都经由该加载器;标签文件内容因任务而异。

data.yaml 结构

data.yaml
1path: /absolute/path/to/dataset # dataset root
2train: images/train # directory path, relative to path
3val: images/val # directory path, relative to path
4test: images/test # optional
5
6nc: 80 # number of classes
7names: [ # class names
8 "person", "bicycle", "car", "motorcycle", "airplane",
9 "bus", "train", "truck", "boat", "traffic light",
10 # ...
11]

配置解析与下载

数据集配置按以下顺序解析:明确路径、当前工作目录,然后是 libreyolo/config/datasets/ 下的内置配置。数据集根目录默认在 ~/datasets 下,可通过 LIBREYOLO_DATASETS_DIR 覆盖。

trainvaltest 可以是目录、.txt 文件或路径列表。YAML 下载钩子受保护;仅对可信配置传入 allow_download_scripts=True

文件列表变体

同样的 YAML 格式也可将 trainvaltest 指向每行一个图像路径的 .txt 文件:

coco.yaml
1path: /absolute/path/to/coco
2train: train2017.txt
3val: val2017.txt
4test: test-dev2017.txt
5
6nc: 80
7names: ["person", "bicycle", "car", "..."]

目录布局

text
1dataset/
2 images/
3 train/
4 img001.jpg
5 img002.jpg
6 val/
7 img003.jpg
8 labels/
9 train/
10 img001.txt
11 img002.txt
12 val/
13 img003.txt

检测标签格式

每张图像对应一个文本文件。每行表示一个目标:

text
1<class_id> <center_x> <center_y> <width> <height>

所有坐标都相对于图像尺寸归一化到 [0, 1]。

示例(img001.txt):

img001.txt
10 0.5 0.4 0.3 0.6
22 0.1 0.2 0.05 0.1

分割标签格式

分割使用 YOLO 多边形行。数据集加载器从多边形顶点推导边界框,并在启用分割加载时保留多边形环:

text
1<class_id> <x1> <y1> <x2> <y2> ... <xn> <yn>

姿态标签格式

姿态标签在边界框之后追加关键点。在 data.yaml 中添加 kpt_shapeflip_idx,使加载器知道关键点数量和水平翻转排列。

yaml
1kpt_shape: [17, 3]
2flip_idx: [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15]
text
1<class_id> <cx> <cy> <w> <h> <kx1> <ky1> <v1> ... <kxK> <kyK> <vK>

内置数据集

LibreYOLO 在 libreyolo/config/datasets/ 下提供内置数据集配置,并可在首次使用时自动下载支持的数据集:

python
1# These download automatically on first use
2results = model.val(data="coco8.yaml")
3results = model.train(data="coco128.yaml", epochs=10)