这是上一个版本 v1.2.0 的文档。最新文档为 v1.3.0,可在版本菜单中找到。
简介
v1.2.0 验证范围
经过深度测试的路径是 YOLO9 和 RF-DETR 的检测、训练与推理,包括 RF-DETR 分割。
其他模型系列、任务和多 GPU 工作流均可用,但属实验性。
LibreYOLO 是一个采用 MIT 许可证的目标检测工具包。v1.2.0 提供了广泛的模型目录,但经过验证的支持范围有意保持精简:
- YOLO9 检测 - CNN 路径。
- RF-DETR 检测 - Transformer 路径。
- RF-DETR 分割 - 经过深度测试的分割路径。
我们推荐将这些路径作为新项目的默认选择,因为它们在检测、训练和推理方面经过了最充分的测试。其他受支持的系列和任务通过同一个统一的 LibreYOLO() 工厂函数工作,但在 v1.2.0 中仍属实验性。如有特定理由再使用它们。
1 from libreyolo import LibreYOLO, SAMPLE_IMAGE 2 3 # Default: YOLO9 detection 4 model = LibreYOLO("LibreYOLO9c.pt") 5 result = model(SAMPLE_IMAGE, conf=0.25, save=True) 6 7 print(f"Detected {len(result)} objects") 8 print(result.boxes.xyxy) 9 print(result.saved_path)
核心特性
- 针对 YOLO9 检测、RF-DETR 检测和 RF-DETR 分割的深度测试与推荐默认配置
- 统一的
LibreYOLO()工厂函数,用于加载检查点、导出产物和运行时 - 通过一致的 API 支持检测、分割、姿态和视线任务
- 支持图像、目录和视频推理(大尺寸帧可选用分块推理)
- 通过 ByteTrack 内置多目标跟踪
- 支持 ONNX、TorchScript、TensorRT、OpenVINO、NCNN 和 CoreML 导出(含内嵌元数据),并提供匹配的运行时后端
- 兼容 COCO 的验证与 mAP 指标,并提供分割和姿态验证器
- Ultralytics 风格的
libreyolo命令行工具,用于 predict / train / val / export - 接受任意图像格式:文件路径、URL、PIL、NumPy、PyTorch 张量、原始字节
兼容性
可将此矩阵用作 v1.2.0 的快速支持速查表。对勾表示该路径在经过验证的文档范围内受支持,exp 表示该路径存在但属实验性,空白单元格表示当前不受支持或不应依赖。
| 模型系列 | v1.2.0 状态 | 推理 | 训练 | 检测 | 分割 | 姿态 | 视线 | ONNX | TorchScript | TensorRT | OpenVINO | NCNN | CoreML |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| YOLO9 | 已验证 detect,单 GPU | ✓ | ✓ | ✓ | exp | Not currently supported | Not currently supported | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| RF-DETR | 已验证 detect + segment,单 GPU | ✓ | ✓ | ✓ | ✓ | Not currently supported | Not currently supported | ✓ | exp | ✓ | ✓ | Not currently supported | exp |
| YOLOX | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | exp | exp |
| YOLO9-E2E | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | exp | Not currently supported |
| YOLO-NAS | 实验性 | exp | exp | exp | Not currently supported | exp | Not currently supported | exp | exp | exp | exp | exp | Not currently supported |
| D-FINE | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | Not currently supported | Not currently supported |
| DEIM | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | Not currently supported | Not currently supported |
| DEIMv2 | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | Not currently supported | Not currently supported |
| RT-DETR | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | Not currently supported | exp |
| PicoDet | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | exp | Not currently supported |
| EC | 实验性 | exp | exp | exp | exp | exp | Not currently supported | exp | exp | exp | exp | Not currently supported | Not currently supported |
| RT-DETRv2 | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | exp | Not currently supported |
| RT-DETRv4 | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | exp | Not currently supported |
| DAMO-YOLO | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | exp | Not currently supported |
| RTMDet | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | exp | Not currently supported |
| L2CS | 实验性,仅推理 | exp | Not currently supported | Not currently supported | Not currently supported | Not currently supported | exp | Not currently supported | Not currently supported | Not currently supported | Not currently supported | Not currently supported | Not currently supported |
CoreML 导出会生成 .mlpackage 包,并需要 libreyolo[coreml]。CoreML 推理仅限 macOS,不支持 INT8,且 RF-DETR、D-FINE、DEIM、DEIMv2 或 EC 不提供内嵌的 CoreML NMS。
安装
环境要求
- Python 3.10+
- PyTorch 1.13+ 与 torchvision 0.11+
从 PyPI 安装
1 pip install libreyolo
本文档对应即将发布的 v1.2.0 开发分支。在 v1.2.0 发布到 PyPI 之前,请使用源码安装以获得本页所述功能。
从源码安装
1 git clone https://github.com/LibreYOLO/libreyolo.git 2 cd libreyolo 3 git checkout dev 4 pip install -e .
可选依赖
1 # ONNX export and inference 2 pip install libreyolo[onnx] 3 # or: pip install onnx onnxsim onnxruntime 4 5 # RT-DETR compatibility extra (currently no extra packages) 6 pip install libreyolo[rtdetr] 7 8 # RF-DETR support 9 pip install libreyolo[rfdetr] 10 # or: pip install transformers 11 12 # TensorRT export and inference (NVIDIA GPU) 13 pip install libreyolo[tensorrt] 14 # Installs TensorRT CUDA 12 Python packages on Linux/Windows. 15 # Host driver/CUDA compatibility still matters. 16 17 # OpenVINO export and inference (Intel CPU/GPU/VPU) 18 pip install libreyolo[openvino] 19 # INT8 export also needs: pip install nncf 20 21 # NCNN export and inference 22 pip install libreyolo[ncnn] 23 # or: pip install pnnx ncnn 24 25 # ByteTrack API compatibility extra 26 pip install libreyolo[tracking] 27 # Tracking dependencies are part of the base install in v1.2.0 dev. 28 29 # CoreML export and inference (macOS only for runtime) 30 pip install libreyolo[coreml] 31 # or: pip install coremltools 32 33 # L2CS gaze optional auto-download helper 34 pip install libreyolo[gaze] 35 # Optional parity with the upstream RetinaFace-based L2CS pipeline 36 pip install libreyolo[gaze-retinaface] 37 38 # Install every optional LibreYOLO extra 39 pip install libreyolo[all]
如果使用 uv,最可靠的方式是为每个 extra 创建独立的虚拟环境:
1 # ONNX environment 2 uv venv .venv-onnx 3 uv pip install --python .venv-onnx/bin/python -e '.[onnx]' 4 5 # RT-DETR environment 6 uv venv .venv-rtdetr 7 uv pip install --python .venv-rtdetr/bin/python -e '.[rtdetr]' 8 9 # Repeat with .[rfdetr], .[openvino], .[ncnn], .[coreml], .[gaze], .[tracking], or .[tensorrt] as needed
这可避免改动项目环境,并使可选依赖保持隔离。TensorRT、OpenVINO、NCNN 和 CoreML 等特定厂商的 extra 仍可能需要平台专属的原生包。
快速开始
若要使用经过最充分测试的路径,请选择单 GPU 的 YOLO9 检测、RF-DETR 检测或 RF-DETR 分割。它们通过同一个工厂函数加载,接受相同的输入,并返回相同的 Results 对象,因此你可以在它们之间切换而无需改动周边代码。
YOLO9 - CNN 旗舰
1 from libreyolo import LibreYOLO, SAMPLE_IMAGE 2 3 # Use the official checkpoint name and let the factory resolve the details 4 model = LibreYOLO("LibreYOLO9c.pt") 5 6 # Run on a single image (SAMPLE_IMAGE ships with the package) 7 result = model(SAMPLE_IMAGE) 8 9 print(f"Found {len(result)} objects") 10 print(result.boxes.xyxy) # bounding boxes (N, 4) 11 print(result.boxes.conf) # confidence scores (N,) 12 print(result.boxes.cls) # class IDs (N,)
RF-DETR - Transformer 旗舰
1 from libreyolo import LibreYOLO, SAMPLE_IMAGE 2 3 # Same factory, same call shape - just point at an RF-DETR checkpoint 4 model = LibreYOLO("LibreRFDETRs.pt") 5 result = model(SAMPLE_IMAGE) 6 7 print(f"Found {len(result)} objects") 8 print(result.boxes.xyxy)
保存标注后的输出
1 result = model(SAMPLE_IMAGE, save=True) 2 print(result.saved_path) # e.g. runs/detect/predict/parkour.jpg
处理目录
1 results = model("images/", save=True, batch=4) 2 for r in results: 3 print(f"{r.path}: {len(r)} detections")
可用模型
推荐的已验证路径:YOLO9 检测或 RF-DETR 检测 / 分割
这些模型的检测、训练与推理获得了最充分的测试。在 v1.2.0 中,请将其他系列、任务和多 GPU 工作流视为实验性。
LibreYOLO 在 v1.2.0 中提供一小部分经过验证的范围,外加更广泛的受支持模型目录。每个模型都通过同一个 LibreYOLO() 工厂函数加载,但只有下方经过验证的路径才应视为经过深度测试。
YOLO9 - CNN 旗舰
推荐| 尺寸 | 代号 | 输入尺寸 | 适用场景 | 检测检查点 |
|---|---|---|---|---|
| Tiny | "t" | 640 | 快速推理 | LibreYOLO9t.pt |
| Small | "s" | 640 | 均衡 | LibreYOLO9s.pt |
| Medium | "m" | 640 | 更高精度 | LibreYOLO9m.pt |
| Compact | "c" | 640 | 最佳精度 | LibreYOLO9c.pt |
实验性 分割检查点: LibreYOLO9t-seg.pt, LibreYOLO9s-seg.pt, LibreYOLO9m-seg.pt, LibreYOLO9c-seg.pt。参见 分割 章节。
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("LibreYOLO9c.pt") 4 # Experimental segmentation variant 5 # model = LibreYOLO("LibreYOLO9c-seg.pt")
RF-DETR - Transformer 旗舰
推荐| 尺寸 | 代号 | 输入尺寸 | 适用场景 | 检测检查点 |
|---|---|---|---|---|
| Nano | "n" | 384 | 边缘设备 | LibreRFDETRn.pt |
| Small | "s" | 512 | 均衡 | LibreRFDETRs.pt |
| Medium | "m" | 576 | 更高精度 | LibreRFDETRm.pt |
| Large | "l" | 704 | 最高精度 | LibreRFDETRl.pt |
深度测试 分割检查点: LibreRFDETRn-seg.pt, LibreRFDETRs-seg.pt, LibreRFDETRm-seg.pt, LibreRFDETRl-seg.pt, LibreRFDETRx-seg.pt, LibreRFDETRxx-seg.pt。参见 分割 章节。
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("LibreRFDETRs.pt") 4 # Segmentation variants exist for every RF-DETR size 5 # model = LibreYOLO("LibreRFDETRs-seg.pt")
其他受支持的系列
具备检测能力的系列,与经过验证的路径共享同一个工厂函数和 API。它们在 v1.2.0 中属实验性。每个检查点名称都链接到 LibreYOLO 组织 在 Hugging Face 上的模型卡;将任意名称传给 LibreYOLO(),工厂函数会在首次使用时自动获取。
托管说明: YOLO-NAS 检查点(上方以纯文本显示)托管在 Deci 的 CDN 上,遵循其专有权重许可证,而非 LibreYOLO 的 Hugging Face 组织。工厂函数仍会在首次使用时自动下载它们。
专用模型
| 系列 | 状态 | 任务 | 检查点 |
|---|---|---|---|
| L2CS | 实验性 | gaze(仅推理) - 参见 视线估计 | LibreL2CSr50.pt |
L2CS 架构尺寸包括 r18、r34、r50、r101 和 r152,但上游发布的 Gaze360 检查点是 ResNet-50。安装 libreyolo[gaze] 以使用可选的 Google Drive 辅助下载,或为其他尺寸传入本地检查点路径。
工厂函数
对所有模型和运行时都使用 LibreYOLO() 工厂函数。传入官方检查点名称或导出产物路径,让它自动选择正确的模型系列、任务、类别数和运行时:
1 from libreyolo import LibreYOLO 2 3 # Default: YOLO9 detection 4 model = LibreYOLO("LibreYOLO9c.pt") 5 6 # Flagship: RF-DETR 7 model = LibreYOLO("LibreRFDETRs.pt") 8 9 # Segmentation checkpoints use the same factory path 10 model = LibreYOLO("LibreRFDETRs-seg.pt") # validated segmentation 11 model = LibreYOLO("LibreYOLO9c-seg.pt") # experimental segmentation 12 13 # Exported deployment formats 14 model = LibreYOLO("model.onnx") # ONNX Runtime 15 model = LibreYOLO("model.engine") # TensorRT 16 model = LibreYOLO("model.mlpackage") # CoreML (macOS) 17 model = LibreYOLO("model_openvino/") # OpenVINO (directory) 18 model = LibreYOLO("model_ncnn/") # NCNN (directory)
对于可识别的官方检查点文件名,LibreYOLO 可自动下载缺失的权重。对于自定义文件名,请指向明确的本地路径。实验性系列仍通过同一个工厂函数加载,但新项目请坚持使用 YOLO9 检测或 RF-DETR 检测/分割。如有特定理由再使用它们。
任务与文件名
LibreYOLO 使用统一的文件名约定,使工厂函数仅凭检查点名称即可识别系列、尺寸和任务:
1 Libre<FAMILY><size>[-<task>].pt
任务后缀
| 任务 | 规范名称 | 文件名后缀 |
|---|---|---|
| 检测 | "detect" | (无 - 隐式) |
| 实例分割 | "segment" | -seg |
| 姿态估计 | "pose" | -pose |
| 分类 | "classify" | -cls |
| 视线估计 | "gaze" | -gaze |
工厂函数在 API 边界接受别名("detection"、"seg"、"keypoints" 等) - 但文件名中只出现规范名称。
解析优先级
加载模型时,任务按以下顺序解析:
1 explicit task= → checkpoint["task"] → filename suffix → family default
1 from libreyolo import LibreYOLO 2 3 # 1. Filename suffix decides → segment 4 model = LibreYOLO("LibreRFDETRs-seg.pt") 5 6 # 2. Override regardless of filename 7 model = LibreYOLO("custom_weights.pt", task="segment") 8 9 # 3. Detection is implicit 10 model = LibreYOLO("LibreYOLO9c.pt") # task="detect"
各系列任务支持
| 系列 | v1.2.0 状态 | 默认 | 支持的任务 |
|---|---|---|---|
| YOLO9 | detect 单 GPU 深度测试;segment 与多 GPU 实验性 | detect | detect, segment |
| RF-DETR | detect 与 segment 单 GPU 深度测试;多 GPU 实验性 | detect | detect, segment |
| YOLOX | 实验性 | detect | detect |
| YOLO9-E2E | 实验性 | detect | detect |
| YOLO-NAS | 实验性 | detect | detect, pose |
| D-FINE / DEIM / DEIMv2 | 实验性 | detect | detect |
| RT-DETR / RT-DETRv2 / RT-DETRv4 | 实验性 | detect | detect |
| PicoDet | 实验性 | detect | detect |
| EdgeCrafter (EC) | 实验性 | detect | detect, pose, segment |
| DAMO-YOLO / RTMDet | 实验性 | detect | detect |
| L2CS | 实验性 | gaze | gaze(仅推理) |
示例
1 # Detection (implicit) 2 LibreYOLO9c.pt 3 LibreRFDETRs.pt 4 LibreRTDETRr50.pt 5 6 # Segmentation 7 LibreYOLO9c-seg.pt 8 LibreRFDETRs-seg.pt 9 LibreECm-seg.pt 10 11 # Pose 12 LibreYOLONASn-pose.pt 13 LibreECs-pose.pt 14 15 # Gaze 16 LibreL2CSr50.pt # gaze is L2CS's only task - suffix optional
已弃用的别名
LibreYOLORTDETR 和 LibreYOLORFDETR 分别是 LibreRTDETR 和 LibreRFDETR 的旧名称。它们仍可解析,但会触发 DeprecationWarning - 请在方便时更新导入。
预测
单 GPU 预测路径针对 YOLO9 检测、RF-DETR 检测和 RF-DETR 分割经过深度测试。其他系列和任务使用相同的 API,但在 v1.2.0 中属实验性。
基本预测
1 result = model("image.jpg")
全部预测参数
1 result = model( 2 "image.jpg", 3 conf=0.25, # confidence threshold (default: 0.25) 4 iou=0.45, # NMS IoU threshold (default: 0.45) 5 imgsz=640, # input size override (default: model's native) 6 device="auto", # "auto", "cpu", "mps", "0", "cuda:0", ... 7 classes=[0, 2, 5], # filter to specific class IDs (default: all) 8 max_det=300, # max detections per image (default: 300) 9 augment=False, # test-time augmentation where implemented 10 save=True, # save annotated image (default: False) 11 batch=4, # directory batch size 12 stream=False, # video only: yield frame results instead of a list 13 vid_stride=1, # video only: process every N-th frame 14 show=False, # video only: display annotated frames 15 tiling=False, # large-image tiled detection 16 overlap_ratio=0.2, # tile overlap ratio 17 output_path="out/", # where to save (default: runs/detect/predict*/) 18 color_format="auto", # "auto", "rgb", or "bgr" 19 output_file_format="png", # output format: "jpg", "png", "webp" 20 )
model.predict(...) 是 model(...) 的别名。
支持的输入格式
LibreYOLO 接受以下任意格式的图像:
1 # File path (string or pathlib.Path) 2 result = model("photo.jpg") 3 result = model(Path("photo.jpg")) 4 5 # URL 6 result = model("https://example.com/image.jpg") 7 result = model("s3://bucket/image.jpg") 8 result = model("gs://bucket/image.jpg") 9 10 # PIL Image 11 from PIL import Image 12 img = Image.open("photo.jpg") 13 result = model(img) 14 15 # NumPy array (HWC or CHW, RGB or BGR, uint8 or float32) 16 import numpy as np 17 arr = np.random.randint(0, 255, (480, 640, 3), dtype=np.uint8) 18 result = model(arr) 19 20 # OpenCV (BGR) - specify color_format 21 import cv2 22 frame = cv2.imread("photo.jpg") 23 result = model(frame, color_format="bgr") 24 25 # PyTorch tensor (CHW or NCHW) 26 import torch 27 tensor = torch.randn(3, 640, 640) 28 result = model(tensor) 29 30 # Raw bytes 31 with open("photo.jpg", "rb") as f: 32 result = model(f.read()) 33 34 # BytesIO 35 from io import BytesIO 36 result = model(BytesIO(open("photo.jpg", "rb").read())) 37 38 # Directory of images 39 results = model("images/", batch=4)
处理结果
每次预测都会返回一个 Results 对象(处理目录时返回其列表):
1 result = model("image.jpg") 2 3 # Number of detections 4 len(result) # e.g., 5 5 6 # Bounding boxes in xyxy format (x1, y1, x2, y2) 7 result.boxes.xyxy # tensor of shape (N, 4) 8 9 # Bounding boxes in xywh format (center_x, center_y, width, height) 10 result.boxes.xywh # tensor of shape (N, 4) 11 12 # Confidence scores 13 result.boxes.conf # tensor of shape (N,) 14 15 # Class IDs 16 result.boxes.cls # tensor of shape (N,) 17 18 # Combined data: [x1, y1, x2, y2, conf, cls] 19 # Tracking adds a track_id column before conf/cls. 20 result.boxes.data # shape (N, 6), or (N, 7) when tracked 21 22 # Metadata 23 result.orig_shape # (height, width) of original image 24 result.path # source file path (or None) 25 result.names # {0: "person", 1: "bicycle", ...} 26 27 # Move to CPU / convert to numpy 28 result_cpu = result.cpu() 29 boxes_np = result.boxes.numpy()
类别过滤
将检测结果过滤为特定类别 ID:
1 # Only detect people (class 0) and cars (class 2) 2 result = model("image.jpg", classes=[0, 2])
分块推理
对于远大于模型输入尺寸的图像(如卫星影像、无人机航拍),分块推理会将图像切分为相互重叠的小块,分别在每块上运行检测,再合并结果。
在 v1.2.0 dev 中分块仅支持检测。它不接受分割掩码,且无法与 augment=True 组合使用。
1 result = model( 2 "large_aerial_image.jpg", 3 tiling=True, 4 overlap_ratio=0.2, # 20% overlap between tiles (default) 5 save=True, 6 ) 7 8 # Extra metadata on tiled results 9 result.tiled # True 10 result.num_tiles # number of tiles used 11 result.saved_path # output directory when save=True 12 result.tiles_path # directory containing per-tile crops 13 result.grid_path # grid visualization image
在分块时设置 save=True,LibreYOLO 会保存:
final_image.jpg- 绘制了全部合并检测结果的完整图像grid_visualization.jpg- 显示分块网格叠加的图像tiles/- 各个分块裁剪图metadata.json- 分块参数和检测数量
如果图像本身已小于模型的输入尺寸,则会自动跳过分块。
视频推理
将任意视频文件传给旗舰模型,LibreYOLO 会根据扩展名自动识别格式。支持:.mp4、.avi、.mov、.mkv、.webm、.gif 以及其他常见容器格式。
保存标注后的视频
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("LibreYOLO9c.pt") 4 results = model("clip.mp4", save=True) 5 # Saved under runs/detect/predict*/clip.mp4
流式结果(内存恒定)
对于长视频,传入 stream=True 可获得一个生成器。每次迭代产出一帧的 Results - 不会在内存中缓存完整列表。
1 for result in model("long_clip.mp4", stream=True): 2 print(f"frame {result.frame_idx}: {len(result)} detections")
帧抽样
1 # Process every 2nd frame (halves compute and saved fps) 2 results = model("clip.mp4", vid_stride=2, save=True)
实时预览
1 # Display annotated frames in an OpenCV window while processing 2 results = model("clip.mp4", show=True)
用于自定义流水线的 VideoSource / VideoWriter
当你需要完全掌控解码和编码时 - 自定义帧变换、混入跟踪器输出、写入非默认编解码器 - 可直接使用这些构建模块:
1 from libreyolo import LibreYOLO 2 from libreyolo.utils.video import VideoSource, VideoWriter 3 4 model = LibreYOLO("LibreYOLO9c.pt") 5 6 with VideoSource("clip.mp4", vid_stride=1) as src, \ 7 VideoWriter("out.mp4", fps=src.fps, width=src.width, height=src.height) as out: 8 for frame_bgr, frame_idx in src: 9 result = model(frame_bgr, color_format="bgr") 10 # ... draw, transform, etc. 11 out.write_frame(frame_bgr)
跟踪
LibreYOLO 内置 ByteTrack 多目标跟踪器,可消费任意检测器输出的 Results 并添加持久的跟踪 ID。它在单 GPU 的 YOLO9 检测和 RF-DETR 检测上测试最充分;其他检测系列在 v1.2.0 中属实验性。
安装
1 pip install libreyolo[tracking] # compatibility extra; tracking deps ship in base dev install
视频跟踪辅助函数
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("LibreYOLO9c.pt") 4 5 for result in model.track( 6 "clip.mp4", 7 track_conf=0.25, 8 iou=0.45, 9 save=True, # writes runs/track/<video_stem>.mp4 by default 10 vid_stride=1, 11 ): 12 print(result.frame_idx, result.track_id)
model.track() 是用于视频文件的生成器。它逐帧运行检测,内部使用较低的 ByteTrack 置信度进行恢复,并产出已填充 result.track_id 和 result.boxes.id 的 Results。
基本循环
1 from libreyolo import LibreYOLO, ByteTracker 2 from libreyolo.utils.video import VideoSource 3 4 model = LibreYOLO("LibreYOLO9c.pt") 5 tracker = ByteTracker() 6 7 with VideoSource("clip.mp4") as src: 8 for frame_bgr, frame_idx in src: 9 result = model(frame_bgr, color_format="bgr", conf=0.1) 10 tracked = tracker.update(result) 11 12 for i in range(len(tracked.boxes)): 13 track_id = int(tracked.boxes.id[i]) 14 xyxy = tracked.boxes.xyxy[i].tolist() 15 cls = int(tracked.boxes.cls[i]) 16 print(f"frame {frame_idx} - id {track_id} cls {cls} {xyxy}")
调用 tracker.update() 后,result.boxes.id 保存跟踪 ID,且 result.boxes.is_track 为 True。
TrackConfig 可调参数
1 from libreyolo import ByteTracker, TrackConfig 2 3 cfg = TrackConfig( 4 track_high_thresh=0.25, # first-stage match threshold 5 track_low_thresh=0.1, # second-stage (low-conf recovery) 6 new_track_thresh=0.25, # minimum conf to start a new track 7 match_thresh=0.8, # IoU cost cutoff (stage 1) 8 match_thresh_low=0.5, # IoU cost cutoff (stage 2) 9 match_thresh_unconfirmed=0.7, # IoU cost cutoff for unconfirmed tracks 10 track_buffer=30, # frames to keep lost tracks before removal 11 frame_rate=30, # scales track_buffer 12 fuse_score=True, # multiply IoU by detection score 13 minimum_consecutive_frames=1, # frames to confirm a new track 14 ) 15 tracker = ByteTracker(config=cfg)
片段之间重置
1 tracker.reset() # clears tracked / lost / removed lists and the ID counter
分割
v1.2.0 验证范围
经过深度测试的路径是 YOLO9 和 RF-DETR 的检测、训练与推理,包括 RF-DETR 分割。
其他模型系列、任务和多 GPU 工作流均可用,但属实验性。
RF-DETR 分割是 v1.2.0 中经过深度测试的分割路径。YOLO9 分割和 EdgeCrafter 分割可通过相同的 -seg 后缀使用,但属实验性。
运行分割
1 from libreyolo import LibreYOLO 2 3 # RF-DETR segmentation, heavily tested on single GPU 4 model = LibreYOLO("LibreRFDETRs-seg.pt") 5 result = model("photo.jpg") 6 7 # YOLO9 segmentation is available but experimental in v1.2.0 8 # model = LibreYOLO("LibreYOLO9c-seg.pt") 9 10 # Segmentation returns boxes + masks 11 print(result.boxes.xyxy) # bounding boxes (N, 4) 12 print(result.boxes.cls) # class IDs (N,) 13 print(result.masks.data.shape) # (N, H, W) tensor of binary masks
掩码表示
1 # Raw bitmasks 2 result.masks.data # tensor (N, H, W) - original image resolution 3 4 # Polygon contours (one ndarray of (M, 2) per instance) 5 result.masks.xy # absolute pixel coords 6 result.masks.xyn # normalized to [0, 1] 7 8 # Move / convert like Boxes 9 result.masks.cpu() 10 result.masks.numpy()
保存标注后的输出
save=True 会自动绘制边界框和半透明掩码叠加层。
1 model("photo.jpg", save=True)
训练分割模型
RF-DETR 分割使用 RF-DETR 的 COCO 格式训练流水线,属于经过深度测试的单 GPU 范围。YOLO9 分割和 EdgeCrafter 分割训练可用,但在 v1.2.0 中属实验性。
姿态估计
姿态(人体关键点)估计在 YOLO-NAS (-pose) 和 EdgeCrafter (-pose) 上受支持。每个姿态模型均为单类别("person"),包含 17 个 COCO 关键点。
运行姿态估计
1 from libreyolo import LibreYOLO 2 3 # YOLO-NAS pose 4 model = LibreYOLO("LibreYOLONASs-pose.pt") 5 result = model("people.jpg") 6 7 # EdgeCrafter pose 8 # model = LibreYOLO("LibreECs-pose.pt") 9 10 # Per-person bbox + 17 keypoints 11 print(result.boxes.xyxy) # person boxes (N, 4) 12 print(result.keypoints.xy.shape) # (N, 17, 2) pixel coordinates
关键点 API
1 result.keypoints.xy # (N, K, 2) absolute pixel coords 2 result.keypoints.xyn # (N, K, 2) normalized to [0, 1] 3 result.keypoints.conf # (N, K) per-keypoint confidence (None if model doesn't emit it) 4 result.keypoints.has_visible # (N, K) bool - conf > 0 5 6 result.keypoints.cpu() 7 result.keypoints.numpy()
保存标注后的输出
1 model("people.jpg", save=True) # draws boxes + skeleton
YOLO-NAS 支持姿态训练;EdgeCrafter 姿态目前仅支持推理。YOLO9 和 RF-DETR 尚未提供姿态检查点。
视线估计
视线方向估计由 LibreL2CS 系列提供,它是 L2CS-Net 的移植版,采用 ResNet 主干和两个角度分箱分类头。这是一个两阶段模型:上游的人脸检测器先定位人脸,然后视线头以弧度预测每张人脸的俯仰角(pitch)和偏航角(yaw)。它仅支持推理,且在 v1.2.0 中属实验性。
安装
1 pip install libreyolo[gaze] # optional Google Drive helper for Gaze360 weights
已发布的 L2CS ResNet-50 权重在 Gaze360 上训练,LibreYOLO 不进行镜像。若没有可选辅助工具,请传入本地检查点路径,或按照 LibreL2CS 打印的手动下载说明操作。
两阶段推理
1 from libreyolo import LibreYOLO 2 from libreyolo.models.l2cs.face import resolve_face_detector 3 4 # Gaze head 5 gaze = LibreYOLO("LibreL2CSr50.pt") 6 7 # Wire any LibreYOLO detector trained on faces 8 face = LibreYOLO("path/to/face-detector.pt") 9 gaze.face_detector = resolve_face_detector(face) 10 11 result = gaze("portrait.jpg") 12 print(result.boxes.xyxy) # face boxes 13 print(result.gaze.data) # (N, 2) tensor - pitch, yaw in radians
解码角度
1 import math 2 3 for i in range(len(result.gaze)): 4 pitch_rad, yaw_rad = result.gaze.data[i].tolist() 5 pitch_deg = pitch_rad * 180.0 / math.pi 6 yaw_deg = yaw_rad * 180.0 / math.pi 7 print(f"face {i}: pitch={pitch_deg:.1f} deg, yaw={yaw_deg:.1f} deg")
通过命令行:libreyolo predict model=LibreL2CSr50.pt source=portrait.jpg --face-detector path/to/face.pt。
训练
v1.2.0 验证范围
经过深度测试的路径是 YOLO9 和 RF-DETR 的检测、训练与推理,包括 RF-DETR 分割。
其他模型系列、任务和多 GPU 工作流均可用,但属实验性。
经过深度测试的训练路径是单 GPU 的 YOLO9 检测、RF-DETR 检测和 RF-DETR 分割。其他模型系列的训练器、YOLO9 分割训练和多 GPU 工作流可用,但在 v1.2.0 中属实验性。
YOLO9 - CNN 旗舰训练
1 from libreyolo import LibreYOLO 2 3 # Fine-tune from a pretrained checkpoint (recommended) 4 model = LibreYOLO("LibreYOLO9c.pt") 5 6 results = model.train( 7 data="coco128.yaml", # path to data.yaml (required) 8 9 # Schedule 10 epochs=300, # default: 300 11 batch=16, 12 imgsz=640, 13 14 # Optimizer 15 lr0=0.01, # initial learning rate 16 optimizer="SGD", # "SGD", "Adam", "AdamW" 17 18 # System 19 device="0", # "" | "cpu" | "cuda" | "0" | "0,1" 20 workers=8, 21 seed=0, 22 23 # Output 24 project="runs/train", 25 name="yolo9_exp", 26 exist_ok=False, 27 28 # Training features 29 amp=True, # automatic mixed precision 30 patience=50, # early stopping patience 31 resume=False, # resume from loaded checkpoint 32 ) 33 34 print(f"Best mAP50-95: {results['best_mAP50_95']:.3f}") 35 print(f"Best checkpoint: {results['best_checkpoint']}")
训练完成后,模型实例会自动以最佳权重重新加载,因此你可以立即调用 model(...)。YOLO9 分割训练可通过 LibreYOLO("LibreYOLO9c-seg.pt") 使用,但在 v1.2.0 中属实验性。
RF-DETR - Transformer 旗舰训练
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("LibreRFDETRs.pt") 4 5 results = model.train( 6 data="path/to/data.yaml", 7 epochs=100, 8 batch_size=4, # NOTE: RF-DETR uses batch_size, not batch 9 lr=1e-4, 10 output_dir="runs/train/rfdetr_exp", 11 )
RF-DETR 有自己的训练签名(batch_size、lr、output_dir),但它使用 LibreYOLO 的数据集配置加载器。为检测或分割传入 data.yaml;该配置可引用 COCO/Roboflow 风格的标注布局。
训练结果字典
1 { 2 "final_loss": 2.31, 3 "best_mAP50": 0.682, 4 "best_mAP50_95": 0.451, 5 "best_epoch": 87, 6 "save_dir": "runs/train/yolo9_exp", 7 "best_checkpoint": "runs/train/yolo9_exp/weights/best.pt", 8 "last_checkpoint": "runs/train/yolo9_exp/weights/last.pt", 9 }
恢复训练
1 # Load the checkpoint with the factory, then resume 2 model = LibreYOLO("runs/train/yolo9_exp/weights/last.pt") 3 results = model.train(data="coco128.yaml", resume=True)
自定义数据集 YAML 格式
1 path: /path/to/dataset 2 train: images/train 3 val: images/val 4 test: images/test # optional 5 6 nc: 3 7 names: ["cat", "dog", "bird"]
其他训练路径
其他系列也有训练器钩子,但它们不是 v1.2.0 中的推荐路径。新工作请坚持使用 YOLO9 检测或 RF-DETR 检测/分割;仅在兼容、复现基准或定向研究时使用实验性训练器。DAMO-YOLO、PicoDet、RTMDet 和 EC 的训练需要显式的 allow_experimental=True 确认。
通过 YAML 配置训练
每个 model.train(...) 都接受 cfg="train.yaml",以从文件加载全部参数。显式关键字参数仍优先于 yaml 中的值,因此你可以用 yaml 作为基线,并在每次运行时覆盖单个字段。
1 model = LibreYOLO("LibreYOLO9c.pt") 2 results = model.train(cfg="configs/yolo9_finetune.yaml") 3 # Override individual fields: 4 # results = model.train(cfg="configs/yolo9_finetune.yaml", epochs=50)
梯度累积
传入 nbs(名义批量大小)以启用梯度累积。训练器每 nbs / batch 次前向传播执行一次优化器步进,使你能在较小的硬件上以配方的参考批量大小进行训练。
1 # Effective batch 64 on a single GPU that only fits batch=8 2 model.train(data="coco128.yaml", batch=8, nbs=64)
分布式训练(DDP,实验性)
YOLO9 和 RF-DETR 通过 PyTorch DistributedDataParallel 支持多 GPU 训练,但多 GPU 不在 v1.2.0 的深度测试范围内。使用 torchrun 启动训练脚本:
1 # 4-GPU node 2 torchrun --nproc_per_node=4 train_yolo9.py 3 4 # Multi-node - see PyTorch's torchrun docs for --nnodes / --rdzv_endpoint
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("LibreYOLO9c.pt") 4 # Pass device="" (auto-detect) and let torchrun set the rank 5 model.train(data="coco128.yaml", epochs=300, batch=16)
验证
在验证集上运行 COCO 标准评估。经过深度测试的验证路径是单 GPU 的 YOLO9 检测、RF-DETR 检测和 RF-DETR 分割。
1 results = model.val( 2 data="coco128.yaml", # dataset config 3 batch=16, 4 imgsz=640, 5 conf=0.001, # low conf for mAP calculation 6 iou=0.6, # NMS IoU threshold 7 split="val", # "val", "test", or "train" 8 save_json=False, # save predictions as COCO JSON 9 verbose=True, # print per-class metrics 10 ) 11 12 print(f"mAP50: {results['metrics/mAP50']:.3f}") 13 print(f"mAP50-95: {results['metrics/mAP50-95']:.3f}")
验证结果字典
默认情况下,LibreYOLO 使用 COCO 评估,返回精确率、召回率、AP/AR 指标以及每张图像的耗时:
1 { 2 "metrics/mAP50-95": 0.489, # COCO primary metric (AP@[.5:.95]) 3 "metrics/mAP50": 0.721, # AP@0.5 (PASCAL VOC style) 4 "metrics/mAP75": 0.534, # AP@0.75 (strict) 5 "metrics/precision": 0.68, 6 "metrics/recall": 0.61, 7 "metrics/precision(B)": 0.68, # bbox aliases 8 "metrics/recall(B)": 0.61, 9 "metrics/mAP50(B)": 0.721, 10 "metrics/mAP50-95(B)": 0.489, 11 "metrics/mAP_small": 0.291, 12 "metrics/mAP_medium": 0.532, 13 "metrics/mAP_large": 0.648, 14 "metrics/AR1": 0.362, # Average Recall (max 1 det) 15 "metrics/AR10": 0.571, 16 "metrics/AR100": 0.601, 17 "metrics/AR_small": 0.387, 18 "metrics/AR_medium": 0.641, 19 "metrics/AR_large": 0.739, 20 "speed/preprocess_ms": 1.2, 21 "speed/inference_ms": 6.8, 22 "speed/postprocess_ms": 0.9, 23 "speed/total_ms": 8.9, 24 "speed/total_s": 12.3, 25 "speed/images_seen": 1382, 26 }
分割验证返回带 (M) 后缀的掩码指标,以及带 (B) 后缀的边界框指标。姿态验证通过 PoseValidator 返回 COCO 关键点指标。
导出
将 PyTorch 模型导出为 ONNX、TorchScript、TensorRT、OpenVINO、NCNN 或 CoreML 以便部署。经过深度测试的导出与运行时后端路径是单 GPU 的 YOLO9 检测、RF-DETR 检测和 RF-DETR 分割。其他系列和任务属实验性。
快速导出
1 # ONNX (default) 2 model.export() 3 4 # TorchScript 5 model.export(format="torchscript") 6 7 # TensorRT (requires NVIDIA GPU + TensorRT) 8 model.export(format="tensorrt") 9 10 # OpenVINO (optimized for Intel hardware) 11 model.export(format="openvino") 12 13 # NCNN (via PNNX) 14 model.export(format="ncnn") 15 16 # CoreML (.mlpackage, macOS runtime) 17 model.export(format="coreml")
全部导出参数
1 path = model.export( 2 format="onnx", # "onnx", "torchscript", "tensorrt", "openvino", "ncnn", or "coreml" 3 output_path="model.onnx", # output file (auto-generated if None) 4 imgsz=640, # input resolution (default: model's native) 5 opset=None, # ONNX opset (auto: 13, or 17 for wrappers that need it) 6 simplify=True, # run onnxsim graph simplification 7 dynamic=True, # enable dynamic batch axis 8 half=False, # export in FP16 9 batch=1, # batch size for static graph 10 device=None, # device to trace on (default: model's current device) 11 int8=False, # INT8 quantization (TensorRT / OpenVINO only) 12 data=None, # calibration dataset for INT8 13 fraction=1.0, # fraction of calibration data to use 14 allow_download_scripts=False, # allow data.yaml download hooks during calibration 15 workspace=4.0, # TensorRT workspace size (GB) 16 min_batch=1, # TensorRT dynamic profile minimum batch 17 opt_batch=1, # TensorRT dynamic profile optimal batch 18 max_batch=8, # TensorRT dynamic profile maximum batch 19 hardware_compatibility="none", # TensorRT compatibility mode 20 gpu_device=0, # GPU device index for TensorRT 21 trt_config=None, # optional TensorRT YAML config path 22 compute_units="all", # CoreML routing: all, cpu_only, cpu_and_gpu, cpu_and_ne 23 nms=False, # CoreML embedded NMS where supported 24 iou=0.45, # CoreML embedded NMS IoU threshold 25 conf=0.25, # CoreML embedded NMS confidence threshold 26 verbose=False, # verbose logging 27 )
OpenVINO INT8 导出还需要 nncf。NCNN 导出会写出一个目录,包含 model.ncnn.param、model.ncnn.bin 和 metadata.yaml。CoreML 导出会写出 .mlpackage 包,需要 coremltools,且不支持 INT8。
ONNX 元数据
导出的 ONNX 文件包含内嵌元数据:
| 键 | 示例值 |
|---|---|
libreyolo_version | "1.2.0" |
model_family | "yolox" |
model_size | "s" |
nb_classes | "80" |
names | '{"0": "person", "1": "bicycle", ...}' |
imgsz | "640" |
dynamic | "True" |
half | "False" |
使用 LibreYOLO("model.onnx") 加载导出文件时,会自动读回此元数据。
TorchScript 推理
通过相同的运行时后端预测 API 运行导出的 .torchscript 模型。
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("model.torchscript") 4 5 result = model("image.jpg", conf=0.25, iou=0.45, save=True) 6 print(result.boxes.xyxy)
ONNX 推理
使用 ONNX Runtime 而非 PyTorch 运行推理。适用于没有 PyTorch 的部署环境。
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("model.onnx") 4 5 result = model("image.jpg", conf=0.25, iou=0.45, save=True) 6 print(result.boxes.xyxy)
自动元数据
如果 ONNX 文件由 LibreYOLO 导出,类别名称和类别数会自动从内嵌元数据中读取:
1 # Export with metadata 2 model.export(format="onnx", output_path="model.onnx") 3 4 # Load - names and nb_classes auto-populated 5 onnx_model = LibreYOLO("model.onnx") 6 print(onnx_model.names) # {0: "person", 1: "bicycle", ...} 7 print(onnx_model.nb_classes) # 80
对于没有元数据的 ONNX 文件(如由其他工具导出),请手动指定 nb_classes:
1 model = LibreYOLO("external_model.onnx", nb_classes=20)
设备选择
1 # Auto-detect (CUDA if available, else CPU) 2 model = LibreYOLO("model.onnx", device="auto") 3 4 # Force CPU 5 model = LibreYOLO("model.onnx", device="cpu") 6 7 # Force CUDA 8 model = LibreYOLO("model.onnx", device="cuda")
预测参数
通过 LibreYOLO() 加载的运行时产物支持共享的运行时预测 API:
1 result = model( 2 "image.jpg", 3 conf=0.25, 4 iou=0.45, 5 imgsz=640, 6 classes=[0, 2], 7 max_det=300, 8 save=True, 9 output_path="output/annotated.jpg", # final file path when save=True 10 color_format="auto", 11 )
运行时后端不暴露仅限 PyTorch 的选项,如 tiling、overlap_ratio 或 output_file_format。
运行时后端的保存方式与 PyTorch 封装略有不同:如果设置 output_path,请传入最终文件路径,而非目录。如果省略它,当前后端默认保存在 runs/detections/ 下。
TensorRT 推理
使用 TensorRT 在 NVIDIA GPU 上以最大吞吐量运行推理。需要 CUDA 以及 TensorRT 的 Python 绑定。
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("model.engine") 4 5 result = model("image.jpg", conf=0.25, iou=0.45, save=True) 6 print(result.boxes.xyxy)
通过 LibreYOLO() 加载的 TensorRT 产物支持与 ONNX 和 OpenVINO 相同的核心运行时预测 API,包括 save=True 时同样仅接受文件路径的 output_path 行为。
OpenVINO 推理
使用 OpenVINO 运行推理,针对 Intel CPU、GPU 和 VPU 进行了优化。
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("model_openvino/") 4 5 result = model("image.jpg", conf=0.25, iou=0.45, save=True) 6 print(result.boxes.xyxy)
通过 LibreYOLO() 加载的 OpenVINO 目录会在存在时读取 metadata.yaml,并支持相同的核心运行时预测 API。
NCNN 推理
使用 NCNN 运行推理,以在 CPU 或支持 Vulkan 的 GPU 目标上实现轻量化部署。
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("model_ncnn/") 4 5 result = model("image.jpg", conf=0.25, iou=0.45, save=True) 6 print(result.boxes.xyxy)
NCNN 导出目录包含 model.ncnn.param、model.ncnn.bin,通常还有 metadata.yaml。
CoreML 推理
在 macOS 上通过 CoreML 运行导出的 .mlpackage。CoreML 使用 compute_units 而非 PyTorch 设备字符串来路由执行。
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("model.mlpackage", compute_units="all") 4 5 result = model("image.jpg", conf=0.25, iou=0.45, save=True) 6 print(result.boxes.xyxy)
支持的 compute_units 取值为 all、cpu_only、cpu_and_gpu 和 cpu_and_ne。
命令行工具
安装 LibreYOLO 会在你的 PATH 中注册一个 libreyolo 命令(入口点在 pyproject.toml 中)。该命令行镜像 Python API,并遵循 Ultralytics 风格的 key=value 语法。
子命令
| 命令 | 用途 |
|---|---|
predict | 对图像、目录或视频运行推理 |
train | 在数据集上训练模型 |
val | 在数据集上评估模型 |
export | 导出为 ONNX / TorchScript / TensorRT / OpenVINO / NCNN / CoreML |
checks | 打印 Python、torch、CUDA、GPU 及可选包信息 |
models | 列出已注册的模型系列和 CLI 快捷名称 |
formats | 列出支持的导出格式 |
cfg | 打印默认的训练配置 YAML |
info | 加载模型并打印解析出的系列、尺寸、任务、设备和类别 |
metadata | 检视 .pt 文件中的原始检查点元数据 |
version | 打印 LibreYOLO + Python + torch 版本 |
模型名称快捷方式
命令行接受短名称(yolo9-c),它们会解析为权重文件名(LibreYOLO9c.pt) - 可通过 libreyolo models 查看。你也可以传入任意明确的检查点路径。
常用选项
| 命令 | 重要选项 |
|---|---|
predict | conf, iou, imgsz, classes, max_det, half, batch, tiling, overlap_ratio, output_file_format, project, name, exist_ok, face_detector |
train | epochs, batch, imgsz, lr0, optimizer, scheduler, workers, seed, resume, amp, allow_download_scripts, dry_run |
val | split, batch, imgsz, conf, iou, max_det, half, data_dir, use_coco_eval, project, name, exist_ok, save_json, allow_download_scripts |
export | format, imgsz, batch, half, int8, dynamic, simplify, opset, data, fraction, device, allow_download_scripts, verbose |
预测
1 # Flagship: YOLO9 2 libreyolo predict model=yolo9-c source=image.jpg conf=0.25 save=true 3 4 # Flagship: RF-DETR 5 libreyolo predict model=rfdetr-s source=image.jpg save=true 6 7 # Video - saved under runs/detect/predict*/ 8 libreyolo predict model=yolo9-c source=clip.mp4 save=true 9 10 # Tiled inference for very large images 11 libreyolo predict model=yolo9-c source=aerial.jpg tiling=true save=true 12 13 # Gaze (requires a face detector) 14 libreyolo predict model=LibreL2CSr50.pt source=portrait.jpg \ 15 --face-detector path/to/face.pt save=true
训练
1 libreyolo train model=yolo9-c data=coco128.yaml epochs=300 batch=16 device=0 2 3 # Dry-run prints the resolved config without launching training 4 libreyolo train model=yolo9-c data=coco128.yaml --dry-run
验证
1 libreyolo val model=runs/train/exp/weights/best.pt data=coco128.yaml split=val
导出
1 libreyolo export model=runs/train/exp/weights/best.pt format=onnx dynamic=true 2 libreyolo export model=best.pt format=tensorrt half=true 3 libreyolo export model=best.pt format=openvino int8=true data=coco128.yaml 4 libreyolo export model=best.pt format=coreml
机器可读输出
每个命令都接受 --json(结构化的标准输出,便于管道传入脚本或智能体)和 --quiet(抑制 stderr 进度行)。核心的 predict、train、val 和 export 命令还接受 --help-json,可将其参数 schema 以 JSON 形式导出。
1 libreyolo predict model=yolo9-c source=img.jpg --json | jq . 2 3 libreyolo train --help-json > train_schema.json
API 参考
LibreYOLO(工厂函数)
1 LibreYOLO( 2 model_path: str, 3 *, 4 device: str = "auto", 5 task: str | None = None, # override only when a custom artifact is ambiguous 6 nb_classes: int | None = None, # mainly for external exported artifacts 7 compute_units: str = "all", # CoreML only: all, cpu_only, cpu_and_gpu, cpu_and_ne 8 ) -> model wrapper or runtime backend
优先使用官方检查点文件名和导出产物路径,再让工厂函数解析细节。它可处理 PyTorch 检查点、.onnx、.torchscript、.engine、.tensorrt、.mlpackage、包含 model.xml 的 OpenVINO 目录,以及包含 model.ncnn.param 和 model.ncnn.bin 的 NCNN 目录。task 参数用于含糊的自定义产物;否则解析来自检查点元数据、文件名后缀和系列默认值。
预测(PyTorch 模型封装)
1 model( 2 source, # image input (see supported formats) 3 *, 4 conf: float = 0.25, 5 iou: float = 0.45, 6 imgsz: int = None, 7 device: str = "auto", 8 classes: list[int] = None, 9 max_det: int = 300, 10 augment: bool = False, 11 save: bool = False, 12 batch: int = 1, 13 stream: bool = False, 14 vid_stride: int = 1, 15 show: bool = False, 16 output_path: str = None, 17 color_format: str = "auto", 18 tiling: bool = False, 19 overlap_ratio: float = 0.2, 20 output_file_format: str = None, 21 ) -> Results | list[Results] | Generator[Results, None, None]
预测(运行时后端)
1 backend( 2 source, 3 *, 4 conf: float = 0.25, 5 iou: float = 0.45, 6 imgsz: int = None, 7 classes: list[int] = None, 8 max_det: int = 300, 9 save: bool = False, 10 batch: int = 1, 11 output_path: str = None, # final file path when save=True 12 color_format: str = "auto", 13 ) -> Results | list[Results]
如果运行时后端省略 output_path,当前默认保存位置为 runs/detections/。
Results
1 result = Results( 2 boxes: Boxes | None, 3 orig_shape: tuple[int, int], # (height, width) 4 path: str | None, 5 names: dict[int, str], 6 masks: Masks | None = None, 7 keypoints: Keypoints | None = None, 8 probs: Probs | None = None, 9 obb: OBB | None = None, 10 gaze: Gaze | None = None, 11 speed: dict[str, float] | None = None, 12 track_id = None, 13 frame_idx: int | None = None, 14 ) 15 16 len(result) # number of detections 17 result.cpu() # copy with tensors on CPU 18 result.cuda() # copy with tensors on CUDA 19 result.numpy() # copy with numpy arrays 20 result.summary() # list[dict] with boxes, masks, gaze, and track_id when present 21 result.to_json() # JSON string from summary()
Boxes
1 boxes = Boxes(boxes, conf, cls) 2 3 boxes.xyxy # (N, 4) tensor - x1, y1, x2, y2 4 boxes.xywh # (N, 4) tensor - cx, cy, w, h 5 boxes.conf # (N,) tensor - confidence scores 6 boxes.cls # (N,) tensor - class IDs 7 boxes.id # (N,) track IDs when tracking, else None 8 boxes.is_track # True when track IDs are attached 9 boxes.data # (N, 6) [xyxy, conf, cls], or (N, 7) with track IDs 10 11 len(boxes) # number of boxes 12 boxes.cpu() # copy on CPU 13 boxes.numpy() # copy as numpy arrays
任务负载
1 result.masks.data # segmentation masks, (N, H, W) 2 result.masks.xy # list of mask contours in pixel coordinates 3 result.masks.xyn # normalized mask contours 4 5 result.keypoints.xy # pose keypoint coordinates 6 result.keypoints.xyn # normalized keypoint coordinates 7 result.keypoints.conf # keypoint confidence when present 8 9 result.gaze.data # (N, 2): pitch, yaw in radians 10 result.gaze.pitch_deg # pitch in degrees 11 result.gaze.yaw_deg # yaw in degrees 12 result.gaze.direction_3d # approximate 3D direction vectors
model.export()
1 model.export( 2 format: str = "onnx", # "onnx", "torchscript", "tensorrt", "openvino", "ncnn", or "coreml" 3 *, 4 output_path: str | None = None, 5 imgsz: int | None = None, 6 opset: int | None = None, # auto: 13, or 17 for wrappers that need it 7 simplify: bool = True, 8 dynamic: bool = True, 9 half: bool = False, 10 batch: int = 1, 11 device: str | None = None, 12 int8: bool = False, 13 data: str | None = None, # calibration data for INT8 14 fraction: float = 1.0, # fraction of calibration data 15 allow_download_scripts: bool = False, 16 workspace: float = 4.0, # TensorRT workspace (GB) 17 min_batch: int = 1, # TensorRT dynamic profile minimum batch 18 opt_batch: int = 1, # TensorRT dynamic profile optimal batch 19 max_batch: int = 8, # TensorRT dynamic profile maximum batch 20 hardware_compatibility: str = "none", 21 gpu_device: int = 0, 22 trt_config = None, # optional TensorRT YAML config path 23 compute_units: str = "all", # CoreML only 24 nms: bool = False, # CoreML embedded NMS where supported 25 iou: float = 0.45, # CoreML embedded NMS IoU threshold 26 conf: float = 0.25, # CoreML embedded NMS confidence threshold 27 verbose: bool = False, 28 ) -> str # path to exported file or directory
model.val()
1 model.val( 2 data: str = None, # path to data.yaml 3 batch: int = 16, 4 imgsz: int = None, 5 conf: float = 0.001, 6 iou: float = 0.6, 7 workers: int = 4, 8 allow_download_scripts: bool = False, 9 device: str = None, 10 split: str = "val", # "val", "test", or "train" 11 augment: bool = False, 12 save_json: bool = False, 13 verbose: bool = True, 14 ) -> dict
返回(COCO 评估,默认):
1 { 2 "metrics/mAP50-95": float, # COCO primary metric 3 "metrics/mAP50": float, 4 "metrics/mAP75": float, 5 "metrics/mAP_small": float, 6 "metrics/mAP_medium": float, 7 "metrics/mAP_large": float, 8 "metrics/AR1": float, 9 "metrics/AR10": float, 10 "metrics/AR100": float, 11 "metrics/AR_small": float, 12 "metrics/AR_medium": float, 13 "metrics/AR_large": float, 14 }
model.train() (YOLO9)
1 model.train( 2 data: str, # path to data.yaml (required) 3 *, 4 epochs: int = 300, 5 batch: int = 16, 6 imgsz: int = 640, 7 lr0: float = 0.01, 8 optimizer: str = "SGD", 9 device: str = "", 10 workers: int = 8, 11 seed: int = 0, 12 project: str = "runs/train", 13 name: str = "yolo9_exp", 14 exist_ok: bool = False, 15 resume: bool = False, 16 amp: bool = True, 17 patience: int = 50, 18 allow_download_scripts: bool = False, 19 callbacks = None, 20 ) -> dict
返回标准的 LibreYOLO 训练字典,包含 final_loss、best_mAP50、best_mAP50_95、best_epoch、save_dir、best_checkpoint 和 last_checkpoint。
model.train() (RF-DETR)
1 model.train( 2 data: str, # path to data.yaml 3 epochs: int = 100, 4 batch_size: int = 4, 5 lr: float = 1e-4, 6 output_dir: str = "runs/train", 7 resume: str = None, 8 **kwargs, # additional RF-DETR training args 9 ) -> dict
YOLO-NAS、D-FINE、DEIM、DEIMv2、EC、PicoDet、DAMO-YOLO、RT-DETRv2/v4 和 RTMDet 还提供额外的实验性训练器。它们遵循相同的 model.train(data="...yaml", ...) 形式,但其默认值和实验性门控因系列而异。
运行时产物加载
通过 LibreYOLO() 加载导出产物,方式与加载 PyTorch 检查点相同。工厂函数会根据路径选择 ONNX Runtime、TorchScript、TensorRT、OpenVINO、NCNN 或 CoreML:
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("model.onnx") 4 model = LibreYOLO("model.torchscript") 5 model = LibreYOLO("model.engine") 6 model = LibreYOLO("model_openvino/") 7 model = LibreYOLO("model_ncnn/") 8 model = LibreYOLO("model.mlpackage", compute_units="all")
高级集成可以访问更底层的运行时模块,但普通应用代码应坚持使用工厂函数路径。
ValidationConfig
1 from libreyolo import ValidationConfig 2 3 config = ValidationConfig( 4 data="coco128.yaml", 5 data_dir=None, # override dataset root directory 6 split="val", # "val", "test", or "train" 7 batch_size=16, 8 imgsz=640, 9 conf_thres=0.001, 10 iou_thres=0.6, 11 max_det=300, 12 iou_thresholds=( # mAP IoU sweep 13 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, 0.95, 14 ), 15 device="auto", 16 save_dir=None, 17 save_json=False, 18 verbose=True, 19 num_workers=4, 20 half=False, 21 augment=False, # test-time augmentation (TTA) 22 allow_download_scripts=False, 23 # Pose-only fields (PoseValidator) 24 keypoints_json=None, 25 images_dir=None, 26 oks_sigmas=None, 27 ) 28 29 # Load/save YAML 30 config = ValidationConfig.from_yaml("config.yaml") 31 config.to_yaml("config.yaml")
架构指南
本节面向希望了解代码库内部实现的贡献者。
基类设计
PyTorch 模型系列继承自 libreyolo/models/base/model.py 中的 BaseModel。子类需实现以下抽象方法:
| 方法 | 用途 |
|---|---|
_init_model() | 构建并返回 nn.Module |
_get_available_layers() | 返回层名称到模块的映射 |
_get_preprocess_numpy() | 返回用于导出/校准的 NumPy 预处理器 |
_preprocess() | 图像到张量的转换 |
_forward() | 模型前向传播 |
_postprocess() | 原始输出到检测字典 |
BaseModel 提供共享的封装行为:预测、导出、验证、尺寸/名称元数据和训练辅助函数。实际的单图、批量和分块推理流程位于 libreyolo/models/base/inference.py,而部署运行时位于 libreyolo/backends/ 下。
包结构
1 libreyolo/ 2 __init__.py # Public API exports + deprecated-alias resolver 3 tasks.py # Task types, suffix conventions, resolution rules 4 assets/parkour.jpg # SAMPLE_IMAGE 5 models/ 6 __init__.py # LibreYOLO() factory + model registry bootstrap 7 base/ 8 model.py # BaseModel - shared wrapper behaviour 9 inference.py # Shared prediction pipeline (image/dir/video/tiled) 10 yolox/ # LibreYOLOX (detect) 11 yolo9/ # LibreYOLO9 (detect, segment) 12 yolo9_e2e/ # LibreYOLO9E2E (detect) 13 yolonas/ # LibreYOLONAS (detect, pose) 14 dfine/ # LibreDFINE (detect) 15 deim/ # LibreDEIM (detect) 16 deimv2/ # LibreDEIMv2 (detect) 17 rtdetr/ # LibreRTDETR (detect) 18 rtdetrv2/ # LibreRTDETRv2 (detect) 19 rtdetrv4/ # LibreRTDETRv4 (detect) 20 rfdetr/ # LibreRFDETR (detect, segment) - lazy-loaded 21 ec/ # LibreEC / EdgeCrafter (detect, pose, segment) 22 picodet/ # LibrePICODET (detect) 23 damoyolo/ # LibreDAMOYOLO (detect) 24 rtmdet/ # LibreRTMDet (detect) 25 l2cs/ # LibreL2CS (gaze, inference-only) 26 backends/ 27 base.py 28 onnx.py # ONNX Runtime loader 29 torchscript.py # TorchScript loader 30 tensorrt.py # TensorRT loader 31 openvino.py # OpenVINO loader 32 ncnn.py # NCNN loader 33 coreml.py # CoreML loader 34 export/ 35 exporter.py # BaseExporter and format registry 36 onnx.py / torchscript.py / tensorrt.py / openvino.py / ncnn.py / coreml.py 37 config.py / calibration.py 38 training/ 39 trainer.py # Shared trainer scaffolding 40 config.py # TrainConfig dataclass (single source of truth) 41 augment.py / callbacks.py / distributed.py / ema.py / scheduler.py 42 artifacts.py / train_config.yaml 43 # Per-family trainers live in models/<family>/trainer.py 44 validation/ 45 config.py # ValidationConfig 46 base.py / preprocessors.py 47 detection_validator.py # DetectionValidator, SegmentationValidator 48 pose_validator.py # PoseValidator 49 coco_evaluator.py # COCOEvaluator 50 tracking/ 51 tracker.py # ByteTracker 52 config.py # TrackConfig 53 kalman_filter.py / matching.py / strack.py 54 cli/ 55 __init__.py # libreyolo entrypoint (Typer app) 56 commands/ # predict / train / val / export / special 57 aliases.py / config.py / parsing.py / output.py / errors.py 58 utils/ 59 results.py # Results, Boxes, Masks, Keypoints, Probs, OBB, Gaze 60 image_loader.py # Unified image loading 61 video.py # VideoSource, VideoWriter, video inference loop 62 general.py # Path helpers, NMS, tiling utilities 63 download.py / drawing.py / logging.py / predict_args.py 64 serialization.py / box_ops.py 65 data/ 66 dataset.py / pose_dataset.py / utils.py / yolo_coco_api.py 67 config/ 68 datasets/ # Built-in dataset YAML configs (coco8, coco128, coco5000, coco, etc.) 69 export/ # TensorRT default YAML
添加新的模型系列
- 1创建
libreyolo/models/newmodel/model.py,其中包含一个继承BaseModel的类 - 2按需设置
FAMILY、FILENAME_PREFIX、INPUT_SIZES、SUPPORTED_TASKS和DEFAULT_TASK - 3实现注册表钩子,如
can_load()、detect_size()、detect_nb_classes()和detect_size_from_filename() - 4实现该系列所需的模型初始化、预处理、前向、后处理、训练和验证钩子
- 5在
libreyolo/models/newmodel/下创建配套的网络和工具 - 6将该导入添加到
libreyolo/models/__init__.py;导入运行时会进行子类注册 - 7从
libreyolo/__init__.py导出该类 - 8(可选)如果验证预处理与标准路径不同,覆盖
val_preprocessor_class
导出架构
用户代码应通过 model.export(...) 导出。在内部,libreyolo/export/exporter.py 中的 BaseExporter 持有格式注册表,具体的导出器通过子类注册来注册自身。
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("LibreYOLO9c.pt") 4 model.export(format="onnx")
要添加新的导出格式,请实现一个具有唯一 format_name 的新 BaseExporter 子类,并从 libreyolo/export/exporter.py 导入它,以填充注册表。
数据集格式
训练和验证使用通过 data.yaml 加载的数据集配置。检测、分割、姿态和 RF-DETR 训练都经由该加载器;标签文件内容因任务而异。
data.yaml 结构
1 path: /absolute/path/to/dataset # dataset root 2 train: images/train # directory path, relative to path 3 val: images/val # directory path, relative to path 4 test: images/test # optional 5 6 nc: 80 # number of classes 7 names: [ # class names 8 "person", "bicycle", "car", "motorcycle", "airplane", 9 "bus", "train", "truck", "boat", "traffic light", 10 # ... 11 ]
配置解析与下载
数据集配置按以下顺序解析:明确路径、当前工作目录,然后是 libreyolo/config/datasets/ 下的内置配置。数据集根目录默认在 ~/datasets 下,可通过 LIBREYOLO_DATASETS_DIR 覆盖。
train、val 和 test 可以是目录、.txt 文件或路径列表。YAML 下载钩子受保护;仅对可信配置传入 allow_download_scripts=True。
文件列表变体
同样的 YAML 格式也可将 train、val 或 test 指向每行一个图像路径的 .txt 文件:
1 path: /absolute/path/to/coco 2 train: train2017.txt 3 val: val2017.txt 4 test: test-dev2017.txt 5 6 nc: 80 7 names: ["person", "bicycle", "car", "..."]
目录布局
1 dataset/ 2 images/ 3 train/ 4 img001.jpg 5 img002.jpg 6 val/ 7 img003.jpg 8 labels/ 9 train/ 10 img001.txt 11 img002.txt 12 val/ 13 img003.txt
检测标签格式
每张图像对应一个文本文件。每行表示一个目标:
1 <class_id> <center_x> <center_y> <width> <height>
所有坐标都相对于图像尺寸归一化到 [0, 1]。
示例(img001.txt):
1 0 0.5 0.4 0.3 0.6 2 2 0.1 0.2 0.05 0.1
分割标签格式
分割使用 YOLO 多边形行。数据集加载器从多边形顶点推导边界框,并在启用分割加载时保留多边形环:
1 <class_id> <x1> <y1> <x2> <y2> ... <xn> <yn>
姿态标签格式
姿态标签在边界框之后追加关键点。在 data.yaml 中添加 kpt_shape 和 flip_idx,使加载器知道关键点数量和水平翻转排列。
1 kpt_shape: [17, 3] 2 flip_idx: [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15]
1 <class_id> <cx> <cy> <w> <h> <kx1> <ky1> <v1> ... <kxK> <kyK> <vK>
内置数据集
LibreYOLO 在 libreyolo/config/datasets/ 下提供内置数据集配置,并可在首次使用时自动下载支持的数据集:
1 # These download automatically on first use 2 results = model.val(data="coco8.yaml") 3 results = model.train(data="coco128.yaml", epochs=10)