These are the frozen docs for v1.2.0. They are kept for anyone pinned to that release and are no longer updated. Read the docs for the current release.
这是早期版本 v1.2.0 的文档。最新文档为 v1.4.0,可在版本菜单中找到。
简介
v1.2.0 验证范围
经过深度测试的路径是 YOLO9 和 RF-DETR 的检测、训练与推理,包括 RF-DETR 分割。
其他模型系列、任务和多 GPU 工作流均可用,但属实验性。
LibreYOLO 是一个采用 MIT 许可证的目标检测工具包。v1.2.0 提供了广泛的模型目录,但经过验证的支持范围有意保持精简:
- YOLO9 检测 - CNN 路径。
- RF-DETR 检测 - Transformer 路径。
- RF-DETR 分割 - 经过深度测试的分割路径。
我们推荐将这些路径作为新项目的默认选择,因为它们在检测、训练和推理方面经过了最充分的测试。其他受支持的系列和任务通过同一个统一的 LibreYOLO() 工厂函数工作,但在 v1.2.0 中仍属实验性。如有特定理由再使用它们。
1 from libreyolo import LibreYOLO, SAMPLE_IMAGE 2 3 # Default: YOLO9 detection 4 model = LibreYOLO("LibreYOLO9c.pt") 5 result = model(SAMPLE_IMAGE, conf=0.25, save=True) 6 7 print(f"Detected {len(result)} objects") 8 print(result.boxes.xyxy) 9 print(result.saved_path)
核心特性
- 针对 YOLO9 检测、RF-DETR 检测和 RF-DETR 分割的深度测试与推荐默认配置
- 统一的
LibreYOLO()工厂函数,用于加载检查点、导出产物和运行时 - 通过一致的 API 支持检测、分割、姿态和视线任务
- 支持图像、目录和视频推理(大尺寸帧可选用分块推理)
- 通过 ByteTrack 内置多目标跟踪
- 支持 ONNX、TorchScript、TensorRT、OpenVINO、NCNN 和 CoreML 导出(含内嵌元数据),并提供匹配的运行时后端
- 兼容 COCO 的验证与 mAP 指标,并提供分割和姿态验证器
- Ultralytics 风格的
libreyolo命令行工具,用于 predict / train / val / export - 接受任意图像格式:文件路径、URL、PIL、NumPy、PyTorch 张量、原始字节
兼容性
可将此矩阵用作 v1.2.0 的快速支持速查表。对勾表示该路径在经过验证的文档范围内受支持,exp 表示该路径存在但属实验性,空白单元格表示当前不受支持或不应依赖。
| 模型系列 | v1.2.0 状态 | 推理 | 训练 | 检测 | 分割 | 姿态 | 视线 | ONNX | TorchScript | TensorRT | OpenVINO | NCNN | CoreML |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| YOLO9 | 已验证 detect,单 GPU | ✓ | ✓ | ✓ | exp | Not currently supported | Not currently supported | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| RF-DETR | 已验证 detect + segment,单 GPU | ✓ | ✓ | ✓ | ✓ | Not currently supported | Not currently supported | ✓ | exp | ✓ | ✓ | Not currently supported | exp |
| YOLOX | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | exp | exp |
| YOLO9-E2E | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | exp | Not currently supported |
| YOLO-NAS | 实验性 | exp | exp | exp | Not currently supported | exp | Not currently supported | exp | exp | exp | exp | exp | Not currently supported |
| D-FINE | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | Not currently supported | Not currently supported |
| DEIM | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | Not currently supported | Not currently supported |
| DEIMv2 | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | Not currently supported | Not currently supported |
| RT-DETR | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | Not currently supported | exp |
| PicoDet | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | exp | Not currently supported |
| EC | 实验性 | exp | exp | exp | exp | exp | Not currently supported | exp | exp | exp | exp | Not currently supported | Not currently supported |
| RT-DETRv2 | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | exp | Not currently supported |
| RT-DETRv4 | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | exp | Not currently supported |
| DAMO-YOLO | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | exp | Not currently supported |
| RTMDet | 实验性 | exp | exp | exp | Not currently supported | Not currently supported | Not currently supported | exp | exp | exp | exp | exp | Not currently supported |
| L2CS | 实验性,仅推理 | exp | Not currently supported | Not currently supported | Not currently supported | Not currently supported | exp | Not currently supported | Not currently supported | Not currently supported | Not currently supported | Not currently supported | Not currently supported |
CoreML 导出会生成 .mlpackage 包,并需要 libreyolo[coreml]。CoreML 推理仅限 macOS,不支持 INT8,且 RF-DETR、D-FINE、DEIM、DEIMv2 或 EC 不提供内嵌的 CoreML NMS。
安装
环境要求
- Python 3.10+
- PyTorch 1.13+ 与 torchvision 0.11+
从 PyPI 安装
1 pip install libreyolo
本文档对应即将发布的 v1.2.0 开发分支。在 v1.2.0 发布到 PyPI 之前,请使用源码安装以获得本页所述功能。
从源码安装
1 git clone https://github.com/LibreYOLO/libreyolo.git 2 cd libreyolo 3 git checkout dev 4 pip install -e .
可选依赖
1 # ONNX export and inference 2 pip install libreyolo[onnx] 3 # or: pip install onnx onnxsim onnxruntime 4 5 # RT-DETR compatibility extra (currently no extra packages) 6 pip install libreyolo[rtdetr] 7 8 # RF-DETR support 9 pip install libreyolo[rfdetr] 10 # or: pip install transformers 11 12 # TensorRT export and inference (NVIDIA GPU) 13 pip install libreyolo[tensorrt] 14 # Installs TensorRT CUDA 12 Python packages on Linux/Windows. 15 # Host driver/CUDA compatibility still matters. 16 17 # OpenVINO export and inference (Intel CPU/GPU/VPU) 18 pip install libreyolo[openvino] 19 # INT8 export also needs: pip install nncf 20 21 # NCNN export and inference 22 pip install libreyolo[ncnn] 23 # or: pip install pnnx ncnn 24 25 # ByteTrack API compatibility extra 26 pip install libreyolo[tracking] 27 # Tracking dependencies are part of the base install in v1.2.0 dev. 28 29 # CoreML export and inference (macOS only for runtime) 30 pip install libreyolo[coreml] 31 # or: pip install coremltools 32 33 # L2CS gaze optional auto-download helper 34 pip install libreyolo[gaze] 35 # Optional parity with the upstream RetinaFace-based L2CS pipeline 36 pip install libreyolo[gaze-retinaface] 37 38 # Install every optional LibreYOLO extra 39 pip install libreyolo[all]
如果使用 uv,最可靠的方式是为每个 extra 创建独立的虚拟环境:
1 # ONNX environment 2 uv venv .venv-onnx 3 uv pip install --python .venv-onnx/bin/python -e '.[onnx]' 4 5 # RT-DETR environment 6 uv venv .venv-rtdetr 7 uv pip install --python .venv-rtdetr/bin/python -e '.[rtdetr]' 8 9 # Repeat with .[rfdetr], .[openvino], .[ncnn], .[coreml], .[gaze], .[tracking], or .[tensorrt] as needed
这可避免改动项目环境,并使可选依赖保持隔离。TensorRT、OpenVINO、NCNN 和 CoreML 等特定厂商的 extra 仍可能需要平台专属的原生包。
快速开始
若要使用经过最充分测试的路径,请选择单 GPU 的 YOLO9 检测、RF-DETR 检测或 RF-DETR 分割。它们通过同一个工厂函数加载,接受相同的输入,并返回相同的 Results 对象,因此你可以在它们之间切换而无需改动周边代码。
YOLO9 - CNN 旗舰
1 from libreyolo import LibreYOLO, SAMPLE_IMAGE 2 3 # Use the official checkpoint name and let the factory resolve the details 4 model = LibreYOLO("LibreYOLO9c.pt") 5 6 # Run on a single image (SAMPLE_IMAGE ships with the package) 7 result = model(SAMPLE_IMAGE) 8 9 print(f"Found {len(result)} objects") 10 print(result.boxes.xyxy) # bounding boxes (N, 4) 11 print(result.boxes.conf) # confidence scores (N,) 12 print(result.boxes.cls) # class IDs (N,)
RF-DETR - Transformer 旗舰
1 from libreyolo import LibreYOLO, SAMPLE_IMAGE 2 3 # Same factory, same call shape - just point at an RF-DETR checkpoint 4 model = LibreYOLO("LibreRFDETRs.pt") 5 result = model(SAMPLE_IMAGE) 6 7 print(f"Found {len(result)} objects") 8 print(result.boxes.xyxy)
保存标注后的输出
1 result = model(SAMPLE_IMAGE, save=True) 2 print(result.saved_path) # e.g. runs/detect/predict/parkour.jpg
处理目录
1 results = model("images/", save=True, batch=4) 2 for r in results: 3 print(f"{r.path}: {len(r)} detections")
可用模型
推荐的已验证路径:YOLO9 检测或 RF-DETR 检测 / 分割
这些模型的检测、训练与推理获得了最充分的测试。在 v1.2.0 中,请将其他系列、任务和多 GPU 工作流视为实验性。
LibreYOLO 在 v1.2.0 中提供一小部分经过验证的范围,外加更广泛的受支持模型目录。每个模型都通过同一个 LibreYOLO() 工厂函数加载,但只有下方经过验证的路径才应视为经过深度测试。
YOLO9 - CNN 旗舰
推荐| 尺寸 | 代号 | 输入尺寸 | 适用场景 | 检测检查点 |
|---|---|---|---|---|
| Tiny | "t" | 640 | 快速推理 | LibreYOLO9t.pt |
| Small | "s" | 640 | 均衡 | LibreYOLO9s.pt |
| Medium | "m" | 640 | 更高精度 | LibreYOLO9m.pt |
| Compact | "c" | 640 | 最佳精度 | LibreYOLO9c.pt |
实验性 分割检查点: LibreYOLO9t-seg.pt, LibreYOLO9s-seg.pt, LibreYOLO9m-seg.pt, LibreYOLO9c-seg.pt。参见 分割 章节。
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("LibreYOLO9c.pt") 4 # Experimental segmentation variant 5 # model = LibreYOLO("LibreYOLO9c-seg.pt")
RF-DETR - Transformer 旗舰
推荐| 尺寸 | 代号 | 输入尺寸 | 适用场景 | 检测检查点 |
|---|---|---|---|---|
| Nano | "n" | 384 | 边缘设备 | LibreRFDETRn.pt |
| Small | "s" | 512 | 均衡 | LibreRFDETRs.pt |
| Medium | "m" | 576 | 更高精度 | LibreRFDETRm.pt |
| Large | "l" | 704 | 最高精度 | LibreRFDETRl.pt |
深度测试 分割检查点: LibreRFDETRn-seg.pt, LibreRFDETRs-seg.pt, LibreRFDETRm-seg.pt, LibreRFDETRl-seg.pt, LibreRFDETRx-seg.pt, LibreRFDETRxx-seg.pt。参见 分割 章节。
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("LibreRFDETRs.pt") 4 # Segmentation variants exist for every RF-DETR size 5 # model = LibreYOLO("LibreRFDETRs-seg.pt")
其他受支持的系列
具备检测能力的系列,与经过验证的路径共享同一个工厂函数和 API。它们在 v1.2.0 中属实验性。每个检查点名称都链接到 LibreYOLO 组织 在 Hugging Face 上的模型卡;将任意名称传给 LibreYOLO(),工厂函数会在首次使用时自动获取。
托管说明: YOLO-NAS 检查点(上方以纯文本显示)托管在 Deci 的 CDN 上,遵循其专有权重许可证,而非 LibreYOLO 的 Hugging Face 组织。工厂函数仍会在首次使用时自动下载它们。
专用模型
| 系列 | 状态 | 任务 | 检查点 |
|---|---|---|---|
| L2CS | 实验性 | gaze(仅推理) - 参见 视线估计 | LibreL2CSr50.pt |
L2CS 架构尺寸包括 r18、r34、r50、r101 和 r152,但上游发布的 Gaze360 检查点是 ResNet-50。安装 libreyolo[gaze] 以使用可选的 Google Drive 辅助下载,或为其他尺寸传入本地检查点路径。
工厂函数
对所有模型和运行时都使用 LibreYOLO() 工厂函数。传入官方检查点名称或导出产物路径,让它自动选择正确的模型系列、任务、类别数和运行时:
1 from libreyolo import LibreYOLO 2 3 # Default: YOLO9 detection 4 model = LibreYOLO("LibreYOLO9c.pt") 5 6 # Flagship: RF-DETR 7 model = LibreYOLO("LibreRFDETRs.pt") 8 9 # Segmentation checkpoints use the same factory path 10 model = LibreYOLO("LibreRFDETRs-seg.pt") # validated segmentation 11 model = LibreYOLO("LibreYOLO9c-seg.pt") # experimental segmentation 12 13 # Exported deployment formats 14 model = LibreYOLO("model.onnx") # ONNX Runtime 15 model = LibreYOLO("model.engine") # TensorRT 16 model = LibreYOLO("model.mlpackage") # CoreML (macOS) 17 model = LibreYOLO("model_openvino/") # OpenVINO (directory) 18 model = LibreYOLO("model_ncnn/") # NCNN (directory)
对于可识别的官方检查点文件名,LibreYOLO 可自动下载缺失的权重。对于自定义文件名,请指向明确的本地路径。实验性系列仍通过同一个工厂函数加载,但新项目请坚持使用 YOLO9 检测或 RF-DETR 检测/分割。如有特定理由再使用它们。
任务与文件名
LibreYOLO 使用统一的文件名约定,使工厂函数仅凭检查点名称即可识别系列、尺寸和任务:
1 Libre<FAMILY><size>[-<task>].pt
任务后缀
| 任务 | 规范名称 | 文件名后缀 |
|---|---|---|
| 检测 | "detect" | (无 - 隐式) |
| 实例分割 | "segment" | -seg |
| 姿态估计 | "pose" | -pose |
| 分类 | "classify" | -cls |
| 视线估计 | "gaze" | -gaze |
工厂函数在 API 边界接受别名("detection"、"seg"、"keypoints" 等) - 但文件名中只出现规范名称。
解析优先级
加载模型时,任务按以下顺序解析:
1 explicit task= → checkpoint["task"] → filename suffix → family default
1 from libreyolo import LibreYOLO 2 3 # 1. Filename suffix decides → segment 4 model = LibreYOLO("LibreRFDETRs-seg.pt") 5 6 # 2. Override regardless of filename 7 model = LibreYOLO("custom_weights.pt", task="segment") 8 9 # 3. Detection is implicit 10 model = LibreYOLO("LibreYOLO9c.pt") # task="detect"
各系列任务支持
| 系列 | v1.2.0 状态 | 默认 | 支持的任务 |
|---|---|---|---|
| YOLO9 | detect 单 GPU 深度测试;segment 与多 GPU 实验性 | detect | detect, segment |
| RF-DETR | detect 与 segment 单 GPU 深度测试;多 GPU 实验性 | detect | detect, segment |
| YOLOX | 实验性 | detect | detect |
| YOLO9-E2E | 实验性 | detect | detect |
| YOLO-NAS | 实验性 | detect | detect, pose |
| D-FINE / DEIM / DEIMv2 | 实验性 | detect | detect |
| RT-DETR / RT-DETRv2 / RT-DETRv4 | 实验性 | detect | detect |
| PicoDet | 实验性 | detect | detect |
| EdgeCrafter (EC) | 实验性 | detect | detect, pose, segment |
| DAMO-YOLO / RTMDet | 实验性 | detect | detect |
| L2CS | 实验性 | gaze | gaze(仅推理) |
示例
1 # Detection (implicit) 2 LibreYOLO9c.pt 3 LibreRFDETRs.pt 4 LibreRTDETRr50.pt 5 6 # Segmentation 7 LibreYOLO9c-seg.pt 8 LibreRFDETRs-seg.pt 9 LibreECm-seg.pt 10 11 # Pose 12 LibreYOLONASn-pose.pt 13 LibreECs-pose.pt 14 15 # Gaze 16 LibreL2CSr50.pt # gaze is L2CS's only task - suffix optional
已弃用的别名
LibreYOLORTDETR 和 LibreYOLORFDETR 分别是 LibreRTDETR 和 LibreRFDETR 的旧名称。它们仍可解析,但会触发 DeprecationWarning - 请在方便时更新导入。
预测
单 GPU 预测路径针对 YOLO9 检测、RF-DETR 检测和 RF-DETR 分割经过深度测试。其他系列和任务使用相同的 API,但在 v1.2.0 中属实验性。
基本预测
1 result = model("image.jpg")
全部预测参数
1 result = model( 2 "image.jpg", 3 conf=0.25, # confidence threshold (default: 0.25) 4 iou=0.45, # NMS IoU threshold (default: 0.45) 5 imgsz=640, # input size override (default: model's native) 6 device="auto", # "auto", "cpu", "mps", "0", "cuda:0", ... 7 classes=[0, 2, 5], # filter to specific class IDs (default: all) 8 max_det=300, # max detections per image (default: 300) 9 augment=False, # test-time augmentation where implemented 10 save=True, # save annotated image (default: False) 11 batch=4, # directory batch size 12 stream=False, # video only: yield frame results instead of a list 13 vid_stride=1, # video only: process every N-th frame 14 show=False, # video only: display annotated frames 15 tiling=False, # large-image tiled detection 16 overlap_ratio=0.2, # tile overlap ratio 17 output_path="out/", # where to save (default: runs/detect/predict*/) 18 color_format="auto", # "auto", "rgb", or "bgr" 19 output_file_format="png", # output format: "jpg", "png", "webp" 20 )
model.predict(...) 是 model(...) 的别名。
支持的输入格式
LibreYOLO 接受以下任意格式的图像:
1 # File path (string or pathlib.Path) 2 result = model("photo.jpg") 3 result = model(Path("photo.jpg")) 4 5 # URL 6 result = model("https://example.com/image.jpg") 7 result = model("s3://bucket/image.jpg") 8 result = model("gs://bucket/image.jpg") 9 10 # PIL Image 11 from PIL import Image 12 img = Image.open("photo.jpg") 13 result = model(img) 14 15 # NumPy array (HWC or CHW, RGB or BGR, uint8 or float32) 16 import numpy as np 17 arr = np.random.randint(0, 255, (480, 640, 3), dtype=np.uint8) 18 result = model(arr) 19 20 # OpenCV (BGR) - specify color_format 21 import cv2 22 frame = cv2.imread("photo.jpg") 23 result = model(frame, color_format="bgr") 24 25 # PyTorch tensor (CHW or NCHW) 26 import torch 27 tensor = torch.randn(3, 640, 640) 28 result = model(tensor) 29 30 # Raw bytes 31 with open("photo.jpg", "rb") as f: 32 result = model(f.read()) 33 34 # BytesIO 35 from io import BytesIO 36 result = model(BytesIO(open("photo.jpg", "rb").read())) 37 38 # Directory of images 39 results = model("images/", batch=4)
处理结果
每次预测都会返回一个 Results 对象(处理目录时返回其列表):
1 result = model("image.jpg") 2 3 # Number of detections 4 len(result) # e.g., 5 5 6 # Bounding boxes in xyxy format (x1, y1, x2, y2) 7 result.boxes.xyxy # tensor of shape (N, 4) 8 9 # Bounding boxes in xywh format (center_x, center_y, width, height) 10 result.boxes.xywh # tensor of shape (N, 4) 11 12 # Confidence scores 13 result.boxes.conf # tensor of shape (N,) 14 15 # Class IDs 16 result.boxes.cls # tensor of shape (N,) 17 18 # Combined data: [x1, y1, x2, y2, conf, cls] 19 # Tracking adds a track_id column before conf/cls. 20 result.boxes.data # shape (N, 6), or (N, 7) when tracked 21 22 # Metadata 23 result.orig_shape # (height, width) of original image 24 result.path # source file path (or None) 25 result.names # {0: "person", 1: "bicycle", ...} 26 27 # Move to CPU / convert to numpy 28 result_cpu = result.cpu() 29 boxes_np = result.boxes.numpy()
类别过滤
将检测结果过滤为特定类别 ID:
1 # Only detect people (class 0) and cars (class 2) 2 result = model("image.jpg", classes=[0, 2])
分块推理
对于远大于模型输入尺寸的图像(如卫星影像、无人机航拍),分块推理会将图像切分为相互重叠的小块,分别在每块上运行检测,再合并结果。
在 v1.2.0 dev 中分块仅支持检测。它不接受分割掩码,且无法与 augment=True 组合使用。
1 result = model( 2 "large_aerial_image.jpg", 3 tiling=True, 4 overlap_ratio=0.2, # 20% overlap between tiles (default) 5 save=True, 6 ) 7 8 # Extra metadata on tiled results 9 result.tiled # True 10 result.num_tiles # number of tiles used 11 result.saved_path # output directory when save=True 12 result.tiles_path # directory containing per-tile crops 13 result.grid_path # grid visualization image
在分块时设置 save=True,LibreYOLO 会保存:
final_image.jpg- 绘制了全部合并检测结果的完整图像grid_visualization.jpg- 显示分块网格叠加的图像tiles/- 各个分块裁剪图metadata.json- 分块参数和检测数量
如果图像本身已小于模型的输入尺寸,则会自动跳过分块。
视频推理
将任意视频文件传给旗舰模型,LibreYOLO 会根据扩展名自动识别格式。支持:.mp4、.avi、.mov、.mkv、.webm、.gif 以及其他常见容器格式。
保存标注后的视频
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("LibreYOLO9c.pt") 4 results = model("clip.mp4", save=True) 5 # Saved under runs/detect/predict*/clip.mp4
流式结果(内存恒定)
对于长视频,传入 stream=True 可获得一个生成器。每次迭代产出一帧的 Results - 不会在内存中缓存完整列表。
1 for result in model("long_clip.mp4", stream=True): 2 print(f"frame {result.frame_idx}: {len(result)} detections")
帧抽样
1 # Process every 2nd frame (halves compute and saved fps) 2 results = model("clip.mp4", vid_stride=2, save=True)
实时预览
1 # Display annotated frames in an OpenCV window while processing 2 results = model("clip.mp4", show=True)
用于自定义流水线的 VideoSource / VideoWriter
当你需要完全掌控解码和编码时 - 自定义帧变换、混入跟踪器输出、写入非默认编解码器 - 可直接使用这些构建模块:
1 from libreyolo import LibreYOLO 2 from libreyolo.utils.video import VideoSource, VideoWriter 3 4 model = LibreYOLO("LibreYOLO9c.pt") 5 6 with VideoSource("clip.mp4", vid_stride=1) as src, \ 7 VideoWriter("out.mp4", fps=src.fps, width=src.width, height=src.height) as out: 8 for frame_bgr, frame_idx in src: 9 result = model(frame_bgr, color_format="bgr") 10 # ... draw, transform, etc. 11 out.write_frame(frame_bgr)
跟踪
LibreYOLO 内置 ByteTrack 多目标跟踪器,可消费任意检测器输出的 Results 并添加持久的跟踪 ID。它在单 GPU 的 YOLO9 检测和 RF-DETR 检测上测试最充分;其他检测系列在 v1.2.0 中属实验性。
安装
1 pip install libreyolo[tracking] # compatibility extra; tracking deps ship in base dev install
视频跟踪辅助函数
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("LibreYOLO9c.pt") 4 5 for result in model.track( 6 "clip.mp4", 7 track_conf=0.25, 8 iou=0.45, 9 save=True, # writes runs/track/<video_stem>.mp4 by default 10 vid_stride=1, 11 ): 12 print(result.frame_idx, result.track_id)
model.track() 是用于视频文件的生成器。它逐帧运行检测,内部使用较低的 ByteTrack 置信度进行恢复,并产出已填充 result.track_id 和 result.boxes.id 的 Results。
基本循环
1 from libreyolo import LibreYOLO, ByteTracker 2 from libreyolo.utils.video import VideoSource 3 4 model = LibreYOLO("LibreYOLO9c.pt") 5 tracker = ByteTracker() 6 7 with VideoSource("clip.mp4") as src: 8 for frame_bgr, frame_idx in src: 9 result = model(frame_bgr, color_format="bgr", conf=0.1) 10 tracked = tracker.update(result) 11 12 for i in range(len(tracked.boxes)): 13 track_id = int(tracked.boxes.id[i]) 14 xyxy = tracked.boxes.xyxy[i].tolist() 15 cls = int(tracked.boxes.cls[i]) 16 print(f"frame {frame_idx} - id {track_id} cls {cls} {xyxy}")
调用 tracker.update() 后,result.boxes.id 保存跟踪 ID,且 result.boxes.is_track 为 True。
TrackConfig 可调参数
1 from libreyolo import ByteTracker, TrackConfig 2 3 cfg = TrackConfig( 4 track_high_thresh=0.25, # first-stage match threshold 5 track_low_thresh=0.1, # second-stage (low-conf recovery) 6 new_track_thresh=0.25, # minimum conf to start a new track 7 match_thresh=0.8, # IoU cost cutoff (stage 1) 8 match_thresh_low=0.5, # IoU cost cutoff (stage 2) 9 match_thresh_unconfirmed=0.7, # IoU cost cutoff for unconfirmed tracks 10 track_buffer=30, # frames to keep lost tracks before removal 11 frame_rate=30, # scales track_buffer 12 fuse_score=True, # multiply IoU by detection score 13 minimum_consecutive_frames=1, # frames to confirm a new track 14 ) 15 tracker = ByteTracker(config=cfg)
片段之间重置
1 tracker.reset() # clears tracked / lost / removed lists and the ID counter
分割
v1.2.0 验证范围
经过深度测试的路径是 YOLO9 和 RF-DETR 的检测、训练与推理,包括 RF-DETR 分割。
其他模型系列、任务和多 GPU 工作流均可用,但属实验性。
RF-DETR 分割是 v1.2.0 中经过深度测试的分割路径。YOLO9 分割和 EdgeCrafter 分割可通过相同的 -seg 后缀使用,但属实验性。
运行分割
1 from libreyolo import LibreYOLO 2 3 # RF-DETR segmentation, heavily tested on single GPU 4 model = LibreYOLO("LibreRFDETRs-seg.pt") 5 result = model("photo.jpg") 6 7 # YOLO9 segmentation is available but experimental in v1.2.0 8 # model = LibreYOLO("LibreYOLO9c-seg.pt") 9 10 # Segmentation returns boxes + masks 11 print(result.boxes.xyxy) # bounding boxes (N, 4) 12 print(result.boxes.cls) # class IDs (N,) 13 print(result.masks.data.shape) # (N, H, W) tensor of binary masks
掩码表示
1 # Raw bitmasks 2 result.masks.data # tensor (N, H, W) - original image resolution 3 4 # Polygon contours (one ndarray of (M, 2) per instance) 5 result.masks.xy # absolute pixel coords 6 result.masks.xyn # normalized to [0, 1] 7 8 # Move / convert like Boxes 9 result.masks.cpu() 10 result.masks.numpy()
保存标注后的输出
save=True 会自动绘制边界框和半透明掩码叠加层。
1 model("photo.jpg", save=True)
训练分割模型
RF-DETR 分割使用 RF-DETR 的 COCO 格式训练流水线,属于经过深度测试的单 GPU 范围。YOLO9 分割和 EdgeCrafter 分割训练可用,但在 v1.2.0 中属实验性。
姿态估计
姿态(人体关键点)估计在 YOLO-NAS (-pose) 和 EdgeCrafter (-pose) 上受支持。每个姿态模型均为单类别("person"),包含 17 个 COCO 关键点。
运行姿态估计
1 from libreyolo import LibreYOLO 2 3 # YOLO-NAS pose 4 model = LibreYOLO("LibreYOLONASs-pose.pt") 5 result = model("people.jpg") 6 7 # EdgeCrafter pose 8 # model = LibreYOLO("LibreECs-pose.pt") 9 10 # Per-person bbox + 17 keypoints 11 print(result.boxes.xyxy) # person boxes (N, 4) 12 print(result.keypoints.xy.shape) # (N, 17, 2) pixel coordinates
关键点 API
1 result.keypoints.xy # (N, K, 2) absolute pixel coords 2 result.keypoints.xyn # (N, K, 2) normalized to [0, 1] 3 result.keypoints.conf # (N, K) per-keypoint confidence (None if model doesn't emit it) 4 result.keypoints.has_visible # (N, K) bool - conf > 0 5 6 result.keypoints.cpu() 7 result.keypoints.numpy()
保存标注后的输出
1 model("people.jpg", save=True) # draws boxes + skeleton
YOLO-NAS 支持姿态训练;EdgeCrafter 姿态目前仅支持推理。YOLO9 和 RF-DETR 尚未提供姿态检查点。
视线估计
视线方向估计由 LibreL2CS 系列提供,它是 L2CS-Net 的移植版,采用 ResNet 主干和两个角度分箱分类头。这是一个两阶段模型:上游的人脸检测器先定位人脸,然后视线头以弧度预测每张人脸的俯仰角(pitch)和偏航角(yaw)。它仅支持推理,且在 v1.2.0 中属实验性。
安装
1 pip install libreyolo[gaze] # optional Google Drive helper for Gaze360 weights
已发布的 L2CS ResNet-50 权重在 Gaze360 上训练,LibreYOLO 不进行镜像。若没有可选辅助工具,请传入本地检查点路径,或按照 LibreL2CS 打印的手动下载说明操作。
两阶段推理
1 from libreyolo import LibreYOLO 2 from libreyolo.models.l2cs.face import resolve_face_detector 3 4 # Gaze head 5 gaze = LibreYOLO("LibreL2CSr50.pt") 6 7 # Wire any LibreYOLO detector trained on faces 8 face = LibreYOLO("path/to/face-detector.pt") 9 gaze.face_detector = resolve_face_detector(face) 10 11 result = gaze("portrait.jpg") 12 print(result.boxes.xyxy) # face boxes 13 print(result.gaze.data) # (N, 2) tensor - pitch, yaw in radians
解码角度
1 import math 2 3 for i in range(len(result.gaze)): 4 pitch_rad, yaw_rad = result.gaze.data[i].tolist() 5 pitch_deg = pitch_rad * 180.0 / math.pi 6 yaw_deg = yaw_rad * 180.0 / math.pi 7 print(f"face {i}: pitch={pitch_deg:.1f} deg, yaw={yaw_deg:.1f} deg")
通过命令行:libreyolo predict model=LibreL2CSr50.pt source=portrait.jpg --face-detector path/to/face.pt。
训练
v1.2.0 验证范围
经过深度测试的路径是 YOLO9 和 RF-DETR 的检测、训练与推理,包括 RF-DETR 分割。
其他模型系列、任务和多 GPU 工作流均可用,但属实验性。
经过深度测试的训练路径是单 GPU 的 YOLO9 检测、RF-DETR 检测和 RF-DETR 分割。其他模型系列的训练器、YOLO9 分割训练和多 GPU 工作流可用,但在 v1.2.0 中属实验性。
YOLO9 - CNN 旗舰训练
1 from libreyolo import LibreYOLO 2 3 # Fine-tune from a pretrained checkpoint (recommended) 4 model = LibreYOLO("LibreYOLO9c.pt") 5 6 results = model.train( 7 data="coco128.yaml", # path to data.yaml (required) 8 9 # Schedule 10 epochs=300, # default: 300 11 batch=16, 12 imgsz=640, 13 14 # Optimizer 15 lr0=0.01, # initial learning rate 16 optimizer="SGD", # "SGD", "Adam", "AdamW" 17 18 # System 19 device="0", # "" | "cpu" | "cuda" | "0" | "0,1" 20 workers=8, 21 seed=0, 22 23 # Output 24 project="runs/train", 25 name="yolo9_exp", 26 exist_ok=False, 27 28 # Training features 29 amp=True, # automatic mixed precision 30 patience=50, # early stopping patience 31 resume=False, # resume from loaded checkpoint 32 ) 33 34 print(f"Best mAP50-95: {results['best_mAP50_95']:.3f}") 35 print(f"Best checkpoint: {results['best_checkpoint']}")
训练完成后,模型实例会自动以最佳权重重新加载,因此你可以立即调用 model(...)。YOLO9 分割训练可通过 LibreYOLO("LibreYOLO9c-seg.pt") 使用,但在 v1.2.0 中属实验性。
RF-DETR - Transformer 旗舰训练
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("LibreRFDETRs.pt") 4 5 results = model.train( 6 data="path/to/data.yaml", 7 epochs=100, 8 batch_size=4, # NOTE: RF-DETR uses batch_size, not batch 9 lr=1e-4, 10 output_dir="runs/train/rfdetr_exp", 11 )
RF-DETR 有自己的训练签名(batch_size、lr、output_dir),但它使用 LibreYOLO 的数据集配置加载器。为检测或分割传入 data.yaml;该配置可引用 COCO/Roboflow 风格的标注布局。
训练结果字典
1 { 2 "final_loss": 2.31, 3 "best_mAP50": 0.682, 4 "best_mAP50_95": 0.451, 5 "best_epoch": 87, 6 "save_dir": "runs/train/yolo9_exp", 7 "best_checkpoint": "runs/train/yolo9_exp/weights/best.pt", 8 "last_checkpoint": "runs/train/yolo9_exp/weights/last.pt", 9 }
恢复训练
1 # Load the checkpoint with the factory, then resume 2 model = LibreYOLO("runs/train/yolo9_exp/weights/last.pt") 3 results = model.train(data="coco128.yaml", resume=True)
自定义数据集 YAML 格式
1 path: /path/to/dataset 2 train: images/train 3 val: images/val 4 test: images/test # optional 5 6 nc: 3 7 names: ["cat", "dog", "bird"]
其他训练路径
其他系列也有训练器钩子,但它们不是 v1.2.0 中的推荐路径。新工作请坚持使用 YOLO9 检测或 RF-DETR 检测/分割;仅在兼容、复现基准或定向研究时使用实验性训练器。DAMO-YOLO、PicoDet、RTMDet 和 EC 的训练需要显式的 allow_experimental=True 确认。
通过 YAML 配置训练
每个 model.train(...) 都接受 cfg="train.yaml",以从文件加载全部参数。显式关键字参数仍优先于 yaml 中的值,因此你可以用 yaml 作为基线,并在每次运行时覆盖单个字段。
1 model = LibreYOLO("LibreYOLO9c.pt") 2 results = model.train(cfg="configs/yolo9_finetune.yaml") 3 # Override individual fields: 4 # results = model.train(cfg="configs/yolo9_finetune.yaml", epochs=50)
梯度累积
传入 nbs(名义批量大小)以启用梯度累积。训练器每 nbs / batch 次前向传播执行一次优化器步进,使你能在较小的硬件上以配方的参考批量大小进行训练。
1 # Effective batch 64 on a single GPU that only fits batch=8 2 model.train(data="coco128.yaml", batch=8, nbs=64)
分布式训练(DDP,实验性)
YOLO9 和 RF-DETR 通过 PyTorch DistributedDataParallel 支持多 GPU 训练,但多 GPU 不在 v1.2.0 的深度测试范围内。使用 torchrun 启动训练脚本:
1 # 4-GPU node 2 torchrun --nproc_per_node=4 train_yolo9.py 3 4 # Multi-node - see PyTorch's torchrun docs for --nnodes / --rdzv_endpoint
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("LibreYOLO9c.pt") 4 # Pass device="" (auto-detect) and let torchrun set the rank 5 model.train(data="coco128.yaml", epochs=300, batch=16)
验证
在验证集上运行 COCO 标准评估。经过深度测试的验证路径是单 GPU 的 YOLO9 检测、RF-DETR 检测和 RF-DETR 分割。
1 results = model.val( 2 data="coco128.yaml", # dataset config 3 batch=16, 4 imgsz=640, 5 conf=0.001, # low conf for mAP calculation 6 iou=0.6, # NMS IoU threshold 7 split="val", # "val", "test", or "train" 8 save_json=False, # save predictions as COCO JSON 9 verbose=True, # print per-class metrics 10 ) 11 12 print(f"mAP50: {results['metrics/mAP50']:.3f}") 13 print(f"mAP50-95: {results['metrics/mAP50-95']:.3f}")
验证结果字典
默认情况下,LibreYOLO 使用 COCO 评估,返回精确率、召回率、AP/AR 指标以及每张图像的耗时:
1 { 2 "metrics/mAP50-95": 0.489, # COCO primary metric (AP@[.5:.95]) 3 "metrics/mAP50": 0.721, # AP@0.5 (PASCAL VOC style) 4 "metrics/mAP75": 0.534, # AP@0.75 (strict) 5 "metrics/precision": 0.68, 6 "metrics/recall": 0.61, 7 "metrics/precision(B)": 0.68, # bbox aliases 8 "metrics/recall(B)": 0.61, 9 "metrics/mAP50(B)": 0.721, 10 "metrics/mAP50-95(B)": 0.489, 11 "metrics/mAP_small": 0.291, 12 "metrics/mAP_medium": 0.532, 13 "metrics/mAP_large": 0.648, 14 "metrics/AR1": 0.362, # Average Recall (max 1 det) 15 "metrics/AR10": 0.571, 16 "metrics/AR100": 0.601, 17 "metrics/AR_small": 0.387, 18 "metrics/AR_medium": 0.641, 19 "metrics/AR_large": 0.739, 20 "speed/preprocess_ms": 1.2, 21 "speed/inference_ms": 6.8, 22 "speed/postprocess_ms": 0.9, 23 "speed/total_ms": 8.9, 24 "speed/total_s": 12.3, 25 "speed/images_seen": 1382, 26 }
分割验证返回带 (M) 后缀的掩码指标,以及带 (B) 后缀的边界框指标。姿态验证通过 PoseValidator 返回 COCO 关键点指标。
导出
将 PyTorch 模型导出为 ONNX、TorchScript、TensorRT、OpenVINO、NCNN 或 CoreML 以便部署。经过深度测试的导出与运行时后端路径是单 GPU 的 YOLO9 检测、RF-DETR 检测和 RF-DETR 分割。其他系列和任务属实验性。
快速导出
1 # ONNX (default) 2 model.export() 3 4 # TorchScript 5 model.export(format="torchscript") 6 7 # TensorRT (requires NVIDIA GPU + TensorRT) 8 model.export(format="tensorrt") 9 10 # OpenVINO (optimized for Intel hardware) 11 model.export(format="openvino") 12 13 # NCNN (via PNNX) 14 model.export(format="ncnn") 15 16 # CoreML (.mlpackage, macOS runtime) 17 model.export(format="coreml")
全部导出参数
1 path = model.export( 2 format="onnx", # "onnx", "torchscript", "tensorrt", "openvino", "ncnn", or "coreml" 3 output_path="model.onnx", # output file (auto-generated if None) 4 imgsz=640, # input resolution (default: model's native) 5 opset=None, # ONNX opset (auto: 13, or 17 for wrappers that need it) 6 simplify=True, # run onnxsim graph simplification 7 dynamic=True, # enable dynamic batch axis 8 half=False, # export in FP16 9 batch=1, # batch size for static graph 10 device=None, # device to trace on (default: model's current device) 11 int8=False, # INT8 quantization (TensorRT / OpenVINO only) 12 data=None, # calibration dataset for INT8 13 fraction=1.0, # fraction of calibration data to use 14 allow_download_scripts=False, # allow data.yaml download hooks during calibration 15 workspace=4.0, # TensorRT workspace size (GB) 16 min_batch=1, # TensorRT dynamic profile minimum batch 17 opt_batch=1, # TensorRT dynamic profile optimal batch 18 max_batch=8, # TensorRT dynamic profile maximum batch 19 hardware_compatibility="none", # TensorRT compatibility mode 20 gpu_device=0, # GPU device index for TensorRT 21 trt_config=None, # optional TensorRT YAML config path 22 compute_units="all", # CoreML routing: all, cpu_only, cpu_and_gpu, cpu_and_ne 23 nms=False, # CoreML embedded NMS where supported 24 iou=0.45, # CoreML embedded NMS IoU threshold 25 conf=0.25, # CoreML embedded NMS confidence threshold 26 verbose=False, # verbose logging 27 )
OpenVINO INT8 导出还需要 nncf。NCNN 导出会写出一个目录,包含 model.ncnn.param、model.ncnn.bin 和 metadata.yaml。CoreML 导出会写出 .mlpackage 包,需要 coremltools,且不支持 INT8。
ONNX 元数据
导出的 ONNX 文件包含内嵌元数据:
| 键 | 示例值 |
|---|---|
libreyolo_version | "1.2.0" |
model_family | "yolox" |
model_size | "s" |
nb_classes | "80" |
names | '{"0": "person", "1": "bicycle", ...}' |
imgsz | "640" |
dynamic | "True" |
half | "False" |
使用 LibreYOLO("model.onnx") 加载导出文件时,会自动读回此元数据。
TorchScript 推理
通过相同的运行时后端预测 API 运行导出的 .torchscript 模型。
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("model.torchscript") 4 5 result = model("image.jpg", conf=0.25, iou=0.45, save=True) 6 print(result.boxes.xyxy)
ONNX 推理
使用 ONNX Runtime 而非 PyTorch 运行推理。适用于没有 PyTorch 的部署环境。
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("model.onnx") 4 5 result = model("image.jpg", conf=0.25, iou=0.45, save=True) 6 print(result.boxes.xyxy)
自动元数据
如果 ONNX 文件由 LibreYOLO 导出,类别名称和类别数会自动从内嵌元数据中读取:
1 # Export with metadata 2 model.export(format="onnx", output_path="model.onnx") 3 4 # Load - names and nb_classes auto-populated 5 onnx_model = LibreYOLO("model.onnx") 6 print(onnx_model.names) # {0: "person", 1: "bicycle", ...} 7 print(onnx_model.nb_classes) # 80
对于没有元数据的 ONNX 文件(如由其他工具导出),请手动指定 nb_classes:
1 model = LibreYOLO("external_model.onnx", nb_classes=20)
设备选择
1 # Auto-detect (CUDA if available, else CPU) 2 model = LibreYOLO("model.onnx", device="auto") 3 4 # Force CPU 5 model = LibreYOLO("model.onnx", device="cpu") 6 7 # Force CUDA 8 model = LibreYOLO("model.onnx", device="cuda")
预测参数
通过 LibreYOLO() 加载的运行时产物支持共享的运行时预测 API:
1 result = model( 2 "image.jpg", 3 conf=0.25, 4 iou=0.45, 5 imgsz=640, 6 classes=[0, 2], 7 max_det=300, 8 save=True, 9 output_path="output/annotated.jpg", # final file path when save=True 10 color_format="auto", 11 )
运行时后端不暴露仅限 PyTorch 的选项,如 tiling、overlap_ratio 或 output_file_format。
运行时后端的保存方式与 PyTorch 封装略有不同:如果设置 output_path,请传入最终文件路径,而非目录。如果省略它,当前后端默认保存在 runs/detections/ 下。
TensorRT 推理
使用 TensorRT 在 NVIDIA GPU 上以最大吞吐量运行推理。需要 CUDA 以及 TensorRT 的 Python 绑定。
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("model.engine") 4 5 result = model("image.jpg", conf=0.25, iou=0.45, save=True) 6 print(result.boxes.xyxy)
通过 LibreYOLO() 加载的 TensorRT 产物支持与 ONNX 和 OpenVINO 相同的核心运行时预测 API,包括 save=True 时同样仅接受文件路径的 output_path 行为。
OpenVINO 推理
使用 OpenVINO 运行推理,针对 Intel CPU、GPU 和 VPU 进行了优化。
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("model_openvino/") 4 5 result = model("image.jpg", conf=0.25, iou=0.45, save=True) 6 print(result.boxes.xyxy)
通过 LibreYOLO() 加载的 OpenVINO 目录会在存在时读取 metadata.yaml,并支持相同的核心运行时预测 API。
NCNN 推理
使用 NCNN 运行推理,以在 CPU 或支持 Vulkan 的 GPU 目标上实现轻量化部署。
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("model_ncnn/") 4 5 result = model("image.jpg", conf=0.25, iou=0.45, save=True) 6 print(result.boxes.xyxy)
NCNN 导出目录包含 model.ncnn.param、model.ncnn.bin,通常还有 metadata.yaml。
CoreML 推理
在 macOS 上通过 CoreML 运行导出的 .mlpackage。CoreML 使用 compute_units 而非 PyTorch 设备字符串来路由执行。
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("model.mlpackage", compute_units="all") 4 5 result = model("image.jpg", conf=0.25, iou=0.45, save=True) 6 print(result.boxes.xyxy)
支持的 compute_units 取值为 all、cpu_only、cpu_and_gpu 和 cpu_and_ne。
命令行工具
安装 LibreYOLO 会在你的 PATH 中注册一个 libreyolo 命令(入口点在 pyproject.toml 中)。该命令行镜像 Python API,并遵循 Ultralytics 风格的 key=value 语法。
子命令
| 命令 | 用途 |
|---|---|
predict | 对图像、目录或视频运行推理 |
train | 在数据集上训练模型 |
val | 在数据集上评估模型 |
export | 导出为 ONNX / TorchScript / TensorRT / OpenVINO / NCNN / CoreML |
checks | 打印 Python、torch、CUDA、GPU 及可选包信息 |
models | 列出已注册的模型系列和 CLI 快捷名称 |
formats | 列出支持的导出格式 |
cfg | 打印默认的训练配置 YAML |
info | 加载模型并打印解析出的系列、尺寸、任务、设备和类别 |
metadata | 检视 .pt 文件中的原始检查点元数据 |
version | 打印 LibreYOLO + Python + torch 版本 |
模型名称快捷方式
命令行接受短名称(yolo9-c),它们会解析为权重文件名(LibreYOLO9c.pt) - 可通过 libreyolo models 查看。你也可以传入任意明确的检查点路径。
常用选项
| 命令 | 重要选项 |
|---|---|
predict | conf, iou, imgsz, classes, max_det, half, batch, tiling, overlap_ratio, output_file_format, project, name, exist_ok, face_detector |
train | epochs, batch, imgsz, lr0, optimizer, scheduler, workers, seed, resume, amp, allow_download_scripts, dry_run |
val | split, batch, imgsz, conf, iou, max_det, half, data_dir, use_coco_eval, project, name, exist_ok, save_json, allow_download_scripts |
export | format, imgsz, batch, half, int8, dynamic, simplify, opset, data, fraction, device, allow_download_scripts, verbose |
预测
1 # Flagship: YOLO9 2 libreyolo predict model=yolo9-c source=image.jpg conf=0.25 save=true 3 4 # Flagship: RF-DETR 5 libreyolo predict model=rfdetr-s source=image.jpg save=true 6 7 # Video - saved under runs/detect/predict*/ 8 libreyolo predict model=yolo9-c source=clip.mp4 save=true 9 10 # Tiled inference for very large images 11 libreyolo predict model=yolo9-c source=aerial.jpg tiling=true save=true 12 13 # Gaze (requires a face detector) 14 libreyolo predict model=LibreL2CSr50.pt source=portrait.jpg \ 15 --face-detector path/to/face.pt save=true
训练
1 libreyolo train model=yolo9-c data=coco128.yaml epochs=300 batch=16 device=0 2 3 # Dry-run prints the resolved config without launching training 4 libreyolo train model=yolo9-c data=coco128.yaml --dry-run
验证
1 libreyolo val model=runs/train/exp/weights/best.pt data=coco128.yaml split=val
导出
1 libreyolo export model=runs/train/exp/weights/best.pt format=onnx dynamic=true 2 libreyolo export model=best.pt format=tensorrt half=true 3 libreyolo export model=best.pt format=openvino int8=true data=coco128.yaml 4 libreyolo export model=best.pt format=coreml
机器可读输出
每个命令都接受 --json(结构化的标准输出,便于管道传入脚本或智能体)和 --quiet(抑制 stderr 进度行)。核心的 predict、train、val 和 export 命令还接受 --help-json,可将其参数 schema 以 JSON 形式导出。
1 libreyolo predict model=yolo9-c source=img.jpg --json | jq . 2 3 libreyolo train --help-json > train_schema.json
API 参考
LibreYOLO(工厂函数)
1 LibreYOLO( 2 model_path: str, 3 *, 4 device: str = "auto", 5 task: str | None = None, # override only when a custom artifact is ambiguous 6 nb_classes: int | None = None, # mainly for external exported artifacts 7 compute_units: str = "all", # CoreML only: all, cpu_only, cpu_and_gpu, cpu_and_ne 8 ) -> model wrapper or runtime backend
优先使用官方检查点文件名和导出产物路径,再让工厂函数解析细节。它可处理 PyTorch 检查点、.onnx、.torchscript、.engine、.tensorrt、.mlpackage、包含 model.xml 的 OpenVINO 目录,以及包含 model.ncnn.param 和 model.ncnn.bin 的 NCNN 目录。task 参数用于含糊的自定义产物;否则解析来自检查点元数据、文件名后缀和系列默认值。
预测(PyTorch 模型封装)
1 model( 2 source, # image input (see supported formats) 3 *, 4 conf: float = 0.25, 5 iou: float = 0.45, 6 imgsz: int = None, 7 device: str = "auto", 8 classes: list[int] = None, 9 max_det: int = 300, 10 augment: bool = False, 11 save: bool = False, 12 batch: int = 1, 13 stream: bool = False, 14 vid_stride: int = 1, 15 show: bool = False, 16 output_path: str = None, 17 color_format: str = "auto", 18 tiling: bool = False, 19 overlap_ratio: float = 0.2, 20 output_file_format: str = None, 21 ) -> Results | list[Results] | Generator[Results, None, None]
预测(运行时后端)
1 backend( 2 source, 3 *, 4 conf: float = 0.25, 5 iou: float = 0.45, 6 imgsz: int = None, 7 classes: list[int] = None, 8 max_det: int = 300, 9 save: bool = False, 10 batch: int = 1, 11 output_path: str = None, # final file path when save=True 12 color_format: str = "auto", 13 ) -> Results | list[Results]
如果运行时后端省略 output_path,当前默认保存位置为 runs/detections/。
Results
1 result = Results( 2 boxes: Boxes | None, 3 orig_shape: tuple[int, int], # (height, width) 4 path: str | None, 5 names: dict[int, str], 6 masks: Masks | None = None, 7 keypoints: Keypoints | None = None, 8 probs: Probs | None = None, 9 obb: OBB | None = None, 10 gaze: Gaze | None = None, 11 speed: dict[str, float] | None = None, 12 track_id = None, 13 frame_idx: int | None = None, 14 ) 15 16 len(result) # number of detections 17 result.cpu() # copy with tensors on CPU 18 result.cuda() # copy with tensors on CUDA 19 result.numpy() # copy with numpy arrays 20 result.summary() # list[dict] with boxes, masks, gaze, and track_id when present 21 result.to_json() # JSON string from summary()
Boxes
1 boxes = Boxes(boxes, conf, cls) 2 3 boxes.xyxy # (N, 4) tensor - x1, y1, x2, y2 4 boxes.xywh # (N, 4) tensor - cx, cy, w, h 5 boxes.conf # (N,) tensor - confidence scores 6 boxes.cls # (N,) tensor - class IDs 7 boxes.id # (N,) track IDs when tracking, else None 8 boxes.is_track # True when track IDs are attached 9 boxes.data # (N, 6) [xyxy, conf, cls], or (N, 7) with track IDs 10 11 len(boxes) # number of boxes 12 boxes.cpu() # copy on CPU 13 boxes.numpy() # copy as numpy arrays
任务负载
1 result.masks.data # segmentation masks, (N, H, W) 2 result.masks.xy # list of mask contours in pixel coordinates 3 result.masks.xyn # normalized mask contours 4 5 result.keypoints.xy # pose keypoint coordinates 6 result.keypoints.xyn # normalized keypoint coordinates 7 result.keypoints.conf # keypoint confidence when present 8 9 result.gaze.data # (N, 2): pitch, yaw in radians 10 result.gaze.pitch_deg # pitch in degrees 11 result.gaze.yaw_deg # yaw in degrees 12 result.gaze.direction_3d # approximate 3D direction vectors
model.export()
1 model.export( 2 format: str = "onnx", # "onnx", "torchscript", "tensorrt", "openvino", "ncnn", or "coreml" 3 *, 4 output_path: str | None = None, 5 imgsz: int | None = None, 6 opset: int | None = None, # auto: 13, or 17 for wrappers that need it 7 simplify: bool = True, 8 dynamic: bool = True, 9 half: bool = False, 10 batch: int = 1, 11 device: str | None = None, 12 int8: bool = False, 13 data: str | None = None, # calibration data for INT8 14 fraction: float = 1.0, # fraction of calibration data 15 allow_download_scripts: bool = False, 16 workspace: float = 4.0, # TensorRT workspace (GB) 17 min_batch: int = 1, # TensorRT dynamic profile minimum batch 18 opt_batch: int = 1, # TensorRT dynamic profile optimal batch 19 max_batch: int = 8, # TensorRT dynamic profile maximum batch 20 hardware_compatibility: str = "none", 21 gpu_device: int = 0, 22 trt_config = None, # optional TensorRT YAML config path 23 compute_units: str = "all", # CoreML only 24 nms: bool = False, # CoreML embedded NMS where supported 25 iou: float = 0.45, # CoreML embedded NMS IoU threshold 26 conf: float = 0.25, # CoreML embedded NMS confidence threshold 27 verbose: bool = False, 28 ) -> str # path to exported file or directory
model.val()
1 model.val( 2 data: str = None, # path to data.yaml 3 batch: int = 16, 4 imgsz: int = None, 5 conf: float = 0.001, 6 iou: float = 0.6, 7 workers: int = 4, 8 allow_download_scripts: bool = False, 9 device: str = None, 10 split: str = "val", # "val", "test", or "train" 11 augment: bool = False, 12 save_json: bool = False, 13 verbose: bool = True, 14 ) -> dict
返回(COCO 评估,默认):
1 { 2 "metrics/mAP50-95": float, # COCO primary metric 3 "metrics/mAP50": float, 4 "metrics/mAP75": float, 5 "metrics/mAP_small": float, 6 "metrics/mAP_medium": float, 7 "metrics/mAP_large": float, 8 "metrics/AR1": float, 9 "metrics/AR10": float, 10 "metrics/AR100": float, 11 "metrics/AR_small": float, 12 "metrics/AR_medium": float, 13 "metrics/AR_large": float, 14 }
model.train() (YOLO9)
1 model.train( 2 data: str, # path to data.yaml (required) 3 *, 4 epochs: int = 300, 5 batch: int = 16, 6 imgsz: int = 640, 7 lr0: float = 0.01, 8 optimizer: str = "SGD", 9 device: str = "", 10 workers: int = 8, 11 seed: int = 0, 12 project: str = "runs/train", 13 name: str = "yolo9_exp", 14 exist_ok: bool = False, 15 resume: bool = False, 16 amp: bool = True, 17 patience: int = 50, 18 allow_download_scripts: bool = False, 19 callbacks = None, 20 ) -> dict
返回标准的 LibreYOLO 训练字典,包含 final_loss、best_mAP50、best_mAP50_95、best_epoch、save_dir、best_checkpoint 和 last_checkpoint。
model.train() (RF-DETR)
1 model.train( 2 data: str, # path to data.yaml 3 epochs: int = 100, 4 batch_size: int = 4, 5 lr: float = 1e-4, 6 output_dir: str = "runs/train", 7 resume: str = None, 8 **kwargs, # additional RF-DETR training args 9 ) -> dict
YOLO-NAS、D-FINE、DEIM、DEIMv2、EC、PicoDet、DAMO-YOLO、RT-DETRv2/v4 和 RTMDet 还提供额外的实验性训练器。它们遵循相同的 model.train(data="...yaml", ...) 形式,但其默认值和实验性门控因系列而异。
运行时产物加载
通过 LibreYOLO() 加载导出产物,方式与加载 PyTorch 检查点相同。工厂函数会根据路径选择 ONNX Runtime、TorchScript、TensorRT、OpenVINO、NCNN 或 CoreML:
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("model.onnx") 4 model = LibreYOLO("model.torchscript") 5 model = LibreYOLO("model.engine") 6 model = LibreYOLO("model_openvino/") 7 model = LibreYOLO("model_ncnn/") 8 model = LibreYOLO("model.mlpackage", compute_units="all")
高级集成可以访问更底层的运行时模块,但普通应用代码应坚持使用工厂函数路径。
ValidationConfig
1 from libreyolo import ValidationConfig 2 3 config = ValidationConfig( 4 data="coco128.yaml", 5 data_dir=None, # override dataset root directory 6 split="val", # "val", "test", or "train" 7 batch_size=16, 8 imgsz=640, 9 conf_thres=0.001, 10 iou_thres=0.6, 11 max_det=300, 12 iou_thresholds=( # mAP IoU sweep 13 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, 0.95, 14 ), 15 device="auto", 16 save_dir=None, 17 save_json=False, 18 verbose=True, 19 num_workers=4, 20 half=False, 21 augment=False, # test-time augmentation (TTA) 22 allow_download_scripts=False, 23 # Pose-only fields (PoseValidator) 24 keypoints_json=None, 25 images_dir=None, 26 oks_sigmas=None, 27 ) 28 29 # Load/save YAML 30 config = ValidationConfig.from_yaml("config.yaml") 31 config.to_yaml("config.yaml")
架构指南
本节面向希望了解代码库内部实现的贡献者。
基类设计
PyTorch 模型系列继承自 libreyolo/models/base/model.py 中的 BaseModel。子类需实现以下抽象方法:
| 方法 | 用途 |
|---|---|
_init_model() | 构建并返回 nn.Module |
_get_available_layers() | 返回层名称到模块的映射 |
_get_preprocess_numpy() | 返回用于导出/校准的 NumPy 预处理器 |
_preprocess() | 图像到张量的转换 |
_forward() | 模型前向传播 |
_postprocess() | 原始输出到检测字典 |
BaseModel 提供共享的封装行为:预测、导出、验证、尺寸/名称元数据和训练辅助函数。实际的单图、批量和分块推理流程位于 libreyolo/models/base/inference.py,而部署运行时位于 libreyolo/backends/ 下。
包结构
1 libreyolo/ 2 __init__.py # Public API exports + deprecated-alias resolver 3 tasks.py # Task types, suffix conventions, resolution rules 4 assets/parkour.jpg # SAMPLE_IMAGE 5 models/ 6 __init__.py # LibreYOLO() factory + model registry bootstrap 7 base/ 8 model.py # BaseModel - shared wrapper behaviour 9 inference.py # Shared prediction pipeline (image/dir/video/tiled) 10 yolox/ # LibreYOLOX (detect) 11 yolo9/ # LibreYOLO9 (detect, segment) 12 yolo9_e2e/ # LibreYOLO9E2E (detect) 13 yolonas/ # LibreYOLONAS (detect, pose) 14 dfine/ # LibreDFINE (detect) 15 deim/ # LibreDEIM (detect) 16 deimv2/ # LibreDEIMv2 (detect) 17 rtdetr/ # LibreRTDETR (detect) 18 rtdetrv2/ # LibreRTDETRv2 (detect) 19 rtdetrv4/ # LibreRTDETRv4 (detect) 20 rfdetr/ # LibreRFDETR (detect, segment) - lazy-loaded 21 ec/ # LibreEC / EdgeCrafter (detect, pose, segment) 22 picodet/ # LibrePICODET (detect) 23 damoyolo/ # LibreDAMOYOLO (detect) 24 rtmdet/ # LibreRTMDet (detect) 25 l2cs/ # LibreL2CS (gaze, inference-only) 26 backends/ 27 base.py 28 onnx.py # ONNX Runtime loader 29 torchscript.py # TorchScript loader 30 tensorrt.py # TensorRT loader 31 openvino.py # OpenVINO loader 32 ncnn.py # NCNN loader 33 coreml.py # CoreML loader 34 export/ 35 exporter.py # BaseExporter and format registry 36 onnx.py / torchscript.py / tensorrt.py / openvino.py / ncnn.py / coreml.py 37 config.py / calibration.py 38 training/ 39 trainer.py # Shared trainer scaffolding 40 config.py # TrainConfig dataclass (single source of truth) 41 augment.py / callbacks.py / distributed.py / ema.py / scheduler.py 42 artifacts.py / train_config.yaml 43 # Per-family trainers live in models/<family>/trainer.py 44 validation/ 45 config.py # ValidationConfig 46 base.py / preprocessors.py 47 detection_validator.py # DetectionValidator, SegmentationValidator 48 pose_validator.py # PoseValidator 49 coco_evaluator.py # COCOEvaluator 50 tracking/ 51 tracker.py # ByteTracker 52 config.py # TrackConfig 53 kalman_filter.py / matching.py / strack.py 54 cli/ 55 __init__.py # libreyolo entrypoint (Typer app) 56 commands/ # predict / train / val / export / special 57 aliases.py / config.py / parsing.py / output.py / errors.py 58 utils/ 59 results.py # Results, Boxes, Masks, Keypoints, Probs, OBB, Gaze 60 image_loader.py # Unified image loading 61 video.py # VideoSource, VideoWriter, video inference loop 62 general.py # Path helpers, NMS, tiling utilities 63 download.py / drawing.py / logging.py / predict_args.py 64 serialization.py / box_ops.py 65 data/ 66 dataset.py / pose_dataset.py / utils.py / yolo_coco_api.py 67 config/ 68 datasets/ # Built-in dataset YAML configs (coco8, coco128, coco5000, coco, etc.) 69 export/ # TensorRT default YAML
添加新的模型系列
- 1创建
libreyolo/models/newmodel/model.py,其中包含一个继承BaseModel的类 - 2按需设置
FAMILY、FILENAME_PREFIX、INPUT_SIZES、SUPPORTED_TASKS和DEFAULT_TASK - 3实现注册表钩子,如
can_load()、detect_size()、detect_nb_classes()和detect_size_from_filename() - 4实现该系列所需的模型初始化、预处理、前向、后处理、训练和验证钩子
- 5在
libreyolo/models/newmodel/下创建配套的网络和工具 - 6将该导入添加到
libreyolo/models/__init__.py;导入运行时会进行子类注册 - 7从
libreyolo/__init__.py导出该类 - 8(可选)如果验证预处理与标准路径不同,覆盖
val_preprocessor_class
导出架构
用户代码应通过 model.export(...) 导出。在内部,libreyolo/export/exporter.py 中的 BaseExporter 持有格式注册表,具体的导出器通过子类注册来注册自身。
1 from libreyolo import LibreYOLO 2 3 model = LibreYOLO("LibreYOLO9c.pt") 4 model.export(format="onnx")
要添加新的导出格式,请实现一个具有唯一 format_name 的新 BaseExporter 子类,并从 libreyolo/export/exporter.py 导入它,以填充注册表。
数据集格式
训练和验证使用通过 data.yaml 加载的数据集配置。检测、分割、姿态和 RF-DETR 训练都经由该加载器;标签文件内容因任务而异。
data.yaml 结构
1 path: /absolute/path/to/dataset # dataset root 2 train: images/train # directory path, relative to path 3 val: images/val # directory path, relative to path 4 test: images/test # optional 5 6 nc: 80 # number of classes 7 names: [ # class names 8 "person", "bicycle", "car", "motorcycle", "airplane", 9 "bus", "train", "truck", "boat", "traffic light", 10 # ... 11 ]
配置解析与下载
数据集配置按以下顺序解析:明确路径、当前工作目录,然后是 libreyolo/config/datasets/ 下的内置配置。数据集根目录默认在 ~/datasets 下,可通过 LIBREYOLO_DATASETS_DIR 覆盖。
train、val 和 test 可以是目录、.txt 文件或路径列表。YAML 下载钩子受保护;仅对可信配置传入 allow_download_scripts=True。
文件列表变体
同样的 YAML 格式也可将 train、val 或 test 指向每行一个图像路径的 .txt 文件:
1 path: /absolute/path/to/coco 2 train: train2017.txt 3 val: val2017.txt 4 test: test-dev2017.txt 5 6 nc: 80 7 names: ["person", "bicycle", "car", "..."]
目录布局
1 dataset/ 2 images/ 3 train/ 4 img001.jpg 5 img002.jpg 6 val/ 7 img003.jpg 8 labels/ 9 train/ 10 img001.txt 11 img002.txt 12 val/ 13 img003.txt
检测标签格式
每张图像对应一个文本文件。每行表示一个目标:
1 <class_id> <center_x> <center_y> <width> <height>
所有坐标都相对于图像尺寸归一化到 [0, 1]。
示例(img001.txt):
1 0 0.5 0.4 0.3 0.6 2 2 0.1 0.2 0.05 0.1
分割标签格式
分割使用 YOLO 多边形行。数据集加载器从多边形顶点推导边界框,并在启用分割加载时保留多边形环:
1 <class_id> <x1> <y1> <x2> <y2> ... <xn> <yn>
姿态标签格式
姿态标签在边界框之后追加关键点。在 data.yaml 中添加 kpt_shape 和 flip_idx,使加载器知道关键点数量和水平翻转排列。
1 kpt_shape: [17, 3] 2 flip_idx: [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15]
1 <class_id> <cx> <cy> <w> <h> <kx1> <ky1> <v1> ... <kxK> <kyK> <vK>
内置数据集
LibreYOLO 在 libreyolo/config/datasets/ 下提供内置数据集配置,并可在首次使用时自动下载支持的数据集:
1 # These download automatically on first use 2 results = model.val(data="coco8.yaml") 3 results = model.train(data="coco128.yaml", epochs=10)