此归档版本保持可链接,以便旧版安装、搜索结果和智能体能够指向正确的文档。

简介

LibreYOLO 是一个基于 MIT 许可的目标检测库,专注于两个旗舰模型系列:作为 CNN 旗舰的 YOLO9 与作为 Transformer 旗舰的 RF-DETR。其 API 在预测、训练、验证和导出之间保持一致。

python
1from libreyolo import LibreYOLO
2
3model = LibreYOLO("LibreYOLO9c.pt")
4results = model("image.jpg", conf=0.25, save=True)
5print(results.boxes.xyxy)

核心特性

  • 对 YOLO9 和 RF-DETR 的旗舰支持
  • 从权重自动检测模型架构、尺寸和类别数
  • 针对大尺寸/高分辨率图像的分块推理
  • 支持 ONNX、TorchScript、TensorRT、OpenVINO 和 NCNN 导出,并嵌入元数据
  • ONNX Runtime、TensorRT、OpenVINO 和 NCNN 推理后端
  • 兼容 COCO 的验证,提供 mAP 指标
  • 接受任意图像格式:文件路径、URL、PIL、NumPy、PyTorch 张量、原始字节

安装

环境要求

  • Python 3.10+
  • PyTorch 1.7+

从 PyPI 安装

bash
1pip install libreyolo

从源码安装

bash
1git clone https://github.com/LibreYOLO/libreyolo.git
2cd libreyolo
3pip install -e .

可选依赖

bash
1# ONNX export and inference
2pip install libreyolo[onnx]
3# or: pip install onnx onnxsim onnxscript onnxruntime
4
5# RT-DETR support
6pip install libreyolo[rtdetr]
7# or: pip install transformers timm
8
9# RF-DETR support
10pip install libreyolo[rfdetr]
11# or: pip install rfdetr transformers timm supervision
12
13# TensorRT export and inference (NVIDIA GPU)
14pip install libreyolo[tensorrt]
15# Note: TensorRT itself requires manual installation (depends on CUDA version)
16
17# OpenVINO export and inference (Intel CPU/GPU/VPU)
18pip install libreyolo[openvino]
19# INT8 export also needs: pip install nncf
20
21# NCNN export and inference
22pip install libreyolo[ncnn]
23# or: pip install pnnx ncnn

如果使用 uv,最可靠的方式是为每个 extra 使用独立的虚拟环境:

bash
1# ONNX environment
2uv venv .venv-onnx
3uv pip install --python .venv-onnx/bin/python -e '.[onnx]'
4
5# RT-DETR environment
6uv venv .venv-rtdetr
7uv pip install --python .venv-rtdetr/bin/python -e '.[rtdetr]'
8
9# Repeat with .[rfdetr], .[openvino], .[ncnn], or .[tensorrt] as needed

这样可以避免改动项目环境,并使可选依赖保持隔离。TensorRT、OpenVINO 和 NCNN 等特定厂商的 extra 仍可能需要平台特定的原生包。

快速开始

加载模型并运行推理

python
1from libreyolo import LibreYOLO
2
3# Auto-detects architecture and size from the weights file
4model = LibreYOLO("LibreYOLO9c.pt")
5
6# Run on a single image
7result = model("photo.jpg")
8
9print(f"Found {len(result)} objects")
10print(result.boxes.xyxy) # bounding boxes (N, 4)
11print(result.boxes.conf) # confidence scores (N,)
12print(result.boxes.cls) # class IDs (N,)

保存标注后的输出

python
1result = model("photo.jpg", save=True)
2# Saved under runs/detect/predict*/photo.jpg by default

处理整个目录

python
1results = model("images/", save=True, batch=4)
2for r in results:
3 print(f"{r.path}: {len(r)} detections")

可用模型

v1.2.0 文档以两个旗舰模型系列为中心。YOLO9 是 CNN 旗舰,RF-DETR 是 Transformer 旗舰。其他受支持的系列以简要参考形式列出。

YOLO9 旗舰

尺寸代码输入尺寸用途
Tiny"t"640快速推理
Small"s"640均衡
Medium"m"640更高精度
Compact"c"640最佳精度
python
1from libreyolo import LibreYOLO
2
3model = LibreYOLO("LibreYOLO9t.pt")
4# model = LibreYOLO("LibreYOLO9s.pt")
5# model = LibreYOLO("LibreYOLO9m.pt")
6# model = LibreYOLO("LibreYOLO9c.pt")

RF-DETR 旗舰

尺寸代码输入尺寸用途
Nano"n"384边缘端
Small"s"512均衡
Medium"m"576更高精度
Large"l"704最高精度
python
1from libreyolo import LibreRFDETR
2
3model = LibreRFDETR(size="s")

其他受支持的模型

系列模型类模型名称
YOLOXLibreYOLOXLibreYOLOXn.pt, LibreYOLOXt.pt, LibreYOLOXs.pt, LibreYOLOXm.pt, LibreYOLOXl.pt, LibreYOLOXx.pt
YOLO9-E2ELibreYOLO9E2ELibreYOLO9E2Et.pt, LibreYOLO9E2Es.pt, LibreYOLO9E2Em.pt, LibreYOLO9E2Ec.pt
YOLO-NASLibreYOLONASLibreYOLONASs.pt, LibreYOLONASm.pt, LibreYOLONASl.pt, LibreYOLONASn-pose.pt, LibreYOLONASs-pose.pt, LibreYOLONASm-pose.pt, LibreYOLONASl-pose.pt
D-FINELibreDFINELibreDFINEn.pt, LibreDFINEs.pt, LibreDFINEm.pt, LibreDFINEl.pt, LibreDFINEx.pt
DEIMLibreDEIMLibreDEIMn.pt, LibreDEIMs.pt, LibreDEIMm.pt, LibreDEIMl.pt, LibreDEIMx.pt
DEIMv2LibreDEIMv2LibreDEIMv2atto.pt, LibreDEIMv2femto.pt, LibreDEIMv2pico.pt, LibreDEIMv2n.pt, LibreDEIMv2s.pt, LibreDEIMv2m.pt, LibreDEIMv2l.pt, LibreDEIMv2x.pt
RT-DETRLibreRTDETRLibreRTDETRr18.pt, LibreRTDETRr34.pt, LibreRTDETRr50.pt, LibreRTDETRr50m.pt, LibreRTDETRr101.pt, LibreRTDETRl.pt, LibreRTDETRx.pt
RT-DETRv2LibreRTDETRv2LibreRTDETRv2r18.pt, LibreRTDETRv2r34.pt, LibreRTDETRv2r50.pt, LibreRTDETRv2r50m.pt, LibreRTDETRv2r101.pt
RT-DETRv4LibreRTDETRv4LibreRTDETRv4s.pt, LibreRTDETRv4m.pt, LibreRTDETRv4l.pt, LibreRTDETRv4x.pt
PicoDetLibrePICODETLibrePICODETs.pt, LibrePICODETm.pt, LibrePICODETl.pt
EdgeCrafterLibreECLibreECs.pt, LibreECm.pt, LibreECl.pt, LibreECx.pt, LibreECs-pose.pt, LibreECm-pose.pt, LibreECl-pose.pt, LibreECx-pose.pt, LibreECs-seg.pt, LibreECm-seg.pt, LibreECl-seg.pt, LibreECx-seg.pt
DAMO-YOLOLibreDAMOYOLOLibreDAMOYOLOns.pt, LibreDAMOYOLOnm.pt, LibreDAMOYOLOnl.pt, LibreDAMOYOLOt.pt, LibreDAMOYOLOs.pt, LibreDAMOYOLOm.pt, LibreDAMOYOLOl.pt
RTMDetLibreRTMDetLibreRTMDett.pt, LibreRTMDets.pt, LibreRTMDetm.pt, LibreRTMDetl.pt, LibreRTMDetx.pt

工厂函数(推荐)

LibreYOLO() 工厂函数会从权重文件自动检测一切:

python
1from libreyolo import LibreYOLO
2
3# Auto-detects: YOLOX, size=s, 80 classes
4model = LibreYOLO("LibreYOLOXs.pt")
5
6# Auto-detects: YOLO9, size=c, 80 classes
7model = LibreYOLO("LibreYOLO9c.pt")
8
9# Auto-detects: RT-DETR
10model = LibreYOLO("LibreRTDETRr50.pt")
11
12# RF-DETR checkpoints also work when you point at an actual checkpoint file
13model = LibreYOLO("/path/to/checkpoint_best_regular.pth")
14
15# ONNX models work too
16model = LibreYOLO("model.onnx")
17
18# TensorRT engines
19model = LibreYOLO("model.engine")
20
21# OpenVINO models (directory with model.xml)
22model = LibreYOLO("model_openvino/")
23
24# NCNN models (directory with model.ncnn.param + model.ncnn.bin)
25model = LibreYOLO("model_ncnn/")

对于可识别的官方检查点文件名,LibreYOLO 可以自动下载缺失的权重。对于自定义文件名和 RF-DETR 检查点,建议使用明确的本地路径或特定系列的构造函数。

预测

基本预测

python
1result = model("image.jpg")

全部预测参数

python
1result = model(
2 "image.jpg",
3 conf=0.25, # confidence threshold (default: 0.25)
4 iou=0.45, # NMS IoU threshold (default: 0.45)
5 imgsz=640, # input size override (default: model's native)
6 classes=[0, 2, 5], # filter to specific class IDs (default: all)
7 max_det=300, # max detections per image (default: 300)
8 save=True, # save annotated image (default: False)
9 output_path="out/", # where to save (default: runs/detect/predict*/)
10 color_format="auto", # "auto", "rgb", or "bgr"
11 output_file_format="png", # output format: "jpg", "png", "webp"
12)

model.predict(...)model(...) 的别名。

支持的输入格式

LibreYOLO 接受以下任意格式的图像:

python
1# File path (string or pathlib.Path)
2result = model("photo.jpg")
3result = model(Path("photo.jpg"))
4
5# URL
6result = model("https://example.com/image.jpg")
7
8# PIL Image
9from PIL import Image
10img = Image.open("photo.jpg")
11result = model(img)
12
13# NumPy array (HWC or CHW, RGB or BGR, uint8 or float32)
14import numpy as np
15arr = np.random.randint(0, 255, (480, 640, 3), dtype=np.uint8)
16result = model(arr)
17
18# OpenCV (BGR) — specify color_format
19import cv2
20frame = cv2.imread("photo.jpg")
21result = model(frame, color_format="bgr")
22
23# PyTorch tensor (CHW or NCHW)
24import torch
25tensor = torch.randn(3, 640, 640)
26result = model(tensor)
27
28# Raw bytes
29with open("photo.jpg", "rb") as f:
30 result = model(f.read())
31
32# Directory of images
33results = model("images/", batch=4)

处理结果

每次预测都会返回一个 Results 对象(处理目录时则返回它们的列表):

python
1result = model("image.jpg")
2
3# Number of detections
4len(result) # e.g., 5
5
6# Bounding boxes in xyxy format (x1, y1, x2, y2)
7result.boxes.xyxy # tensor of shape (N, 4)
8
9# Bounding boxes in xywh format (center_x, center_y, width, height)
10result.boxes.xywh # tensor of shape (N, 4)
11
12# Confidence scores
13result.boxes.conf # tensor of shape (N,)
14
15# Class IDs
16result.boxes.cls # tensor of shape (N,)
17
18# Combined data: [x1, y1, x2, y2, conf, cls]
19result.boxes.data # tensor of shape (N, 6)
20
21# Metadata
22result.orig_shape # (height, width) of original image
23result.path # source file path (or None)
24result.names # {0: "person", 1: "bicycle", ...}
25
26# Move to CPU / convert to numpy
27result_cpu = result.cpu()
28boxes_np = result.boxes.numpy()

类别过滤

将检测结果过滤到特定的类别 ID:

python
1# Only detect people (class 0) and cars (class 2)
2result = model("image.jpg", classes=[0, 2])

分块推理

对于远大于模型输入尺寸的图像(例如卫星影像、无人机航拍),分块推理会将图像切分为相互重叠的图块,对每个图块运行检测,然后合并结果。

python
1result = model(
2 "large_aerial_image.jpg",
3 tiling=True,
4 overlap_ratio=0.2, # 20% overlap between tiles (default)
5 save=True,
6)
7
8# Extra metadata on tiled results
9result.tiled # True
10result.num_tiles # number of tiles used
11result.saved_path # output directory when save=True
12result.tiles_path # directory containing per-tile crops
13result.grid_path # grid visualization image

在分块时设置 save=True,LibreYOLO 会保存:

  • final_image.jpg - 绘制了所有合并检测结果的完整图像
  • grid_visualization.jpg - 显示图块网格叠加的图像
  • tiles/ - 单个图块裁剪
  • metadata.json - 分块参数和检测计数

如果图像已经小于模型的输入尺寸,则会自动跳过分块。

训练

v1.2.0 的旗舰训练路径是 YOLO9 和 RF-DETR。为保持 API 兼容性,文档仍保留了其他训练示例。

YOLOX 训练

python
1from libreyolo import LibreYOLOX
2
3model = LibreYOLOX(size="s")
4
5results = model.train(
6 data="coco128.yaml", # path to data.yaml (required)
7
8 # Training parameters
9 epochs=100, # default: 100
10 batch=16,
11 imgsz=640,
12
13 # Optimizer
14 lr0=0.01, # initial learning rate
15 optimizer="SGD", # "SGD", "Adam", "AdamW"
16
17 # System
18 device="0", # GPU device ("", "cpu", "cuda", "0", "0,1")
19 workers=8,
20 seed=0,
21
22 # Output
23 project="runs/train",
24 name="exp",
25 exist_ok=False,
26
27 # Training features
28 amp=True, # automatic mixed precision
29 patience=50, # early stopping patience
30 resume=False, # resume from loaded checkpoint
31)
32
33print(f"Best mAP50-95: {results['best_mAP50_95']:.3f}")
34print(f"Best checkpoint: {results['best_checkpoint']}")

训练完成后,模型实例会自动更新为最佳权重。

训练结果字典

python
1{
2 "final_loss": 2.31,
3 "best_mAP50": 0.682,
4 "best_mAP50_95": 0.451,
5 "best_epoch": 87,
6 "save_dir": "runs/train/exp",
7 "best_checkpoint": "runs/train/exp/weights/best.pt",
8 "last_checkpoint": "runs/train/exp/weights/last.pt",
9}

恢复训练

python
1model = LibreYOLOX("runs/train/exp/weights/last.pt", size="s")
2results = model.train(data="coco128.yaml", resume=True)

自定义数据集 YAML 格式

data.yaml
1path: /path/to/dataset
2train: images/train
3val: images/val
4test: images/test # optional
5
6nc: 3
7names: ["cat", "dog", "bird"]

YOLO9 训练

python
1from libreyolo import LibreYOLO9
2
3model = LibreYOLO9("LibreYOLO9c.pt", size="c")
4
5results = model.train(
6 data="coco128.yaml",
7 epochs=300, # default: 300
8 batch=16,
9 imgsz=640,
10 lr0=0.01,
11 optimizer="SGD",
12 device="0",
13 workers=8,
14 seed=0,
15 project="runs/train",
16 name="yolo9_exp", # default: "yolo9_exp"
17 exist_ok=False,
18 resume=False,
19 amp=True,
20 patience=50,
21)
22
23print(f"Best mAP50-95: {results['best_mAP50_95']:.3f}")

YOLO9 训练使用与 YOLOX 相同的参数 API,但默认 epochs=300name="yolo9_exp"。它没有 pretrained 参数。

RT-DETR 训练

python
1from libreyolo import LibreRTDETR
2
3model = LibreRTDETR(size="r50")
4
5results = model.train(
6 data="coco128.yaml",
7 epochs=72, # default: 72
8 batch=4, # default: 4
9 imgsz=640,
10 lr0=1e-4,
11 lr_backbone=1e-5,
12 optimizer="AdamW",
13 scheduler="linear",
14 device="0",
15 workers=4,
16 seed=0,
17 project="runs/train",
18 name="rtdetr_exp",
19 exist_ok=False,
20 pretrained=True,
21 resume=False,
22 amp=True,
23 patience=50,
24)

RT-DETR 训练使用 YOLO 风格的 data.yaml 流程,但有自己的默认值,并额外增加了 lr_backbonescheduler

RF-DETR 训练

RF-DETR 使用不同的训练 API,它封装了原始的 rfdetr 实现:

python
1from libreyolo import LibreRFDETR
2
3model = LibreRFDETR(size="s")
4
5results = model.train(
6 data="path/to/dataset", # Roboflow/COCO format directory
7 epochs=100,
8 batch_size=4,
9 lr=1e-4,
10 output_dir="runs/train",
11)

RF-DETR 数据集使用 COCO 标注格式:

text
1dataset/
2 train/
3 _annotations.coco.json
4 image1.jpg
5 image2.jpg
6 valid/
7 _annotations.coco.json
8 image1.jpg

验证

在验证集上运行 COCO 标准评估:

python
1results = model.val(
2 data="coco128.yaml", # dataset config
3 batch=16,
4 imgsz=640,
5 conf=0.001, # low conf for mAP calculation
6 iou=0.6, # NMS IoU threshold
7 split="val", # "val", "test", or "train"
8 save_json=False, # save predictions as COCO JSON
9 verbose=True, # print per-class metrics
10)
11
12print(f"mAP50: {results['metrics/mAP50']:.3f}")
13print(f"mAP50-95: {results['metrics/mAP50-95']:.3f}")

验证结果字典

默认情况下,LibreYOLO 使用 COCO 评估并返回 12 个标准指标:

python
1{
2 "metrics/mAP50-95": 0.489, # COCO primary metric (AP@[.5:.95])
3 "metrics/mAP50": 0.721, # AP@0.5 (PASCAL VOC style)
4 "metrics/mAP75": 0.534, # AP@0.75 (strict)
5 "metrics/mAP_small": 0.291,
6 "metrics/mAP_medium": 0.532,
7 "metrics/mAP_large": 0.648,
8 "metrics/AR1": 0.362, # Average Recall (max 1 det)
9 "metrics/AR10": 0.571,
10 "metrics/AR100": 0.601,
11 "metrics/AR_small": 0.387,
12 "metrics/AR_medium": 0.641,
13 "metrics/AR_large": 0.739,
14}

ValidationConfig 中设置 use_coco_eval=False 可获得旧版的 precision/recall 指标。

导出

将 PyTorch 模型导出为 ONNX、TorchScript、TensorRT、OpenVINO 或 NCNN 以便部署。

快速导出

python
1# ONNX (default)
2model.export()
3
4# TorchScript
5model.export(format="torchscript")
6
7# TensorRT (requires NVIDIA GPU + TensorRT)
8model.export(format="tensorrt")
9
10# OpenVINO (optimized for Intel hardware)
11model.export(format="openvino")
12
13# NCNN (via PNNX)
14model.export(format="ncnn")

全部导出参数

python
1path = model.export(
2 format="onnx", # "onnx", "torchscript", "tensorrt", "openvino", or "ncnn"
3 output_path="model.onnx", # output file (auto-generated if None)
4 imgsz=640, # input resolution (default: model's native)
5 opset=13, # ONNX opset version (RT-DETR / RF-DETR default to 17)
6 simplify=True, # run onnxsim graph simplification
7 dynamic=True, # enable dynamic batch axis
8 half=False, # export in FP16
9 batch=1, # batch size for static graph
10 device=None, # device to trace on (default: model's current device)
11 int8=False, # INT8 quantization (TensorRT / OpenVINO only)
12 data=None, # calibration dataset for INT8
13 fraction=1.0, # fraction of calibration data to use
14 workspace=4.0, # TensorRT workspace size (GB)
15 hardware_compatibility="none", # TensorRT compatibility mode
16 gpu_device=0, # GPU device index for TensorRT
17 trt_config=None, # optional TensorRT YAML config path
18 verbose=False, # verbose logging
19)

OpenVINO INT8 导出还需要 nncf。NCNN 导出会写入一个目录,其中包含 model.ncnn.parammodel.ncnn.binmetadata.yaml

ONNX 元数据

导出的 ONNX 文件包含嵌入的元数据:

示例值
libreyolo_version"1.0.0"
model_family"yolox"
model_size"s"
nb_classes"80"
names'{"0": "person", "1": "bicycle", ...}'
imgsz"640"
dynamic"True"
half"False"

使用 OnnxBackend 加载模型时,会自动读回这些元数据。

直接使用导出器工厂

python
1from libreyolo.export import BaseExporter
2
3exporter = BaseExporter.create("onnx", model)
4path = exporter(dynamic=True, simplify=True)

ONNX 推理

使用 ONNX Runtime 而非 PyTorch 运行推理。适用于没有 PyTorch 的部署环境。

python
1from libreyolo import OnnxBackend
2
3model = OnnxBackend("model.onnx")
4
5result = model("image.jpg", conf=0.25, iou=0.45, save=True)
6print(result.boxes.xyxy)

自动元数据

如果 ONNX 文件由 LibreYOLO 导出,则会自动从嵌入的元数据中读取类别名称和类别数:

python
1# Export with metadata
2model.export(format="onnx", output_path="model.onnx")
3
4# Load — names and nb_classes auto-populated
5onnx_model = OnnxBackend("model.onnx")
6print(onnx_model.names) # {0: "person", 1: "bicycle", ...}
7print(onnx_model.nb_classes) # 80

对于没有元数据的 ONNX 文件(例如由其他工具导出),请手动指定 nb_classes

python
1model = OnnxBackend("external_model.onnx", nb_classes=20)

设备选择

python
1# Auto-detect (CUDA if available, else CPU)
2model = OnnxBackend("model.onnx", device="auto")
3
4# Force CPU
5model = OnnxBackend("model.onnx", device="cpu")
6
7# Force CUDA
8model = OnnxBackend("model.onnx", device="cuda")

预测参数

OnnxBackend 支持各运行时后端共享的核心预测 API:

python
1result = model(
2 "image.jpg",
3 conf=0.25,
4 iou=0.45,
5 imgsz=640,
6 classes=[0, 2],
7 max_det=300,
8 save=True,
9 output_path="output/annotated.jpg", # final file path when save=True
10 color_format="auto",
11)

运行时后端不暴露仅限 PyTorch 的选项,例如 tilingoverlap_ratiooutput_file_format

运行时后端处理保存的方式也与 PyTorch 包装器略有不同:如果设置 output_path,请传入最终文件路径,而非目录。如果省略它,当前后端默认保存位置在 runs/detections/ 下。

TensorRT 推理

使用 TensorRT 在 NVIDIA GPU 上以最大吞吐量运行推理。需要 CUDA 以及 TensorRT 的 Python 绑定。

python
1from libreyolo import TensorRTBackend
2
3model = TensorRTBackend("model.engine")
4
5result = model("image.jpg", conf=0.25, iou=0.45, save=True)
6print(result.boxes.xyxy)

通过工厂自动检测

LibreYOLO() 工厂函数会自动检测 .engine 文件:

python
1from libreyolo import LibreYOLO
2
3# Auto-detects TensorRT engine
4model = LibreYOLO("model.engine")

TensorRTBackend 支持与 ONNX 和 OpenVINO 相同的核心运行时后端预测 API,包括 save=True 时同样的仅文件路径 output_path 行为。

OpenVINO 推理

使用 OpenVINO 运行推理,针对 Intel CPU、GPU 和 VPU 进行了优化。

python
1from libreyolo import OpenVINOBackend
2
3model = OpenVINOBackend("model_openvino/")
4
5result = model("image.jpg", conf=0.25, iou=0.45, save=True)
6print(result.boxes.xyxy)

通过工厂自动检测

LibreYOLO() 工厂函数会自动检测 OpenVINO 模型目录:

python
1from libreyolo import LibreYOLO
2
3# Auto-detects OpenVINO directory
4model = LibreYOLO("model_openvino/")

OpenVINOBackend 在存在 metadata.yaml 时会读取它,并支持相同的核心运行时后端预测 API。

NCNN 推理

使用 NCNN 在 CPU 或支持 Vulkan 的 GPU 目标上进行轻量级部署推理。

python
1from libreyolo import NcnnBackend
2
3model = NcnnBackend("model_ncnn/")
4
5result = model("image.jpg", conf=0.25, iou=0.45, save=True)
6print(result.boxes.xyxy)

通过工厂自动检测

LibreYOLO() 工厂函数会自动检测 NCNN 模型目录:

python
1from libreyolo import LibreYOLO
2
3# Auto-detects NCNN directory
4model = LibreYOLO("model_ncnn/")

NCNN 导出目录包含 model.ncnn.parammodel.ncnn.bin,通常还有 metadata.yaml

API 参考

LibreYOLO(工厂)

python
1LibreYOLO(
2 model_path: str,
3 size: str = None, # auto-detected from weights
4 reg_max: int = 16, # YOLO9 only
5 nb_classes: int = None, # auto-detected from weights
6 device: str = "auto",
7) -> LibreYOLOX | LibreYOLO9 | LibreRTDETR | LibreRFDETR | OnnxBackend | TensorRTBackend | OpenVINOBackend | NcnnBackend

从权重文件自动检测模型架构、尺寸和类别数。它还可处理 .onnx.engine、包含 model.xml 的 OpenVINO 目录,以及包含 model.ncnn.parammodel.ncnn.bin 的 NCNN 目录。

预测(PyTorch 模型包装器)

python
1model(
2 source, # image input (see supported formats)
3 *,
4 conf: float = 0.25,
5 iou: float = 0.45,
6 imgsz: int = None,
7 classes: list[int] = None,
8 max_det: int = 300,
9 save: bool = False,
10 batch: int = 1,
11 output_path: str = None,
12 color_format: str = "auto",
13 tiling: bool = False,
14 overlap_ratio: float = 0.2,
15 output_file_format: str = None,
16) -> Results | list[Results]

预测(运行时后端)

python
1backend(
2 source,
3 *,
4 conf: float = 0.25,
5 iou: float = 0.45,
6 imgsz: int = None,
7 classes: list[int] = None,
8 max_det: int = 300,
9 save: bool = False,
10 batch: int = 1,
11 output_path: str = None, # final file path when save=True
12 color_format: str = "auto",
13) -> Results | list[Results]

如果运行时后端省略 output_path,当前默认保存位置为 runs/detections/

Results

python
1result = Results(
2 boxes: Boxes,
3 orig_shape: tuple[int, int], # (height, width)
4 path: str | None,
5 names: dict[int, str],
6)
7
8len(result) # number of detections
9result.cpu() # copy with tensors on CPU

Boxes

python
1boxes = Boxes(boxes, conf, cls)
2
3boxes.xyxy # (N, 4) tensor — x1, y1, x2, y2
4boxes.xywh # (N, 4) tensor — cx, cy, w, h
5boxes.conf # (N,) tensor — confidence scores
6boxes.cls # (N,) tensor — class IDs
7boxes.data # (N, 6) tensor — [xyxy, conf, cls]
8
9len(boxes) # number of boxes
10boxes.cpu() # copy on CPU
11boxes.numpy() # copy as numpy arrays

model.export()

python
1model.export(
2 format: str = "onnx", # "onnx", "torchscript", "tensorrt", "openvino", or "ncnn"
3 *,
4 output_path: str = None,
5 imgsz: int = None,
6 opset: int = 13,
7 simplify: bool = True,
8 dynamic: bool = True,
9 half: bool = False,
10 batch: int = 1,
11 device: str = None,
12 int8: bool = False,
13 data: str = None, # calibration data for INT8
14 fraction: float = 1.0, # fraction of calibration data
15 workspace: float = 4.0, # TensorRT workspace (GB)
16 hardware_compatibility: str = "none",
17 gpu_device: int = 0,
18 trt_config = None, # optional TensorRT YAML config path
19 verbose: bool = False,
20) -> str # path to exported file or directory

BaseExporter

python
1from libreyolo.export import BaseExporter
2
3exporter = BaseExporter.create("onnx", model)
4path = exporter(dynamic=True, simplify=True)
5
6BaseExporter.create("ncnn", model)(output_path="model_ncnn")

model.val()

python
1model.val(
2 data: str = None, # path to data.yaml
3 batch: int = 16,
4 imgsz: int = None,
5 conf: float = 0.001,
6 iou: float = 0.6,
7 device: str = None,
8 split: str = "val", # "val", "test", or "train"
9 save_json: bool = False,
10 verbose: bool = True,
11) -> dict

返回值(COCO 评估,默认):

python
1{
2 "metrics/mAP50-95": float, # COCO primary metric
3 "metrics/mAP50": float,
4 "metrics/mAP75": float,
5 "metrics/mAP_small": float,
6 "metrics/mAP_medium": float,
7 "metrics/mAP_large": float,
8 "metrics/AR1": float,
9 "metrics/AR10": float,
10 "metrics/AR100": float,
11 "metrics/AR_small": float,
12 "metrics/AR_medium": float,
13 "metrics/AR_large": float,
14}

model.train()(YOLOX)

python
1model.train(
2 data: str, # path to data.yaml (required)
3 *,
4 epochs: int = 100,
5 batch: int = 16,
6 imgsz: int = 640,
7 lr0: float = 0.01,
8 optimizer: str = "SGD",
9 device: str = "",
10 workers: int = 8,
11 seed: int = 0,
12 project: str = "runs/train",
13 name: str = "exp",
14 exist_ok: bool = False,
15 pretrained: bool = True,
16 resume: bool = False,
17 amp: bool = True,
18 patience: int = 50,
19) -> dict

返回值:

python
1{
2 "final_loss": float,
3 "best_mAP50": float,
4 "best_mAP50_95": float,
5 "best_epoch": int,
6 "save_dir": str,
7 "best_checkpoint": str,
8 "last_checkpoint": str,
9}

model.train()(YOLO9)

python
1model.train(
2 data: str, # path to data.yaml (required)
3 *,
4 epochs: int = 300,
5 batch: int = 16,
6 imgsz: int = 640,
7 lr0: float = 0.01,
8 optimizer: str = "SGD",
9 device: str = "",
10 workers: int = 8,
11 seed: int = 0,
12 project: str = "runs/train",
13 name: str = "yolo9_exp",
14 exist_ok: bool = False,
15 resume: bool = False,
16 amp: bool = True,
17 patience: int = 50,
18) -> dict

返回与 YOLOX 训练相同的字典。

model.train()(RT-DETR)

python
1model.train(
2 data: str, # path to data.yaml (required)
3 *,
4 epochs: int = 72,
5 batch: int = 4,
6 imgsz: int = 640,
7 lr0: float = 1e-4,
8 lr_backbone: float = 1e-5,
9 optimizer: str = "AdamW",
10 scheduler: str = "linear",
11 device: str = "",
12 workers: int = 4,
13 seed: int = 0,
14 project: str = "runs/train",
15 name: str = "rtdetr_exp",
16 exist_ok: bool = False,
17 pretrained: bool = True,
18 resume: bool = False,
19 amp: bool = True,
20 patience: int = 50,
21) -> dict

model.train()(RF-DETR)

python
1model.train(
2 data: str, # path to dataset directory
3 epochs: int = 100,
4 batch_size: int = 4,
5 lr: float = 1e-4,
6 output_dir: str = "runs/train",
7 resume: str = None,
8 **kwargs, # additional RF-DETR training args
9) -> dict

OnnxBackend

python
1OnnxBackend(
2 onnx_path: str,
3 nb_classes: int = 80, # auto-read from metadata if available
4 device: str = "auto",
5)

使用 ONNX Runtime 对 ONNX 模型运行推理。支持上面所示的运行时后端预测 API。

TensorRTBackend

python
1TensorRTBackend(
2 engine_path: str,
3 nb_classes: int | None = None,
4 device: str = "auto",
5)

对 TensorRT .engine 文件运行推理,并可从相邻的 .json 附属文件读取元数据。

OpenVINOBackend

python
1OpenVINOBackend(
2 model_dir: str,
3 nb_classes: int | None = None,
4 device: str = "auto",
5)

对包含 model.xml 以及可选 metadata.yaml 的 OpenVINO 模型目录运行推理。

NcnnBackend

python
1NcnnBackend(
2 model_dir: str,
3 nb_classes: int | None = None,
4 device: str = "auto",
5)

对包含 model.ncnn.parammodel.ncnn.bin 以及可选 metadata.yaml 的 NCNN 模型目录运行推理。

ValidationConfig

python
1from libreyolo import ValidationConfig
2
3config = ValidationConfig(
4 data="coco128.yaml",
5 data_dir=None, # override dataset root directory
6 batch_size=16,
7 imgsz=640,
8 conf_thres=0.001,
9 iou_thres=0.6,
10 max_det=300,
11 split="val", # "val", "test", or "train"
12 device="auto",
13 save_json=False,
14 verbose=True,
15 half=False,
16 use_coco_eval=True, # use COCO eval (12 keys); False for legacy
17 num_workers=4,
18)
19
20# Load/save YAML
21config = ValidationConfig.from_yaml("config.yaml")
22config.to_yaml("config.yaml")

架构指南

本节面向希望了解代码库内部实现的贡献者。

基类设计

PyTorch 模型系列继承自 libreyolo/models/base/model.py 中的 BaseModel。子类需实现以下抽象方法:

方法作用
_init_model()构建并返回 nn.Module
_get_available_layers()返回层名到模块的映射
_get_preprocess_numpy()返回用于导出/校准的 NumPy 预处理器
_preprocess()图像到张量的转换
_forward()模型前向传播
_postprocess()原始输出到检测字典

BaseModel 提供共享的包装行为:预测、导出、验证、尺寸/名称元数据以及训练辅助函数。实际的单图、批量和分块推理流程位于 libreyolo/models/base/inference.py,而部署运行时位于 libreyolo/backends/ 下。

包结构

text
1libreyolo/
2 __init__.py # Public API exports
3 models/
4 __init__.py # LibreYOLO() factory + model registry bootstrap
5 base/
6 model.py # BaseModel
7 inference.py # Shared prediction pipeline
8 yolox/
9 model.py
10 nn.py
11 utils.py
12 yolo9/
13 model.py
14 nn.py
15 utils.py
16 rtdetr/
17 model.py
18 nn.py
19 trainer.py
20 utils.py
21 rfdetr/
22 model.py
23 utils.py
24 train.py
25 backends/
26 base.py # BaseBackend runtime wrapper
27 onnx.py # OnnxBackend
28 tensorrt.py # TensorRTBackend
29 openvino.py # OpenVINOBackend
30 ncnn.py # NcnnBackend
31 utils/
32 results.py # Results and Boxes classes
33 image_loader.py # Unified image loading
34 general.py # Path helpers, NMS, tiling utilities
35 export/
36 exporter.py # BaseExporter and format registry
37 onnx.py # ONNX export logic
38 torchscript.py # TorchScript export logic
39 tensorrt.py # TensorRT export logic
40 openvino.py # OpenVINO export logic
41 ncnn.py # NCNN export logic
42 training/
43 config.py # YOLOXTrainConfig / YOLOv9TrainConfig
44 trainer.py # YOLOXTrainer
45 v9_trainer.py # YOLOv9Trainer
46 dataset.py # Training dataset
47 augment.py # Mosaic, mixup, etc.
48 loss.py # YOLOX loss functions
49 scheduler.py # LR schedulers
50 ema.py # Exponential moving average
51 validation/
52 config.py # ValidationConfig
53 detection_validator.py # DetectionValidator
54 metrics.py # DetMetrics, mAP computation
55 base.py # BaseValidator
56 preprocessors.py # Per-model val preprocessing
57 data/
58 utils.py # Dataset loading, YAML parsing
59 yolo_coco_api.py # YOLO-to-COCO annotation bridge
60 config/
61 datasets/ # Built-in dataset YAML configs (coco8, coco128, coco5000, coco, etc.)

添加新的模型系列

  1. 1libreyolo/models/newmodel/model.py 中创建一个继承 BaseModel 的类
  2. 2实现所有抽象方法
  3. 3libreyolo/models/newmodel/ 下创建配套的网络和工具
  4. 4将导入添加到 libreyolo/models/__init__.py,以便注册表能识别它
  5. 5libreyolo/__init__.py 导出该类
  6. 6(可选)如果验证预处理与标准流程不同,可重写 val_preprocessor_class

导出架构

libreyolo/export/exporter.py 中的 BaseExporter 是导出入口。具体的导出器通过子类注册机制自行注册,调用方使用 BaseExporter.create(format, model) 获取正确的实现:

python
1from libreyolo.export import BaseExporter
2
3onnx_exporter = BaseExporter.create("onnx", model)
4ncnn_exporter = BaseExporter.create("ncnn", model)

要添加新的导出格式,请实现一个具有唯一 format_name 的新 BaseExporter 子类,并从 libreyolo/export/exporter.py 导入它,以填充注册表。

数据集格式

YOLO 风格的模型使用通过 data.yaml 配置的数据集。RF-DETR 使用 COCO 格式的标注,将在下文单独说明。

data.yaml 结构

data.yaml
1path: /absolute/path/to/dataset # dataset root
2train: images/train # directory path, relative to path
3val: images/val # directory path, relative to path
4test: images/test # optional
5
6nc: 80 # number of classes
7names: [ # class names
8 "person", "bicycle", "car", "motorcycle", "airplane",
9 "bus", "train", "truck", "boat", "traffic light",
10 # ...
11]

文件列表变体

相同的 YAML 格式也可以让 trainvaltest 指向每行包含一个图像路径的 .txt 文件:

coco.yaml
1path: /absolute/path/to/coco
2train: train2017.txt
3val: val2017.txt
4test: test-dev2017.txt
5
6nc: 80
7names: ["person", "bicycle", "car", "..."]

目录布局

text
1dataset/
2 images/
3 train/
4 img001.jpg
5 img002.jpg
6 val/
7 img003.jpg
8 labels/
9 train/
10 img001.txt
11 img002.txt
12 val/
13 img003.txt

标签格式

每张图像对应一个文本文件。每一行表示一个目标:

text
1<class_id> <center_x> <center_y> <width> <height>

所有坐标都相对于图像尺寸归一化到 [0, 1]。

示例(img001.txt):

img001.txt
10 0.5 0.4 0.3 0.6
22 0.1 0.2 0.05 0.1

内置数据集

LibreYOLO 在 libreyolo/config/datasets/ 下附带内置数据集配置,并可在首次使用时自动下载受支持的数据集:

python
1# These download automatically on first use
2results = model.val(data="coco8.yaml")
3results = model.train(data="coco128.yaml", epochs=10)

RF-DETR 数据集格式

RF-DETR 使用 COCO 格式的标注(JSON),而非 YOLO 文本标签:

text
1dataset/
2 train/
3 _annotations.coco.json
4 image1.jpg
5 valid/
6 _annotations.coco.json
7 image1.jpg