查看 Markdown

YOLO-NAS

一个卷积检测器，它的骨干和 neck 出自 Deci.AI 的架构搜索，由量化感知的 RepVGG 模块搭成。它的权重属于 Deci.AI，许可仅限非商业用途，LibreYOLO 一个都不发布。

任务
detection, pose, oriented boxes
尺寸
s, m, l at 640 px
安装
pip install libreyolo
支持层级
核心，自 v1.1.0 起。核心的可训练检测器：功能在同一个发布波次里紧跟旗舰。
上游
YOLO-NAS，由 Deci.AI 发布，采用 Deci.AI proprietary, non-commercial 许可。论文、源码
许可
代码采用 Apache-2.0，权重采用 Deci.AI proprietary, non-commercial。商用

安装

YOLO-NAS 除了基础包之外不需要任何额外依赖。

bash
pip install libreyolo

预测

磁盘上还没有的检查点（checkpoint）名字会从 Deci 的公共 CDN 拉取，而不是从 LibreYOLO 组织拉取，那里不托管这些权重中的任何一个。传输开始之前，库会每个进程 打印一次 Deci 的许可条款；下载下来的文件被打开之前，它的 SHA-256 会先和一个钉死 的值做校验。这些条款允许什么，见许可证。

Python
from libreyolo import LibreYOLO, SAMPLE_IMAGE # 磁盘上还没有的名字会从 Deci 的 CDN 拉取；下载时会先打印 Deci# 的许可条款，拿走文件就等于接受它们model = LibreYOLO("LibreYOLONASs.pt")result = model(SAMPLE_IMAGE, save=True) for box in result.boxes:    print(box.cls, box.conf, box.xyxy)
CLI
libreyolo predict model=LibreYOLONASs.pt save=True \  source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg
姿态
from libreyolo import LibreYOLO, SAMPLE_IMAGE # -pose 后缀会选中姿态 head 和它自己那一套权重model = LibreYOLO("LibreYOLONASs-pose.pt")result = model(SAMPLE_IMAGE) print(result.keypoints.xy)

返回的 Results 对象和每个家族返回的都是同一个，所以换成另一个检测器只是一行的 改动。conf 设置置信度阈值，iou 设置 NMS 阈值。数据源、流式处理和结果处理见 预测。

旋转框任务返回 result.obb。已发布的 OBB 计算图使用 1024 像素画布及其记录的 18 类标签集。

变体

检测和姿态是同一套架构配上不同的 head，接受的参数也一样。下表里的尺寸是检测的 那一组；姿态在这些尺寸之外还多发布了一个更小的尺寸。姿态 head 预测的是 COCO 的 关键点集合。

检查点输入（px）mAP 50-95参数量（M）
LibreYOLONASl64056.366.98
LibreYOLONASm64055.451.18
LibreYOLONASs64051.819.05

COCO val2017, 500 images。数据由 LibreYOLO 基准测试工具测得，并发布在 Vision Analysis 上，可在此比较不同硬件和运行时的延迟，并查看完整的运行记录。

训练

Python
from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLONASs.pt")model.train(data="my-dataset.yaml", epochs=100, imgsz=640, batch=16)
CLI
libreyolo train model=LibreYOLONASs.pt data=my-dataset.yaml \  epochs=100 imgsz=640 batch=16
从零开始训练
from libreyolo import LibreYOLONAS # 完全不碰 Deci 的检查点：模型从随机权重开始，所以这次训练的# 产物只源自你自己的数据model = LibreYOLONAS(None, size="s")model.train(data="my-dataset.yaml", imgsz=640, batch=16)

不传 epochs、lr0 和 amp 时，它们会按任务分别解析，所以一次姿态训练的默认值 和一次检测训练不同。优化器默认是 AdamW。类别数来自数据集 YAML，head 会在第一轮 之前为它重建；在姿态 head 上，关键点数量也是同样的处理方式，所以一个 COCO 姿态 检查点可以微调到另一种规模的骨架上。

微调是从 Deci 的权重开始的，而这正是 Deci 的许可覆盖的部分。从随机初始化的模型 开始训练则完全不涉及任何 Deci 检查点，上面第三个代码片段就是这条路。

数据集、数据增强、多卡训练和日志记录器见训练。

检测默认使用 amp=True 和 amp_dtype="float16"；旋转框训练保留 amp=False。OBB head 支持训练、预测和验证，使用翻转/HSV 数据增强，并通过 metrics/mAP50-95(OBB) 选择检查点。load_detect_weights_for_obb() 用检测权重初始化它。

验证

val() 返回一个由 metrics/ 键组成的字典，涵盖查准率、查全率、mAP 50 和 mAP 50-95，衡量对象是任何符合你训练时所用格式的数据集。

Python
from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLONASs.pt")metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/mAP50-95"])print(metrics["metrics/mAP50"])
CLI
libreyolo val model=LibreYOLONASs.pt data=my-dataset.yaml
在 COCO 上验证
# 自带的 COCO yaml 里内嵌了一段下载脚本，所以除非数据集已经在# 本地，否则需要显式授权libreyolo val model=LibreYOLONASl.pt data=coco.yaml imgsz=640 \  allow_download_scripts=True

导出

任务ONNXTorchScriptExecuTorchTensorRTOpenVINOPaddleMNNRKNNncnnTFLiteCoreMLCore AI
DetectionDetection to ONNX：支持Detection to TorchScript：支持Detection to ExecuTorch：支持Detection to TensorRT：支持Detection to OpenVINO：支持Detection to Paddle：支持Detection to MNN：支持Detection to RKNN：支持Detection to ncnn：支持Detection to TFLite：支持Detection to CoreML：不支持Detection to Core AI：支持
PosePose to ONNX：支持Pose to TorchScript：支持Pose to ExecuTorch：支持Pose to TensorRT：支持Pose to OpenVINO：支持Pose to Paddle：支持Pose to MNN：不支持Pose to RKNN：不支持Pose to ncnn：支持Pose to TFLite：不支持Pose to CoreML：不支持Pose to Core AI：不支持
Oriented boxesOriented boxes to ONNX：支持Oriented boxes to TorchScript：支持Oriented boxes to ExecuTorch：支持Oriented boxes to TensorRT：支持Oriented boxes to OpenVINO：支持Oriented boxes to Paddle：不支持Oriented boxes to MNN：不支持Oriented boxes to RKNN：不支持Oriented boxes to ncnn：支持Oriented boxes to TFLite：不支持Oriented boxes to CoreML：不支持Oriented boxes to Core AI：不支持

导出的产物按文件后缀经由 LibreYOLO() 重新加载，所以一个 .onnx 或 .engine 文件的表现和检查点一样，返回同样的 Results。在不装 LibreYOLO 的裸运行时里跑 这张图也是支持的，但那时预处理和后处理就得你自己写。每种格式安装的 extra 不同， 各自也接受几个自己的参数。这两件事都在对应格式的页面上。

一次导出只是把同样的权重换一个容器再存一份。导出一个 Deci 检查点既不会改变权重 的来源，也不会改变覆盖它们的许可。

Python
from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLONASs.pt")model.export(format="onnx", imgsz=640)
CLI
libreyolo export model=LibreYOLONASs.pt format=onnx imgsz=640
使用导出的文件
from libreyolo import LibreYOLO, SAMPLE_IMAGE # 工厂按文件后缀分发，所以导出的产物加载方式和任何检查点一样，# 返回的也是同一个 Results 对象model = LibreYOLO("LibreYOLONASs.onnx")result = model(SAMPLE_IMAGE) print(result.boxes.xyxy)

ONNX 输入与输出

LibreYOLONAS detect

张量images float32 [batch, 3, height, width]
颜色RGB
缩放像素 / 255
归一化无
缩放方式custom, Pillow bilinear
如何复现 Pillow 的双线性缩放

LibreYOLO 使用 Pillow 的 BILINEAR 滤波器(Image.resize)缩放。它不是 OpenCV 的 INTER_LINEAR 或 INTER_AREA:缩小时,Pillow 会按缩小倍数加宽滤波器,而 OpenCV 的缩放会改变检测结果,并使框移动数个像素。要在不使用 Pillow 的情况下复现它,先缩放一个轴,舍入到 8 位,再缩放另一个轴并再次舍入。对每个轴,令 s = in_size / out_size,f = max(s, 1)。输出像素 i 的中心为 c = (i + 0.5) * s,读取源像素 j,范围从 int(c - f + 0.5) 到 int(c + f + 0.5) - 1(限制在图像内),每个像素的权重为 max(0, 1 - |j + 0.5 - c| / f),再除以权重之和。Pillow 使用定点运算,并根据图像形状选择轴的顺序,因此浮点权重在少数像素上可能与它相差一个等级。这不会改变我们检查中的结果。

Square input only. Export rejects a non-square imgsz, and height equals width, 640 by default.

The image is not scaled to fill the canvas. The longest side goes to R = min(636, height, width), so 636 at 640 x 640.

Scale by r = min(R / h, R / w) and resize to (round(w * r), round(h * r)). The rounding is Python round, so an exact .5 goes to the even neighbor.

Paste the resized image onto a canvas filled with 114 at left = (width - new_w) // 2 and top = (height - new_h) // 2.

On the test image, cv2.resize with INTER_LINEAR loses two of the eight detections, and INTER_AREA moves boxes by up to 1.4 px.

Divide by 255 after the resize. There is no mean or std normalization.

Detection，默认导出
输出形状含义
pred_logits[batch, anchors, 4]Boxes, despite the name. x1, y1, x2, y2 in input-canvas pixels. anchors is the cell count over strides 8, 16 and 32, 8400 at 640 x 640.
pred_boxes[batch, anchors, nc]Class scores, despite the name. Sigmoid already applied. 80 classes for the COCO checkpoints.

后处理

  1. The output names are swapped relative to their content. Read boxes from pred_logits and scores from pred_boxes, or tell them apart by the last dimension (4 for boxes).
  2. Take the best class per anchor: score = max over classes, class = argmax. One anchor yields at most one detection.
  3. Keep anchors whose score is strictly above the confidence threshold (0.25 by default).
  4. Keep the 1000 highest scoring candidates.
  5. Map boxes back to the original image: subtract left from x and top from y, divide by r, clip to the image. Drop boxes left with zero width or height.
  6. Run NMS per class with IoU 0.45 and keep at most 300 detections.
  7. nms=True is not available for this family: export raises NotImplementedError.

LibreYOLONAS pose

张量images float32 [batch, 3, height, width]
颜色BGR
缩放像素 / 255
归一化无
缩放方式custom, Pillow bilinear
如何复现 Pillow 的双线性缩放

LibreYOLO 使用 Pillow 的 BILINEAR 滤波器(Image.resize)缩放。它不是 OpenCV 的 INTER_LINEAR 或 INTER_AREA:缩小时,Pillow 会按缩小倍数加宽滤波器,而 OpenCV 的缩放会改变检测结果,并使框移动数个像素。要在不使用 Pillow 的情况下复现它,先缩放一个轴,舍入到 8 位,再缩放另一个轴并再次舍入。对每个轴,令 s = in_size / out_size,f = max(s, 1)。输出像素 i 的中心为 c = (i + 0.5) * s,读取源像素 j,范围从 int(c - f + 0.5) 到 int(c + f + 0.5) - 1(限制在图像内),每个像素的权重为 max(0, 1 - |j + 0.5 - c| / f),再除以权重之和。Pillow 使用定点运算,并根据图像形状选择轴的顺序,因此浮点权重在少数像素上可能与它相差一个等级。这不会改变我们检查中的结果。

Square input only. Export rejects a non-square imgsz, and height equals width, 640 by default.

Channels are BGR, unlike detection. Pad value and position also differ from detection.

The longest side goes to R = min(640, height, width), so it fills the canvas.

Scale by r = min(R / h, R / w) and resize to (round(w * r), round(h * r)). The rounding is Python round, so an exact .5 goes to the even neighbor.

Paste the resized image in the top-left corner of a canvas filled with 127. All padding is on the bottom and the right.

On the test image, cv2.resize with INTER_LINEAR or INTER_AREA moves boxes by up to 5 px and keypoints by more than 10 px.

Divide by 255 after the resize. There is no mean or std normalization.

Pose，默认导出
输出形状含义
boxes[batch, anchors, 4]Person boxes as x1, y1, x2, y2 in input-canvas pixels. anchors is the cell count over strides 8, 16 and 32, 8400 at 640 x 640.
scores[batch, anchors, nc]Class scores, sigmoid already applied. nc is 1 (person) for the COCO pose checkpoints.
keypoints_xy[batch, anchors, num_keypoints, 2]Keypoint x, y in input-canvas pixels. num_keypoints is 17 (COCO order) for the COCO pose checkpoints.
keypoints_conf[batch, anchors, num_keypoints]Per-keypoint visibility score, sigmoid already applied.

后处理

  1. Take the best class per anchor: score = max over classes, class = argmax. With one class this is just the person score.
  2. Keep anchors whose score is at or above the confidence threshold (0.25 by default). This is >=, not > as in detection.
  3. Keep the 1000 highest scoring candidates.
  4. Map boxes and keypoints back to the original image: divide x and y by r. The image sits in the top-left corner, so there is no pad offset to subtract.
  5. Clip boxes to the image and drop boxes left with zero width or height. Keypoints are not clipped and can fall outside the image.
  6. Run NMS per class with IoU 0.45 and keep at most 300 detections. Keep each survivor's keypoints as (x, y, keypoints_conf).
  7. nms=True is not available for this family: export raises NotImplementedError.

LibreYOLONAS obb

张量images float32 [batch, 3, height, width]
颜色BGR
缩放像素 / 255
归一化无
缩放方式custom, bilinear

Square input only. Export rejects a non-square imgsz, and height equals width, 1024 by default.

Channels are BGR, unlike detection. The resize filter and the pad position also differ from detection.

The longest side goes to R = min(1024, height, width), so it fills the canvas.

Scale by r = min(R / h, R / w) and resize to (round(w * r), round(h * r)). The rounding is Python round, so an exact .5 goes to the even neighbor.

The resize is cv2.resize with INTER_LINEAR, not Pillow as in detection and pose. It is skipped when the image already has that size. On the test image, Pillow's filter here changes the detection count.

Paste the resized image in the top-left corner of a canvas filled with 114. All padding is on the bottom and the right.

Divide by 255 after the resize. There is no mean or std normalization.

Oriented boxes，默认导出
输出形状含义
boxes[batch, anchors, 5]Rotated boxes as cx, cy, w, h, angle. cx, cy, w and h are in input-canvas pixels. The angle is in radians. The graph computes it as (sigmoid(logit) - 0.25) x pi, so it lies between -pi/4 and 3pi/4 and is not wrapped yet. The w side points along (cos angle, sin angle) in image coordinates, y down. anchors is the cell count over strides 8, 16 and 32, 21504 at 1024 x 1024.
scores[batch, anchors, nc]Class scores, sigmoid already applied. 18 classes (DOTA 2.0) for the published checkpoints.

后处理

  1. Take the best class per anchor: score = max over classes, class = argmax. One anchor yields at most one detection.
  2. Keep anchors whose score is at or above the confidence threshold (0.25 by default). This is >=, not > as in detection.
  3. Keep the 1000 highest scoring candidates.
  4. Map boxes back to the original image: divide cx, cy, w and h by r. The image sits in the top-left corner, so there is no pad offset, and the angle does not change.
  5. Clip cx to 0 to the image width and cy to 0 to the image height. w and h are not clipped. Drop boxes with w or h at or below 0.
  6. Put the long side first: if h > w, swap w and h and add pi/2 to the angle. Then wrap the angle to [-pi/2, pi/2) with angle = ((angle + pi/2) mod pi) - pi/2.
  7. Run greedy NMS per class on exact rotated-box IoU (polygon intersection area over union, for example cv2.rotatedRectangleIntersection) with IoU 0.45, and keep at most 300 detections. LibreYOLO does not use upstream's matrix soft-NMS.
  8. LibreYOLO also reports an axis-aligned box per detection. With ex = abs(cos angle) * w / 2 + abs(sin angle) * h / 2 and ey = abs(sin angle) * w / 2 + abs(cos angle) * h / 2, it is cx - ex, cy - ey, cx + ex, cy + ey, clipped to the image.
  9. nms=True is not available for this family: export raises NotImplementedError.

要检查你自己的实现，请用 LibreYOLO 运行同一个文件：LibreYOLO("model.onnx").predict("image.jpg") 会返回参考结果。 已用 LibreYOLO 1.7.0 验证。

检查点

这里没有可列的。Deci 的许可禁止再分发，所以 LibreYOLO 组织不发布任何 YOLO-NAS 权重，下载会解析到别处：形如 LibreYOLONAS<size>.pt 的名字、姿态用的 LibreYOLONAS<size>-pose.pt，或者旋转框用的 LibreYOLONAS<size>-obb.pt（s、m、l）， 会映射到 Deci 公共 CDN 上对应的对象。

只有 SHA-256 被库钉死的检查点才能这样拉取。其他的一律失败关闭，而不是去打开一个 未经校验的第三方 pickle，得手动下载并以路径的形式传进来。已经在磁盘上的文件按 路径加载，没有下载，也没有校验和这道关。这也包括保持原始文件名的 Deci .pth， 加载器认得它。

许可证

请检查你所下载的具体权重在 Hugging Face 仓库中的许可。LibreYOLO 组织里的每个检查点都附有许可，同一家族内也不一定相同。该仓库是权威来源；以下摘要说明本页上次验证时适用的情况。

这里只说明涉及的许可证，不构成法律意见。如果答案对商用很重要，请自行阅读许可证并咨询法律顾问。

原始工作
YOLO-NAS, Deci.AI
上游许可
Deci.AI proprietary, non-commercial
上游来源
github.com/Deci-AI/super-gradients
LibreYOLO 代码
MIT
权重
采用 Deci.AI proprietary, non-commercial 许可，由作者分发。LibreYOLO 不托管或镜像这些权重。
解读
Two licenses apply here and they are not the same. The SuperGradients source this port follows is Apache-2.0 and LibreYOLO's own implementation is MIT, so nothing restricts the architecture or the training code. Deci's published checkpoints are a different matter: the YOLO-NAS license grants a revocable, non-transferable right to use them and nothing more. It forbids redistribution and sublicensing, forbids offering them as a managed or remote service, restricts modification and reverse engineering, and rules out commercial use, including any model running in a production environment, unless you hold a separate agreement with Deci. LibreYOLO therefore does not mirror them, and prints those terms once before the download starts. Weights trained from a randomly initialized model on your own data derive from no Deci checkpoint.

LibreYOLO 既不托管也不镜像这些权重：LibreYOLO 的 Hugging Face 组织里没有任何属于 这个家族的东西。所有自动下载都改为走 Deci 的公共 CDN，开始之前按进程打印一次 Deci 的条款，并在文件被打开之前对照一个钉死的 SHA-256 做校验。

另一条路是从随机初始化的模型开始训练。这套架构在上游采用 Apache-2.0 许可，在这里 采用 MIT 许可，所以用你自己的数据这样训练出来的模型不派生自任何 Deci 检查点。

引用

YOLO-NAS 发布时没有论文。下面这一条是它的作者要求引用的，指向 SuperGradients， 也就是它随之发布的那个库。

@misc{supergradients,
  doi = {10.5281/ZENODO.7789328},
  url = {https://zenodo.org/record/7789328},
  author = {Aharon,  Shay and {Louis-Dupont} and {Ofri Masad} and Yurkova,  Kate and {Lotem Fridman} and {Lkdci} and Khvedchenya,  Eugene and Rubin,  Ran and Bagrov,  Natan and Tymchenko,  Borys and Keren,  Tomer and Zhilko,  Alexander and {Eran-Deci}},
  title = {Super-Gradients},
  publisher = {GitHub},
  journal = {GitHub repository},
  year = {2021},
}

复制自作者在 github.com/Deci-AI/super-gradients#citing 上提供的引用块。

已针对 LibreYOLO v1.6.0 验证。本页的支持表、检查点和基准测试数据由已发布的库和权重生成，并非手工编写。