查看 Markdown

RF-DETR

一个检测 transformer，它预测的是一组固定数量的目标，而不是一张稠密网格，所以推理时不需要 NMS。LibreYOLO 支持它做四种任务。

任务
detection, instance segmentation, pose, oriented boxes
尺寸
n, s, m, l for detection and oriented boxes; n through xx for segmentation; x for pose
安装
pip install "libreyolo[rfdetr]"
支持层级
旗舰，自 v1.0.0 起。功能在这里最先设计，并完整地在 GPU 上验证。
上游
RF-DETR，由 Roboflow 发布，采用 Apache-2.0 许可。论文、源码
许可
代码采用 MIT，权重采用 Apache-2.0。商用
LibreRFDETRs，在视频上以 512 px 做检测。

安装

RF-DETR 需要自己的 extra，它会为骨干拉进 transformers。

bash
pip install "libreyolo[rfdetr]"

没有它的话，无论从 Python 还是 CLI 加载 RF-DETR 检查点或模型名称，都会报错中止，错误信息里会给出这行安装命令。

预测

权重在首次使用时从 Hugging Face 下载，并缓存在本地。

Python
from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibreRFDETRs.pt")result = model(SAMPLE_IMAGE, save=True) for box in result.boxes:    print(box.cls, box.conf, box.xyxy)
CLI
libreyolo predict model=LibreRFDETRs.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True
视频
from libreyolo import LibreYOLO model = LibreYOLO("LibreRFDETRs.pt") # 库接受的任何数据源：文件、文件夹、URL、摄像头索引、# RTSP 流，或者一个 .streams 列表for result in model.predict("clip.mp4", stream=True, save=True):    print(len(result.boxes))

返回的 Results 对象和每个家族返回的都是同一个，所以换成另一个检测器只是一行的 改动。conf 和 max_det 过滤的是 query 的选择；没有 NMS 步骤需要调。数据源、 流式处理和结果处理见预测。

检测、分割和旋转框路径使用浮点 OpenCV 双线性缩放，不做抗锯齿；姿态路径保留抗锯齿缩放。矩形 imgsz=(height, width) 必须满足任务的 patch/窗口网格要求。检查点列表包含 UI 检测器。事件直方图使用记录的输入配置。

变体

检测有从 n 到 l 的四种尺寸，而四种任务共用一套架构：分割、姿态和旋转框复用 检测的解码器，只是换了一个 head，所以接受的参数完全一样。分割另外多出 x 和 xx，姿态 只有 x。这些尺寸的参数量相近，主要差别在输入分辨率。

检查点输入（px）mAP 50-95参数量（M）
LibreRFDETRn38451.430.47
LibreRFDETRs51255.232.11
LibreRFDETRm57657.433.69
LibreRFDETRl70458.633.93

COCO val2017, 500 images。数据由 LibreYOLO 基准测试工具测得，并发布在 Vision Analysis 上，可在此比较不同硬件和运行时的延迟，并查看完整的运行记录。

训练

四种任务的训练都从已发布的检查点（checkpoint）开始。 pretrained=False 则会重新初始化整个网络，包括骨干，并从头训练。

Python
from libreyolo import LibreYOLO model = LibreYOLO("LibreRFDETRs.pt")model.train(data="my-dataset.yaml", epochs=50, imgsz=512, batch=8, lr0=1e-4)
CLI
libreyolo train model=LibreRFDETRs.pt data=my-dataset.yaml \  epochs=50 imgsz=512 batch=8 lr0=1e-4
LoRA
from libreyolo import LibreYOLO model = LibreYOLO("LibreRFDETRs.pt")model.train(data="my-dataset.yaml", epochs=50, lora=True)
多卡训练
libreyolo train model=LibreRFDETRs.pt data=my-dataset.yaml \  epochs=50 device=0,1 batch=-1

有两个参数在这里比在 CNN 检测器上更要紧。把 lr0 保持在 1e-4 或更低，因为 YOLO 模型能扛住的学习率会让检测 transformer 发散。除非你有理由改，否则把 imgsz 留在 检查点的原生分辨率上。输入必须能被骨干的 patch 大小乘以窗口数整除；LibreYOLO 会在 训练开始前检查这一点，并给出最接近的合法尺寸。

分割微调从分割检查点开始，因为 RF-DETR 没有从检测到分割的迁移。libreyolo train model=LibreRFDETRn-seg.pt task=segment 会加载这个检查点，而对检测检查点传 task=segment 则会报错中止。

数据集、数据增强、多卡训练和日志记录器见训练。

新训练默认使用 output_dir=None，解析为自动递增的 runs/train/rfdetr_exp，并设置 exist_ok=False。断点续训的运行会继续写入其检查点所在的运行目录。多类别姿态数据集使用以类别索引或名称为键的 kpt_names；空列表表示只有检测框的类别。预测将关键点填充到 kpt_shape；关键点 mAP 的适应度不计入只有检测框的类别。

验证

val() 返回一个由 metrics/ 键组成的字典，涵盖 mAP 50、mAP 50-95、mAP 75 和 COCO 平均查全率，在任何与你训练时所用格式相同的数据集上测得。

Python
from libreyolo import LibreYOLO model = LibreYOLO("LibreRFDETRs.pt") # val() 返回的是一个普通 dict，不是对象metrics = model.val(data="my-dataset.yaml", imgsz=512) print(metrics["metrics/mAP50-95"])print(metrics["metrics/mAP50"])print(metrics["metrics/mAP75"], metrics["metrics/AR100"])
CLI
libreyolo val model=LibreRFDETRs.pt data=my-dataset.yaml imgsz=512
在 COCO 上验证
# 内置的 COCO yaml 带着一个内嵌的下载脚本，所以除非数据集已经# 在本地，否则需要显式放行libreyolo val model=LibreRFDETRn.pt data=coco.yaml imgsz=384 \  allow_download_scripts=True

导出

任务ONNXTorchScriptExecuTorchTensorRTOpenVINOPaddleMNNRKNNncnnTFLiteCoreMLCore AI
DetectionDetection to ONNX：支持Detection to TorchScript：支持Detection to ExecuTorch：支持Detection to TensorRT：支持。Runtime parity coverage lives in tests/e2e/test_tensorrt.py.Detection to OpenVINO：支持。Runtime parity coverage lives in tests/e2e/test_openvino.py.Detection to Paddle：不支持Detection to MNN：支持Detection to RKNN：不支持Detection to ncnn：不支持Detection to TFLite：不支持Detection to CoreML：支持。Conversion is available, but runtime parity requires a macOS runner.Detection to Core AI：支持
Instance segmentationInstance segmentation to ONNX：支持Instance segmentation to TorchScript：支持Instance segmentation to ExecuTorch：支持Instance segmentation to TensorRT：支持。A published Apache-2.0 trained segmentation checkpoint exports and reloads, but public top-k class membership changes.Instance segmentation to OpenVINO：支持。After Hungarian query alignment, the converted-runtime element match rate is 69.0%, below the validation bar.Instance segmentation to Paddle：不支持Instance segmentation to MNN：不支持Instance segmentation to RKNN：不支持Instance segmentation to ncnn：不支持Instance segmentation to TFLite：不支持Instance segmentation to CoreML：不支持Instance segmentation to Core AI：不支持
PosePose to ONNX：支持Pose to TorchScript：支持Pose to ExecuTorch：支持Pose to TensorRT：支持。A published Apache-2.0 trained pose checkpoint exports and reloads, but matched public boxes fall to 0.704 IoU with 41.4-pixel coordinate drift.Pose to OpenVINO：支持。After Hungarian query alignment, the converted-runtime element match rate is 72.75%, below the validation bar.Pose to Paddle：不支持Pose to MNN：不支持Pose to RKNN：不支持Pose to ncnn：不支持Pose to TFLite：不支持Pose to CoreML：不支持Pose to Core AI：不支持
Oriented boxesOriented boxes to ONNX：支持Oriented boxes to TorchScript：支持Oriented boxes to ExecuTorch：支持Oriented boxes to TensorRT：支持。A deterministic synthetic OBB fixture exports and reloads, but public top-k class membership changes.Oriented boxes to OpenVINO：支持。After Hungarian query alignment, the converted-runtime element match rate is 91.25%, below the validation bar.Oriented boxes to Paddle：不支持Oriented boxes to MNN：不支持Oriented boxes to RKNN：不支持Oriented boxes to ncnn：不支持Oriented boxes to TFLite：不支持Oriented boxes to CoreML：不支持Oriented boxes to Core AI：不支持

导出的产物按文件后缀经由 LibreYOLO() 重新加载，所以一个 .onnx 或 .engine 文件的表现和检查点一样，返回同样的 Results。在一个不装 LibreYOLO 的裸运行时里跑 这张计算图同样是支持的，但那样预处理和后处理就得你自己写。

Python
from libreyolo import LibreYOLO model = LibreYOLO("LibreRFDETRs.pt")model.export(format="onnx", imgsz=512)model.export(format="tensorrt", imgsz=512, half=True) # 每种格式都接受的参数：##   format    "onnx" | "torchscript" | "executorch" | "tensorrt"#             | "openvino" | "paddle" | "mnn" | "rknn" | "ncnn"#             | "tflite" | "coreml" | "coreai"。#             "engine" 是 tensorrt 的别名，"litert" 是 tflite 的别名#   imgsz     int，或 (height, width)。默认为检查点的原生分辨率#   batch     int，默认 1#   half      bool，以 FP16 导出。默认 False#   int8      bool，以 INT8 导出。默认 False。需要 `data`#   data      数据集 YAML 的路径，用来校准 int8#   fraction  float，取该校准集的多大比例。默认 1.0#   dynamic   bool，动态轴。默认 True#   simplify  bool，跑一次 ONNX 计算图简化。默认 True#   opset     int，ONNX opset。不给时按家族选定#   device    str，在哪个设备上做 trace。默认为模型所在的设备#   output_path  str，默认是一个由检查点派生出来的名字#   verbose   bool，默认 False#   allow_download_scripts  bool，默认 False。放行一个必须下载的#             数据集 YAML 里内嵌的 Python## 少数几种格式还接受自己额外的参数，比如 RKNN 的目标平台。# 这些参数记录在各自格式的页面上
CLI
libreyolo export model=LibreRFDETRs.pt format=onnx imgsz=512libreyolo export model=LibreRFDETRs.pt format=tensorrt imgsz=512 half=True
使用导出的文件
from libreyolo import LibreYOLO, SAMPLE_IMAGE # 工厂按文件后缀分发，所以导出的产物加载方式和任何检查点一样，# 返回的也是同一个 Results 对象model = LibreYOLO("LibreRFDETRs.onnx")result = model(SAMPLE_IMAGE) print(result.boxes.xyxy)
不用 LibreYOLO
import numpy as npimport onnxruntime as ort # 直接跑这张计算图，意味着预处理和后处理都得你自己做。动手# 接线之前先看清楚签名session = ort.InferenceSession("weights/LibreRFDETRs.onnx")name = session.get_inputs()[0].nameoutputs = session.run(None, {name: np.zeros((1, 3, 512, 512), dtype=np.float32)}) for meta, array in zip(session.get_outputs(), outputs):    print(meta.name, array.shape)

ONNX 输入与输出

张量input float32 [batch, 3, height, width]
颜色RGB
缩放像素 / 255
归一化均值 [0.485,0.456,0.406], 标准差 [0.229,0.224,0.225]
缩放方式custom, bilinear

The input tensor is named input, not images.

Square input only. Export rejects a non-square imgsz. The default size comes from the checkpoint and is stored in the imgsz metadata key: 384 for LibreRFDETRn, 312 for LibreRFDETRn-seg, 576 for LibreRFDETRx-pose.

A custom imgsz must be a multiple of patch size x windows: 32 for detection and OBB, 12 for segmentation n, 24 for the other segmentation sizes and for pose.

Stretch the whole image to width x height. There is no letterbox and no padding, so the aspect ratio is not kept.

Detection, segmentation and OBB: convert the 8-bit RGB image to float32 and divide by 255 first, then resize the float image with cv2.resize INTER_LINEAR (half-pixel centers, no antialiasing). Resizing the 8-bit image before dividing is close but moves scores by up to about 0.01.

Pose resizes differently: divide by 255 the same way, then resize the float image with antialiased bilinear, as torch.nn.functional.interpolate does with mode bilinear, align_corners False and antialias True. This is the filter of Pillow's BILINEAR, which widens by the shrink factor, but in float with no rounding to 8 bits. Per axis, let s = in_size / out_size and f = max(s, 1). Output pixel i has center c = (i + 0.5) * s. It reads source pixels j from int(c - f + 0.5) to int(c + f + 0.5) - 1, clipped to the image, each with weight max(0, 1 - |j + 0.5 - c| / f). Divide the weights by their sum.

The pose model is sensitive to this. On the test image, cv2.resize INTER_LINEAR or INTER_AREA adds 3 to 9 detections, and Pillow on the 8-bit image adds one and moves keypoints by more than 4 px.

Then subtract the mean and divide by the std per channel.

nms=True is not available for this family: export raises NotImplementedError.

Detection，默认导出
输出形状含义
dets[batch, queries, 4]Boxes as cx, cy, w, h, normalized 0 to 1 of the input. 300 queries.
labels[batch, queries, columns]Raw class logits per query, sigmoid not applied. The COCO checkpoints have 91 columns, where column k is COCO category id k. For other checkpoints column j is class j.

后处理

  1. Apply sigmoid to labels.
  2. Flatten the scores of one image to queries x columns and take the top k over all of them, with k equal to queries (300). For flat index i, the query is i // columns and the column is i % columns. One query can yield several classes.
  3. Keep the entries whose score is strictly above the confidence threshold (0.25 by default). There is no NMS.
  4. When labels has 91 columns and nc is 80 (the COCO checkpoints), the column is a COCO category id. Class i is the i-th id, counting from 0, in this list: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 27, 28, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 67, 70, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 84, 85, 86, 87, 88, 89, 90. Drop entries whose id is not in the list. With any other column count the column is the class.
  5. Convert the query's box to x1 = cx - w / 2, y1 = cy - h / 2, x2 = cx + w / 2, y2 = cy + h / 2.
  6. Map boxes back to the original image: multiply x by the original width and y by the original height. The input was stretched, so no pad offset applies. LibreYOLO does not clip these boxes.
Instance segmentation，默认导出
输出形状含义
dets[batch, queries, 4]Boxes as cx, cy, w, h, normalized 0 to 1 of the input. 100 queries for sizes n and s, 200 for m and l, 300 for x and xx.
labels[batch, queries, columns]Raw class logits per query, sigmoid not applied. 91 columns for the COCO checkpoints, where column k is COCO category id k.
masks[batch, queries, mask_h, mask_w]Raw mask logits per query over the whole input, at a quarter of the input size: 78 x 78 at 312 x 312.

后处理

  1. Select, threshold, map classes and decode boxes exactly as for detection, with k equal to queries.
  2. Take the mask of each kept query and resize the logits straight to the original image size with bilinear, half-pixel centers and no antialiasing (cv2.resize INTER_LINEAR on the float logits gives the same).
  3. A pixel is foreground when its resized logit is above 0.
Pose，默认导出
输出形状含义
dets[batch, queries, 4]Boxes as cx, cy, w, h, normalized 0 to 1 of the input. 100 queries.
labels[batch, queries, columns]Raw class logits per query, sigmoid not applied, with the keypoint class term already added inside the graph. Column 0 is an empty slot and column j + 1 is class j, so the person checkpoint has 2 columns.
keypoints[batch, queries, columns * num_keypoints, 8]One block of num_keypoints rows per label column, 2 x 17 = 34 rows here. Each row is x, y normalized 0 to 1 of the input, findable logit, visible logit, log_l11, l21, log_l22 and a class logit term. The num_keypoints_per_class metadata key ([0, 17]) says how many rows of each block are real keypoints.

后处理

  1. Apply sigmoid to labels. Flatten to queries x columns and take the top k over all of them, with k equal to queries (100). For flat index i, the query is i // columns and the column is i % columns.
  2. For each entry take its column's keypoint block: rows column * num_keypoints onward, of which the first num_keypoints_per_class[column] are used.
  3. Rescore each entry whose column has keypoints. Per used keypoint, with a = log_l11, b = l21, d = log_l22 and p = sigmoid(findable logit): sigma = exp(-2a) + exp(-2d) + b^2 * exp(-2a - 2d). Then trace = log(sum of p * sigma / sum of p) over the used keypoints, and score = score * exp(-0.2 * trace). Scores can go above 1.
  4. Drop entries whose column is 0. The class is column - 1.
  5. Keep the entries whose new score is strictly above the confidence threshold (0.25 by default). There is no NMS.
  6. Decode and map the box as for detection. LibreYOLO does not clip these boxes.
  7. Keypoint x is x * original width and keypoint y is y * original height. Keypoint confidence is sigmoid(findable logit). LibreYOLO does not use the visible logit.
Oriented boxes，默认导出
输出形状含义
dets[batch, queries, 4]Rotated boxes as cx, cy, w, h, normalized 0 to 1 of the input. 300 queries.
labels[batch, queries, columns]Raw class logits per query, sigmoid not applied. Column j is class j. The 6-class checkpoint has 7 columns: the last one is not a class and LibreYOLO does not drop it.
angles[batch, queries, 1]Rotation in radians from -pi / 2 to pi / 2, tanh already applied, measured on the square input. The width side points along (cos(angle), sin(angle)) with y pointing down.

后处理

  1. Select and threshold as for detection, with k equal to queries (300). The column is the class.
  2. The stretch scales x and y by different amounts, so map each box back through its corners. In normalized input units, with c = cos(angle) and s = sin(angle), the corners are (cx, cy) - u - v, + u - v, + u + v and - u + v, with u = (c * w / 2, s * w / 2) and v = (-s * h / 2, c * h / 2).
  3. Multiply corner x by the original width and corner y by the original height.
  4. Fit cv2.minAreaRect to the 4 corners and convert its angle to radians. Raise w and h to at least 1e-4. If h > w, swap them and add pi / 2 to the angle. Then wrap the angle to [-pi / 2, pi / 2) with angle = (angle + pi / 2) mod pi - pi / 2.
  5. The result is cx, cy, w, h, angle in original-image pixels, with w the long side.

要检查你自己的实现，请用 LibreYOLO 运行同一个文件：LibreYOLO("model.onnx").predict("image.jpg") 会返回参考结果。 已用 LibreYOLO 1.7.0 验证。

检查点

这个家族已发布的全部权重文件。

文件输入（px）权重许可
Detection
LibreRFDETRl.pt704apache-2.0
LibreRFDETRm.pt576apache-2.0
LibreRFDETRn.pt384apache-2.0
LibreRFDETRs.pt512apache-2.0
LibreRFDETRm-ui.ptmit
Instance segmentation
LibreRFDETRn-seg.pt312apache-2.0
LibreRFDETRs-seg.pt384apache-2.0
LibreRFDETRm-seg.pt432apache-2.0
LibreRFDETRl-seg.pt504apache-2.0
LibreRFDETRx-seg.pt624apache-2.0
LibreRFDETRxx-seg.pt768apache-2.0
Pose
LibreRFDETRx-pose.pt576apache-2.0
Oriented boxes
LibreRFDETRn-obb.pt384cc-by-4.0
LibreRFDETRs-obb.pt512cc-by-4.0
LibreRFDETRm-obb.pt576cc-by-4.0
LibreRFDETRl-obb.pt704cc-by-4.0

上面的每个文件目前都在 LibreYOLO 组织中，并会在首次使用时下载。

许可证

请检查你所下载的具体权重在 Hugging Face 仓库中的许可。LibreYOLO 组织里的每个检查点都附有许可，同一家族内也不一定相同。该仓库是权威来源；以下摘要说明本页上次验证时适用的情况。

这里只说明涉及的许可证，不构成法律意见。如果答案对商用很重要，请自行阅读许可证并咨询法律顾问。

原始工作
RF-DETR, Roboflow
上游许可
Apache-2.0
上游来源
github.com/roboflow/rf-detr
LibreYOLO 代码
MIT
权重
采用 Apache-2.0 许可，重新发布在 huggingface.co/LibreYOLO
解读
Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code, and weights you train yourself on your own data are yours.

引用

@inproceedings{robinson2026rfdetr,
  title     = {RF-DETR: Real-Time Detection Transformer},
  author    = {Robinson, Isaac and Robicheaux, Peter and Popov, Matvei and Ramanan, Deva and Peri, Neehar},
  booktitle = {International Conference on Learning Representations (ICLR)},
  year      = {2026},
  url       = {https://arxiv.org/abs/2511.09554}
}

复制自作者在 github.com/roboflow/rf-detr#citation 上提供的引用块。

已针对 LibreYOLO v1.6.0 验证。本页的支持表、检查点和基准测试数据由已发布的库和权重生成，并非手工编写。