NVIDIA DeepStream

Availability

DeepStream export ships in v1.5.0. It merged into dev on 2026-08-08 in pull request 728, so a current install has it and no branch pin is needed.

pip install "libreyolo[onnx]"

If you cloned the deepstream-export branch before 2026-08-08, replace it. That branch was rebased and force-pushed, and the older history is missing the fix that lets these exports run on a CUDA machine at all.

What the export writes

model.export(format="onnx", deepstream=True) writes three files side by side. For libreyolo9s.pt:

• libreyolo9s.onnx, the detection graph, one output tensor of shape (batch, num_detections, 6), each row [x1, y1, x2, y2, score, class_id] in network-input pixel coordinates.
• config_infer_primary_libreyolo9s.txt, an nvinfer configuration carrying the family's preprocessing constants, class count, thresholds and parser wiring.
• libreyolo9s_labels.txt, one class name per line.

A labels file appears whenever the checkpoint carries class names. Depth models have none, so they get neither the file nor a labelfile-path key.

LibreYOLO does not emit a .so. The .so that DeepStream loads is the bounding-box parser from marcoslucianops/DeepStream-Yolo, compiled once per device, and it is the same binary whichever LibreYOLO detector you point it at. The model is the ONNX. Classification and semantic segmentation need no parser at all, because nvinfer post-processes those itself.

Export the model

from libreyolo import LibreYOLO9, LibreDFINE # Writes libreyolo9s.onnx, config_infer_primary_libreyolo9s.txt# and libreyolo9s_labels.txt into the working directory.LibreYOLO9("libreyolo9s.pt", size="s").export(format="onnx", deepstream=True) # Keep each detection model in its own directory: every detection# config names the same engine cache file. See "Known traps".LibreDFINE("LibreDFINEs.pt", size="s").export(format="onnx", deepstream=True)

model.export(    format="onnx",     # deepstream=True is rejected for every other format    deepstream=True,    conf=0.25,         # seeds pre-cluster-threshold (and classifier-threshold,                       # segmentation-threshold on those tasks)    iou=0.45,          # seeds nms-iou-threshold, omitted at cluster-mode=4    batch=1,           # seeds batch-size and the engine cache filename    half=False,        # True marks the config network-mode=2 (fp16 build)    int8=False,        # True marks the config network-mode=1    dynamic=True,      # dynamic batch axis in the ONNX graph    imgsz=640,         # seeds infer-dims=3;H;W) # deepstream=True and nms=True are mutually exclusive: DeepStream runs# suppression in its clustering stage, so nothing is embedded in the graph.

curl -L -o LibreDFINEs.pt \  https://huggingface.co/LibreYOLO/LibreDFINEs/resolve/main/LibreDFINEs.pt

LibreDFINE._load_weights raises FileNotFoundError when the file is not already on disk, without attempting a download, so fetch LibreDFINEs.pt yourself first. That gap is tracked as issue #727. YOLO9 weights download on first use.

The flag is Python only. libreyolo export on this branch has no deepstream option, and the CLI builds its export arguments from a fixed list rather than passing unknown keys through.

Build the bounding-box parser

Detection needs the parser library, instance segmentation needs a different one, and the remaining tasks need none. Two things on the DeepStream 8.0 image break the documented build command, and both are environmental rather than LibreYOLO problems.

The image ships cuda, cuda-12, cuda-12.5, cuda-12.8 and cuda-12.9 under /usr/local. Only cuda-12.5 has a complete toolkit. It also ships libcublas.so.12 and libcublas.so.12.8.4.1 but not the unversioned libcublas.so that -lcublas resolves against. The script below works around both.

set -egit clone --depth 1 https://github.com/marcoslucianops/DeepStream-Yolo.git # /usr/local/cuda-12 on this image is a stub and the build dies on it with# "fatal error: crt/host_defines.h: No such file or directory". Resolve a# toolkit that actually carries the header; on the 8.0 image that is cuda-12.5.CUDA_DIR=$(readlink -f /usr/local/cuda)[ -f "$CUDA_DIR/include/crt/host_defines.h" ] || \  CUDA_DIR=$(ls -d /usr/local/cuda-*.* | sort -Vr | \             while read d; do [ -f "$d/include/crt/host_defines.h" ] && echo "$d" && break; done) # The image ships libcublas.so.12 and libcublas.so.12.8.4.1 but not the# unversioned libcublas.so that -lcublas needs, so the link step fails with# "/usr/bin/ld: cannot find -lcublas". Give the linker the names it wants.mkdir -p /tmp/cudalibsfor lib in cublas cublasLt cudart; do  real=$(find /usr/local -name "lib${lib}.so.1*" | grep -v stubs | sort -V | tail -1)  ln -sf "$real" "/tmp/cudalibs/lib${lib}.so"doneexport LIBRARY_PATH="/tmp/cudalibs:$LIBRARY_PATH" make -C DeepStream-Yolo/nvdsinfer_custom_impl_Yolo CUDA_VER="${CUDA_DIR##*/cuda-}"

git clone --depth 1 https://github.com/marcoslucianops/DeepStream-Yolo-Seg.gitmake -C DeepStream-Yolo-Seg/nvdsinfer_custom_impl_Yolo_seg \  CUDA_VER="${CUDA_DIR##*/cuda-}"

Then point custom-lib-path in the generated config at the built libnvdsinfer_custom_impl_Yolo.so. The generated value is the relative path nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so, which resolves when deepstream-app runs from the DeepStream-Yolo checkout and needs editing otherwise.

Run the pipeline

Check that the container can see the GPU before spending time on anything else. This is the check the validation run made first, on a Blackwell card under WSL2.

docker run --rm --gpus all nvcr.io/nvidia/tritonserver:26.04-py3 \  nvidia-smi --query-gpu=name,driver_version,compute_cap --format=csv

name, driver_version, compute_cap
NVIDIA GeForce RTX 5070 Ti, 591.86, 12.0

The validation run drove deepstream-app with one file source, no display sink, the on-screen display on, and gie-kitti-output-dir set so every frame's detections landed on disk as KITTI text. A config with those settings:

[application]enable-perf-measurement=1perf-measurement-interval-sec=5gie-kitti-output-dir=kitti [tiled-display]enable=0 [source0]enable=1type=3uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4num-sources=1gpu-id=0 [streammux]gpu-id=0batch-size=1batched-push-timeout=40000width=1920height=1080live-source=0 [primary-gie]enable=1gpu-id=0gie-unique-id=1config-file=config_infer_primary_libreyolo9s.txt [osd]enable=1border-width=2text-size=15 [sink0]enable=1type=1sync=0 [tests]file-loop=0

deepstream-app -c deepstream_app_config.txt

App run successful

docker run --rm --gpus all -v "$PWD:/work" -w /work \  nvcr.io/nvidia/deepstream:8.0-samples-multiarch \  bash -c "bash build_parser.sh && deepstream-app -c deepstream_app_config.txt"

nvinfer builds the TensorRT engine from the ONNX on first run and caches it next to the model, so the first run pays for the engine build and later ones load the cache.

The generated config

Both configs below were written by the exporter for the validation run, not edited afterwards.

The two configs differ in three places: maintain-aspect-ratio, cluster-mode, and whether nms-iou-threshold is present at all. D-FINE's config omits that key entirely, which is what cluster-mode=4 calls for.

Heads that emit at most one prediction per object get cluster-mode=4, so DeepStream runs no clustering over them; clustering would merge genuinely distinct detections. That covers rfdetr, dfine, deim, deimv2, ec, rtdetr, rtdetrv2, rtdetrv4 and yolo9_e2e. Grid and anchor heads get cluster-mode=2 plus nms-iou-threshold.

Detection configs also carry engine-create-func-name=NvDsInferYoloCudaEngineGet, which hands engine building to the parser library. That is what fixes the engine cache filename, and it is the source of the collision described under known traps.

Supported tasks and families

Forty-three family and task combinations export. deepstream_supported_tasks() and deepstream_supported_families(task) in libreyolo/export/deepstream.py return the same lists at runtime.

network-type=100 means DeepStream has no post-processor for the task. Those configs set output-tensor-meta=1, the graph's native outputs pass through untouched, and the application decodes them from the tensor metadata. Multi-output graphs are fine there: every output layer reaches the metadata with the same output names and dynamic axes as a plain ONNX export.

Instance segmentation rows are the detection row followed by that instance's mask, flattened at (netH / 4, netW / 4), which is the resolution the seg parser hardcodes, as probabilities for segmentation-threshold.

Classification and gaze run as secondary inference. Set process-mode=2 and operate-on-gie-id in the generated config to put a classifier behind a detector. Gaze is a head-only contract, one face crop per input, so it needs a face detector in front of it.

Three families are absent on purpose. segformer is not wired to the shared semantic export contract and cannot export to ONNX in any format. RTMDet-Ins and YOLO9 have their instance segmentation export blocked in LibreYOLO itself. depth_anything3 has no export implementation.

Two rows in the table have checkpoint gaps behind them. Only the l EoMT semantic checkpoint is published, and DINOv2 classification has no published checkpoint at all, so that combination needs your own fine-tuned weights.

Preprocessing differences

nvinfer computes net-scale-factor * (x - offsets) per channel with a scalar scale, which cannot express per-channel standard deviation. Families that need one (rfdetr, ec, the DINO-backboned deimv2 sizes, rtmdet, picodet, and every classification family) have the normalization baked into the exported graph, and the generated config feeds the graph the matching raw input space.

The geometry is where LibreYOLO's own Python pipelines and nvinfer still diverge:

• Letterbox families (yolo9, yolox, yolonas, rtmdet, yolo2, yolo3, yolo4, yolo7) pad with gray natively. nvinfer pads black.
• yolonas detection natively resizes the longest side to 636 inside its 640 canvas. nvinfer's maintain-aspect-ratio uses the full 640.
• Classification natively resizes the shortest side then center-crops. nvinfer stretches the frame or object ROI to the network input, so tightly cropped subjects differ.
• EoMT natively runs sliding-window tiles for semantic segmentation. The exported graph is a single stretched canvas, which is faster and less accurate.
• pidnet emits a class map at 1/8 of the input resolution and lingbotvision at 1/16. DeepStream upsamples the class map for display.

The ONNX parity gate feeds already-preprocessed tensors, so it checks graph outputs and cannot catch a wrong color order or padding policy in the config. Validate on your own data before deploying an exact-parity workload.

Known traps

Two detection models in one directory load each other's engine

Every detection config carries the same line:

model-engine-file=model_b1_gpu0_fp32.engine

The parser's engine builder requires that basename and it does not vary by model. Export a second detection model into the same directory and the second run loads the first model's cached engine. Nothing crashes; the boxes are just wrong. Give each detection model its own directory. The validation run had to isolate D-FINE into one before it could be tested at all.

A box can only carry one class

nvinfer's row format is [x1, y1, x2, y2, score, class_id], one class per box, so the export collapses class scores to their argmax. A box that predict reports under two classes survives under one. Measured case: LibreYOLO reports vase 0.773 and bottle 0.383 on the same box, and the DeepStream graph keeps vase. This follows from the parser's row format and cannot be changed without leaving that contract, so it is expected behavior rather than a regression.

Validated

deepstream-app ran to EOS with App run successful on both detector head types, over NVIDIA's bundled sample_1080p_h264.mp4 (1443 frames), with per-frame KITTI dumps enabled.

Class histograms over all 1443 frames put cars first and people second for both models, which is right for a street scene. The four-fold gap in detection count is the cluster-mode difference doing its job: D-FINE at cluster-mode=4 runs no clustering, so every query above threshold survives, near-duplicates included.

Two independently trained models put the dominant object in the same place:

YOLO9  bus  [706.72,  0.82, 1916.34, 1062.97]  conf 0.965D-FINE bus  [702.73,  2.93, 1916.24, 1069.32]  conf 0.965

That run establishes five things: TensorRT builds an engine from the exported ONNX on sm_120, nvinfer accepts every key in the generated config, NvDsInferParseYolo reads the tensor layout correctly, boxes land in source-resolution 1920x1080 coordinates, and labels resolve against the generated labels file.

The environment it ran in:

Alongside the pipeline run, tests/unit/test_deepstream_export.py covers the graph adapters and the generated config keys, and its 35 tests pass on this commit.

Not validated

Stated so the scope above is not read wider than it is.

• Jetson and aarch64. The export contract does not depend on the architecture, but the pipeline has only been run on an x86 discrete GPU.
• Forty-one of the 43 combinations. Only detection with yolo9 and detection with dfine went through DeepStream. Classification, semantic segmentation, instance segmentation and the raw-tensor tasks are covered by unit tests and ONNX parity checks, not by a pipeline run.
• FP16 and INT8. Only network-mode=0 was exercised.
• Multi-stream and batching. One source, batch-size=1.
• Accuracy against a ground-truth dataset. Detections were checked for semantic plausibility and cross-model agreement, not scored as mAP through DeepStream.