MiDaS

Install

MiDaS needs the midas extra for its timm encoders.

pip install "libreyolo[midas]"

Predict

The s and l checkpoints download from LibreYOLO's mirrors under the publisher's MIT grant and are cached locally.

from libreyolo import LibreYOLO, SAMPLE_IMAGE # Downloads the mirrored checkpoint on first use.model = LibreYOLO("LibreMiDaSl-depth.pt")result = model(SAMPLE_IMAGE, save=True) depth = result.depth_mapprint(depth.min, depth.max, depth.mean)

libreyolo predict model=LibreMiDaSl-depth.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True

from libreyolo import LibreYOLO, SAMPLE_IMAGE # EfficientNet-Lite3 encoder, smaller and faster than the DPT-Large l size.model = LibreYOLO("LibreMiDaSs-depth.pt")result = model(SAMPLE_IMAGE, save=True)

result.depth_map carries a dense relative inverse-depth map: higher values mean closer to the camera, and the values have no metric unit or cross-image scale. save=True writes a colormapped visualization of that map to disk; Results.plot() renders the depth map. See prediction for sources, streaming and result handling.

Variants

Two variants with different encoders, not just different scales of the same one. s is MiDaS v2.1 Small, an EfficientNet-Lite3 encoder. l is DPT-Large, a ViT-L/16 encoder with the DPT decoder MiDaS introduced for dense prediction. They also preprocess differently: s uses an upper-bound aspect resize with ImageNet mean/std normalization, l uses a minimal aspect resize with mean and std of 0.5. Pick s for a lighter CNN, l for the transformer decoder's accuracy.

Training is not offered for this family. LibreMiDaS.train() raises NotImplementedError unconditionally.

Validate

val() runs the shared depth validator: it aligns each prediction to its ground truth with a per-image least-squares scale and shift, then reports the standard zero-shot relative-depth metrics, AbsRel, RMSE and the three delta thresholds.

from libreyolo import LibreYOLO model = LibreYOLO("LibreMiDaSl-depth.pt")metrics = model.val(data="my-dataset.yaml") print(metrics["metrics/abs_rel"])print(metrics["metrics/rmse"])print(metrics["metrics/delta1"])

libreyolo val model=LibreMiDaSl-depth.pt data=my-dataset.yaml

Export

An exported artifact loads back through LibreYOLO() on its file suffix, so a .onnx or .engine file behaves like a checkpoint and returns the same Results, with depth_map in place of boxes.

from libreyolo import LibreYOLO model = LibreYOLO("LibreMiDaSl-depth.pt")model.export(format="onnx")model.export(format="tensorrt", half=True)

libreyolo export model=LibreMiDaSl-depth.pt format=onnxlibreyolo export model=LibreMiDaSl-depth.pt format=tensorrt half=True

from libreyolo import LibreYOLO, SAMPLE_IMAGE # The factory routes on the file suffix, so an exported artifact loads# like any checkpoint and returns the same Results object.model = LibreYOLO("LibreMiDaSl-depth.onnx")result = model(SAMPLE_IMAGE) print(result.depth_map.data.shape)

Licensing

Citation

@ARTICLE {Ranftl2022,
    author  = "Ren\'{e} Ranftl and Katrin Lasinger and David Hafner and Konrad Schindler and Vladlen Koltun",
    title   = "Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer",
    journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",
    year    = "2022",
    volume  = "44",
    number  = "3"
}

@article{Ranftl2021,
	author    = {Ren\'{e} Ranftl and Alexey Bochkovskiy and Vladlen Koltun},
	title     = {Vision Transformers for Dense Prediction},
	journal   = {ICCV},
	year      = {2021},
}