# MoGe-3
MoGe-3 is Microsoft's monocular geometry model. From one RGB image it predicts metric depth in metres, a camera-space point map, the camera intrinsics, a validity mask and surface normals. It is MoGe-2 plus a sparse 3D UNet that refines the predicted depth on a voxel grid. LibreYOLO ships the official ViT-L and ViT-g checkpoints for inference, validation and normal export.
Tasks: Depth, Surface normals. Install: pip install libreyolo.
Verified against LibreYOLO v1.8.0.

## Install

MoGe-3 needs no optional extra. The sparse refiner is plain PyTorch and camera
recovery uses SciPy, which is in the base install.

```bash
pip install libreyolo
```

## Predict

Weights download from the LibreYOLO Hugging Face mirrors on first use.

**Python**

```python
from libreyolo import LibreYOLO, SAMPLE_IMAGE

model = LibreYOLO("LibreMoGe3l-depth.pt")
depth = model(SAMPLE_IMAGE).depth_map

print(depth.data.shape)        # (H, W) metres, inf outside the valid mask
print(depth.point_map.shape)   # (H, W, 3) camera-frame points in metres
print(depth.intrinsics)        # (3, 3) OpenCV pixel intrinsics
print(depth.valid_mask.shape)  # (H, W) bool
```

**Normals**

```python
from libreyolo import LibreYOLO, SAMPLE_IMAGE

model = LibreYOLO("LibreMoGe3l-normal.pt")
normal = model(SAMPLE_IMAGE).normal_map
print(normal.array.shape)      # (H, W, 3) unit vectors, OpenCV frame
```

**CLI**

```bash
libreyolo predict model=LibreMoGe3l-depth.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/guggenheim-bilbao.jpg save=True
```

With a `-depth` checkpoint, `result.depth_map` holds metric depth in metres
(`encoding="depth"`, `metric=True`). The point map, intrinsics and validity
mask are attached to it as `depth_map.point_map`, `depth_map.intrinsics` and
`depth_map.valid_mask`; pixels outside the mask are `inf`. Points use OpenCV
axes, and the intrinsics put pixel centres at integer coordinates. With a
`-normal` checkpoint, `result.normal_map` holds unit normals in the OpenCV
camera frame.

`imgsz` sets the ViT token budget, `(imgsz // 14) ** 2` tokens laid out to the
image aspect ratio. The default 840 is upstream's highest resolution level
(3600 tokens); 490 is close to its lowest. Inputs keep their native
resolution and every output is interpolated back to it. Inference runs in
FP32 and always applies three refinement steps, as upstream does by default.
One image runs per forward pass; there is no stacked-batch path.

CPU, CUDA and MPS all work without a compiled extension. The refiner's memory
grows with `imgsz`, not with the input resolution.

## Variants

| Weight file | Backbone | Task |
| --- | --- | --- |
| `LibreMoGe3l-depth.pt` | DINOv2 ViT-L/14 | depth, point map, intrinsics, mask |
| `LibreMoGe3l-normal.pt` | DINOv2 ViT-L/14 | normals |
| `LibreMoGe3g-depth.pt` | DINOv2 ViT-g/14 | depth, point map, intrinsics, mask |
| `LibreMoGe3g-normal.pt` | DINOv2 ViT-g/14 | normals |

The `-depth` and `-normal` files of one size hold the same tensors; the
suffix picks the task. Against upstream on an RTX 3090 in FP32, the
single-forward graph matches bit for bit and refined depth matches to
floating-point rounding (p99 relative difference 4.7e-6 for `l`, 3.7e-5 for
`g`).

MoGe-3 is not trainable in LibreYOLO: `train()` raises. Zero-shot `val()`
runs the shared depth and normal validators.

## Export

| Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | Axelera | ncnn | TFLite | CoreML | Core AI |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Depth |  |  |  |  |  |  |  |  |  |  |  |  |  |
| Surface normals | yes | yes |  |  |  |  |  |  |  |  |  |  |  |

A "yes" means the export is supported. An empty cell means the exporter refuses that combination.

Only the normal graph exports, to ONNX and TorchScript, on a fixed square
canvas. Depth export is blocked: the refiner builds data-dependent voxel sets
and camera recovery is a CPU solve.

**Python**

```python
from libreyolo import LibreYOLO

# Only the normal graph exports; depth export is blocked.
model = LibreYOLO("LibreMoGe3l-normal.pt")
model.export(format="onnx")
model.export(format="torchscript")
```

## Checkpoints

| File | Input (px) | Task | Weights license |
| --- | --- | --- | --- |
| `LibreMoGe3l-normal.pt` | 840 | Surface normals | mit |
| `LibreMoGe3l-depth.pt` |  | Depth | mit |
| `LibreMoGe3g-depth.pt` |  | Depth | mit |
| `LibreMoGe3g-normal.pt` | 840 | Surface normals | mit |

## Licensing

Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice.

- Original work: MoGe-3, Microsoft
- Upstream license: MIT
- Upstream source: https://github.com/microsoft/MoGe
- LibreYOLO code: MIT
- Weights: MIT, republished at https://huggingface.co/LibreYOLO
- Interpretation: MIT is a permissive license, so the MoGe-3 code and weights can be used in commercial and closed-source products; the only obligation is to keep the copyright and license notice with copies. LibreYOLO ports the forward pass, sparse refinement and camera recovery from microsoft/MoGe at commit 74fbce054ebed49800de42d0ad0e83495065719a (MIT, Copyright Microsoft Corporation), re-expresses the FlexGEMM sparse kernels in plain PyTorch without vendoring them (FlexGEMM is MIT), and follows MIT utils3d helpers; the DINOv2 backbone keeps its separate Apache-2.0 attribution (libreyolo/models/moge3/NOTICE). The official checkpoints Ruicheng/moge-3-vitl (revision 184008f877d7ad1ad4c2cd2182a9bd1f63d0e5be) and Ruicheng/moge-3-vitg (revision 6ef26c5a4b4148dab5ccaacdb08b72dc66380475) declare MIT; LibreYOLO mirrors them unchanged, plus checkpoint metadata, as LibreYOLO/LibreMoGe3{l,g}-{depth,normal}, not gated.

MoGe-3 is by Microsoft ([microsoft/MoGe](https://github.com/microsoft/MoGe)).
Code and weights are MIT, so commercial use is fine. The DINOv2 backbone keeps
its Apache-2.0 attribution. LibreYOLO mirrors the official
`Ruicheng/moge-3-vitl` and `Ruicheng/moge-3-vitg` checkpoints unchanged as
`LibreYOLO/LibreMoGe3{l,g}-{depth,normal}`, not gated.

## Citation

```bibtex
@misc{kong2026finedetailmonoculargeometryestimation,
      title={Fine-Detail Monocular Geometry Estimation with Self-Guided Sparse Volumetric Refinement},
      author={Lingyu Kong and Ruicheng Li and Ruicheng Wang and Sicheng Xu and Chengtang Yao and Jianfeng Xiang and Jiaolong Yang},
      year={2026},
      eprint={2607.17967},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.17967},
}
```

Copied from https://github.com/microsoft/MoGe#-citation
