# SenseNova-Vision
SenseNova-Vision is a unified multimodal model that casts vision tasks as prompted generation on a shared decoder: boxes, points, keypoints and OCR words come out as tagged text, and depth, mask and panoptic maps come out as images a decoder renders. LibreYOLO loads it through LibreVLM and supports seven tasks from the one 7B checkpoint.
Tasks: Detection, Instance segmentation, Panoptic segmentation, Pose, Point detection, Depth, OCR. Install: pip install "libreyolo[sensenova]".
Verified against LibreYOLO v1.5.0.

## Install

SenseNova-Vision needs its own extra, which pulls in `accelerate` for the big-model dispatch this checkpoint needs and, on non-macOS platforms, `bitsandbytes` for 4-bit loading.

```bash
pip install "libreyolo[sensenova]"
```

The checkpoint is mirrored on Hugging Face under LibreYOLO's own org and downloads automatically on first use; it is CC BY-NC 4.0, non-commercial use only, and the loader prints that notice before every automatic download. See Licensing below.

## Predict

**Python**

```python
from libreyolo import LibreVLM

model = LibreVLM("sensenova-vision", task="detect")
model.set_classes(["bird", "boat"])
result = model.predict("image.jpg")
print(result.boxes.xyxy)

# set_task() switches tasks on the same loaded model.
model.set_task("depth")
result = model.predict("image.jpg")
depth = result.depth_map.data
```

**Referring segmentation and panoptic**

```python
from libreyolo import LibreVLM

model = LibreVLM("sensenova-vision", task="segment")
# Segmentation is referring: it needs a target phrase, not a class list.
model.set_classes(["the person furthest to the right"])
result = model.predict("street.jpg")
mask = result.masks.data[0]

model.set_task("panoptic")
# With no custom vocabulary, panoptic falls back to the COCO panoptic
# categories the checkpoint was tuned on.
result = model.predict("street.jpg")
segment_map = result.panoptic.data
for segment in result.panoptic.segments_info:
    print(segment)
```

**Points, pose and OCR**

```python
from libreyolo import LibreVLM

model = LibreVLM("sensenova-vision", task="point")
model.set_classes(["screw"])
result = model.predict("board.jpg")
print(result.points.xy)

# With no vocabulary set, pose falls back to "person".
model.set_task("pose")
result = model.predict("gym.jpg")
print(result.boxes.xyxy, result.keypoints.data.shape)

model.set_task("ocr")
result = model.predict("sign.jpg")
print(result.ocr.texts)
```

Every prediction is a diffusion decode over the shared Bagel-MoT backbone, so it is a capability model rather than a real-time one: expect noticeably higher per-image latency than a purpose-built detector or segmenter. `dtype="auto"` (the default) loads bf16 on a GPU with enough memory and falls back to 4-bit NF4 quantization elsewhere, which needs `bitsandbytes`; pass `dtype="bf16"` to force full precision on a large enough GPU. `noise_seed=42` at construction seeds the diffusion sampler for reproducible dense outputs; pass `noise_seed=None` to disable seeding.

The seven tasks share one loaded checkpoint: `set_task()` switches between them without reloading. `set_classes()` sets the active vocabulary; detection, points, pose and panoptic accept a class list, while segmentation is referring and needs exactly the phrase to isolate. Each task returns the standard `Results` object with a different payload populated: `boxes` for detect, `points` for point, `boxes` and `keypoints` for pose, `ocr` for OCR, `depth_map` for depth, `masks` for segment, and `panoptic` (with `segments_info`) for panoptic. See [prediction](/docs/predict) for sources, streaming and result handling.

## Checkpoints



## Licensing

Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice.

- Original work: SenseNova-Vision, SenseTime (OpenSenseNova)
- Upstream license: Apache-2.0 (code); CC BY-NC 4.0 (weights)
- Upstream source: https://github.com/OpenSenseNova/SenseNova-Vision
- LibreYOLO code: Apache-2.0
- Weights: Apache-2.0 (code); CC BY-NC 4.0 (weights), republished at https://huggingface.co/LibreYOLO
- Interpretation: LibreYOLO's SenseNova-Vision port is Apache-2.0 code adapted from SenseTime's OpenSenseNova/SenseNova-Vision release, which is itself built on Apache-2.0 sources: ByteDance's Bagel decoder, Hugging Face Transformers' Qwen2 and SigLIP modules, and Black Forest Labs' FLUX autoencoder. The SenseNova-Vision-7B-MoT checkpoint is a separate artifact under CC BY-NC 4.0, NON-COMMERCIAL USE ONLY. LibreYOLO mirrors it byte-identical, with attribution, at LibreYOLO/SenseNovaVision7b, but mirroring does not change the license: it stays non-commercial, and the loader prints that notice before every automatic download. One upstream file, modeling/bagel/modeling_utils.py, carries an incompatible CC BY-NC 4.0 license inherited from Meta's DiT; LibreYOLO did not port it. The small permissive routines it would have supplied were re-derived independently instead, from Hugging Face Transformers' ViTMAE implementation (Apache-2.0) and OpenAI's guided-diffusion (MIT).

## Citation

```bibtex
@misc{sensenova2026sensenovavision,
      title={Vision as Unified Multimodal Generation}, 
      author={Xiaoyang Han and Jianhua Li and Kewang Deng and Zukai Chen and Xuanke Shi and Sihan Wang and Boxuan Li and Linyan Wang and Siyi Xie and Xin You and Jinsheng Quan and Zhongang Cai and Haiwen Diao and Ziwei Liu and Lei Yang and Dahua Lin and Quan Wang},
      year={2026},
      eprint={2607.06560},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.06560}, 
}
```

Copied from https://github.com/OpenSenseNova/SenseNova-Vision#citation