# Florence-2
Florence-2 is Microsoft's vision foundation model, prompted with a task token instead of run through a fixed detection head. LibreYOLO wraps it as an open-vocabulary object detector: supply the class list at predict time.
Tasks: Detection. Install: pip install "libreyolo[vlm]".
Verified against LibreYOLO v1.5.0.

## Install

Florence-2 belongs to LibreYOLO's VLM-as-detector tier, a separate product
surface from the checkpoint-based families with its own factory. It needs the
`vlm` extra.

```bash
pip install "libreyolo[vlm]"
```

## Predict

Weights download from Hugging Face on first use and are cached locally.
LibreYOLO downloads the florence-community re-upload of the checkpoint rather
than the original `microsoft/Florence-2-*` repository; see Licensing for why.

**Python**

```python
from libreyolo import LibreVLM, SAMPLE_IMAGE

model = LibreVLM("florence-2-base")
model.set_classes(["car", "person", "traffic light"])
result = model.predict(SAMPLE_IMAGE, save=True)

for box in result.boxes:
    print(box.cls, box.conf, box.xyxy)
```

**Video**

```python
from libreyolo import LibreVLM

model = LibreVLM("florence-2-base")
model.set_classes(["car", "person", "traffic light"])

# Any source the library accepts: file, folder, URL, webcam index,
# RTSP stream, or a .streams list
for result in model.predict("clip.mp4", stream=True, save=True):
    print(len(result.boxes))
```

This family loads through the `LibreVLM()` factory, not `LibreYOLO()`: VLM
families declare no checkpoint loader, so the file-suffix routing described on
other model pages does not apply here. `set_classes()` sets the vocabulary
Florence-2 is asked to find in the image; it is sticky, so it stays in effect
across every later `predict()`/`track()` call until you set it again. The
returned `Results` carries `boxes` in the same shape as any other family, but
every detection carries the same placeholder confidence, so `conf` filtering is
all-or-nothing rather than a ranking, and `iou` has no effect: Florence-2's
wrapper builds the detection list directly from the parsed task-token output,
with no deduplication step. `chat()` raises `NotImplementedError` here, because
Florence-2 is driven by the `<OPEN_VOCABULARY_DETECTION>` task token rather than
a chat template. LibreYOLO's CLI does not cover this tier: there is no
`libreyolo predict model=...` form for it. See [prediction](/docs/predict) for
sources, streaming and result handling.

## Variants

Two sizes: Florence-2-base and Florence-2-large, both at 768 px, loaded as
`LibreVLM("florence-2-base")` or `LibreVLM("florence-2-large")`. LibreYOLO has
not published a benchmark comparing accuracy between them.

LibreYOLO does not train, validate or export Florence-2: `train()`, `val()` and
`export()` all raise `NotImplementedError` for every family in this tier (see
the support tier above). Fine-tune Florence-2 upstream and load the resulting
weights if you need a custom vocabulary baked in; check `predict()` output by
eye instead of a COCO-style validation pass, since every detection carries the
same placeholder confidence.

## Licensing

Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice.

- Original work: Florence-2, Microsoft
- Upstream license: MIT
- Upstream source: https://huggingface.co/microsoft/Florence-2-large
- LibreYOLO code: MIT
- Weights: MIT, distributed by their authors. LibreYOLO does not host or mirror them.
- Interpretation: MIT is a permissive license, so these weights can be used in commercial and closed-source products, provided the copyright notice and license text travel with any copy you redistribute. LibreYOLO downloads the florence-community re-upload of the checkpoint (florence-community/Florence-2-base and florence-community/Florence-2-large) rather than the original microsoft/Florence-2-* repositories, because those ship with custom remote code that no longer loads on current transformers releases. florence-community republishes the same weights through the native Florence2ForConditionalGeneration class, also under MIT, so nothing about the license changes.

## Citation

```bibtex
@article{xiao2023florence,
  title={Florence-2: Advancing a unified representation for a variety of vision tasks},
  author={Xiao, Bin and Wu, Haiping and Xu, Weijian and Dai, Xiyang and Hu, Houdong and Lu, Yumao and Zeng, Michael and Liu, Ce and Yuan, Lu},
  journal={arXiv preprint arXiv:2311.06242},
  year={2023}
}
```

Copied from https://huggingface.co/microsoft/Florence-2-large#bibtex


## Florence-2 architecture

- Florence-2 base: [SVG](/diagrams/models/florence-2/base-detect.svg), [interactive diagram](/diagrams/models/florence-2/base-detect.html)
- Florence-2 large: [SVG](/diagrams/models/florence-2/large-detect.svg), [interactive diagram](/diagrams/models/florence-2/large-detect.html)
- Florence-2 family: [SVG](/diagrams/models/florence-2/family.svg), [interactive diagram](/diagrams/models/florence-2/family.html)