# SigLIP2
SigLIP2 is a dual-tower model that scores an image against text prompts with an independent sigmoid per class, instead of a shared softmax over a fixed label set. LibreYOLO supports it for zero-shot classification and image/text embedding, with no training step.
Tasks: Classification, Embeddings. Install: pip install "libreyolo[siglip2]".
Verified against LibreYOLO v1.5.0.

## Install

SigLIP2 needs its own extra, which pulls in the SentencePiece package its multilingual tokenizer uses.

```bash
pip install "libreyolo[siglip2]"
```

## Predict

Weights download from Hugging Face on first use and are cached locally.

**Python**

```python
from libreyolo import LibreYOLO, SAMPLE_IMAGE

model = LibreYOLO("LibreSigLIP2b16-cls.pt")
model.set_classes(["a forklift", "an empty aisle", "a spill"])
result = model(SAMPLE_IMAGE, save=True)

print(model.names[result.probs.top1], float(result.probs.top1conf))
```

**CLI**

```bash
# With no set_classes() call, CLI predict uses the 1,000 ImageNet
# class names the model loads with by default.
libreyolo predict model=LibreSigLIP2b16-cls.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True
```

**Multi-label sigmoid scoring**

```python
from libreyolo import LibreYOLO, SAMPLE_IMAGE

model = LibreYOLO("LibreSigLIP2b16-cls.pt")
model.set_classes(["a dog", "a cat", "outdoors"], multi_label=True)
r = model(SAMPLE_IMAGE)

# Independent per-class probabilities: more than one, or none, can
# score high at once. Softmax (the default) instead normalizes them
# into a single-label distribution, matching LibreCLIP's behavior.
for i, name in model.names.items():
    print(name, float(r.probs.data[i]))
```

**Image and text embedding**

```python
from libreyolo import LibreYOLO, SAMPLE_IMAGE

model = LibreYOLO("LibreSigLIP2b16-cls.pt", task="embed")
image_embed = model(SAMPLE_IMAGE).embeddings.data
text_embed = model.embed_text("a photo of a forklift")

# Both are L2-normalized, so a plain dot product is cosine similarity.
similarity = (image_embed @ text_embed.T).item()
```

`set_classes()` is the one primitive that makes this an open-vocabulary classifier: it renders each label into every prompt template, encodes and averages the results, and caches the resulting `[K, D]` matrix as the classifier head, so it is not recomputed per image. Call it again to change classes at any time. With no call, LibreSigLIP2 loads with the 1,000 ImageNet-1k class names already set.

SigLIP scores each class independently: `logit = scale * (image . text) + bias`. By default that logit set is still passed through a softmax, giving a single-label distribution that matches LibreCLIP's `top1`/`top5` behavior. Passing `multi_label=True` to `set_classes()` (or at construction) switches to independent sigmoid probabilities instead, so more than one class, or none, can score high on the same image. The tokenizer is a multilingual SentencePiece model (Gemma vocabulary), so class names in languages other than English work the same way.

With `task="embed"`, prediction returns one L2-normalized image vector per input instead of class probabilities, and `embed_text()` returns normalized text rows in the same vector space, so a plain dot product between them is cosine similarity. `iou` has no effect on either task; there is no NMS step. See [prediction](/docs/predict) for sources, streaming and result handling.

## Validate

`val()` reads the class-folder names under an ImageFolder `train/` split, calls `set_classes()` with them, then measures zero-shot top-1 and top-5 accuracy under softmax scoring. Accuracy depends on how the class names read as prompts, not on any weight update, since there is nothing to train. Validation only covers `task="classify"`; `task="embed"` has no dataset validator.

**Python**

```python
from libreyolo import LibreYOLO

model = LibreYOLO("LibreSigLIP2b16-cls.pt")

# data is an ImageFolder root with a train/ split; its folder names
# become the zero-shot class prompts for this run.
metrics = model.val(data="imagenette160")

print(metrics["metrics/accuracy_top1"])
print(metrics["metrics/accuracy_top5"])
```

**CLI**

```bash
libreyolo val model=LibreSigLIP2b16-cls.pt data=imagenette160
```

## Export

| Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Classification | yes | yes | yes | yes | yes |  |  |  |  | yes |  | yes |
| Embeddings | yes | yes | yes | yes | yes |  |  |  |  | yes |  |  |

A "yes" means the export is supported. An empty cell means the exporter refuses that combination.

Export bakes the model's current state into a fixed graph. For `task="classify"`, whatever labels `set_classes()` last set, and the resolution at export time, are baked into a final linear layer with the learned scale and bias, so the exported graph is an ordinary `[B, K]` image classifier with no text tower and no tokenizer; export again after changing the classes or the size. Exporting in `multi_label=True` mode is not implemented; set it back to `False` first. `task="embed"` export traces the image tower alone. Both need ONNX opset 14 or higher, which the exporter sets by default.

**Python**

```python
from libreyolo import LibreYOLO

model = LibreYOLO("LibreSigLIP2b16-cls.pt")
model.set_classes(["a forklift", "an empty aisle", "a spill"])
model.export(format="onnx")

# The current set_classes() labels and the input resolution are baked
# into the graph. Re-export after changing either one. multi_label
# must be False (the default) at export time.
```

**CLI**

```bash
# No set_classes() call here, so this bakes in the default 1,000
# ImageNet classes the model loads with.
libreyolo export model=LibreSigLIP2b16-cls.pt format=onnx
```

**Embedding export**

```python
from libreyolo import LibreYOLO

# task="embed" traces the image tower alone; no classes needed.
model = LibreYOLO("LibreSigLIP2b16-cls.pt", task="embed")
model.export(format="onnx")
```

## Checkpoints

Every published weight file for this family. Both are converted from Google's Apache-2.0 `siglip2-base-patch16-256` and `siglip2-so400m-patch14-384` checkpoints, not from any COCO training run.

| File | Input (px) | Task | Weights license |
| --- | --- | --- | --- |
| `LibreSigLIP2b16-cls.pt` |  | Classification | apache-2.0 |
| `LibreSigLIP2so400m-cls.pt` |  | Classification | apache-2.0 |

## Licensing

Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice.

- Original work: SigLIP 2, Google DeepMind
- Upstream license: Apache-2.0
- Upstream source: https://github.com/huggingface/transformers
- LibreYOLO code: MIT
- Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO
- Interpretation: Apache-2.0 is a permissive license, so these weights can be used in commercial and closed-source products. It asks you to keep its license text and attribution notices with any copy of the weights you redistribute, and it grants a patent license. It places no obligation on your own application code. LibreSigLIP2's image and text towers are a clean-room re-implementation structured to match Hugging Face Transformers' SigLIP reference implementation, built without transformers as a runtime dependency; the multilingual SentencePiece tokenizer (Gemma vocabulary) is shipped verbatim from the Apache-2.0 google/siglip2-* release. The shipped checkpoints (b16, so400m) are converted from Google's Apache-2.0 siglip2-base-patch16-256 and siglip2-so400m-patch14-384 weights, a metadata wrap with the learned parameters unchanged. Training is not offered for this family: LibreSigLIP2 is zero-shot, and set_classes() replaces the fine-tuning step a trained classifier would otherwise need.

## Citation

```bibtex
@misc{tschannen2025siglip2multilingualvisionlanguage,
      title={SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features}, 
      author={Michael Tschannen and Alexey Gritsenko and Xiao Wang and Muhammad Ferjad Naeem and Ibrahim Alabdulmohsin and Nikhil Parthasarathy and Talfan Evans and Lucas Beyer and Ye Xia and Basil Mustafa and Olivier Hénaff and Jeremiah Harmsen and Andreas Steiner and Xiaohua Zhai},
      year={2025},
      eprint={2502.14786},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2502.14786}, 
}
```

Copied from https://huggingface.co/google/siglip2-base-patch16-256#bibtex-entry-and-citation-info


## SigLIP2 architecture

- SigLIP2 b16 classify: [SVG](/diagrams/models/siglip2/b16-classify.svg), [interactive diagram](/diagrams/models/siglip2/b16-classify.html)
- SigLIP2 b16 embed: [SVG](/diagrams/models/siglip2/b16-embed.svg), [interactive diagram](/diagrams/models/siglip2/b16-embed.html)
- SigLIP2 so400m classify: [SVG](/diagrams/models/siglip2/so400m-classify.svg), [interactive diagram](/diagrams/models/siglip2/so400m-classify.html)
- SigLIP2 so400m embed: [SVG](/diagrams/models/siglip2/so400m-embed.svg), [interactive diagram](/diagrams/models/siglip2/so400m-embed.html)
- SigLIP2 family classify: [SVG](/diagrams/models/siglip2/family-classify.svg), [interactive diagram](/diagrams/models/siglip2/family-classify.html)
- SigLIP2 family embed: [SVG](/diagrams/models/siglip2/family-embed.svg), [interactive diagram](/diagrams/models/siglip2/family-embed.html)