> This is the documentation for LibreYOLO 1.6.0. Current version: https://www.libreyolo.com/docs/models/pe

# Perception Encoder
Perception Encoder maps images, text and finite video clips into a shared embedding space.
Tasks: Classification, Embeddings. Install: pip install "libreyolo[clip]".
Verified against LibreYOLO v1.6.0.

## Install

```bash
pip install "libreyolo[clip]"
```

## Predict

**Python**

```python
from libreyolo import LibreYOLO, SAMPLE_IMAGE

model = LibreYOLO("LibrePEt16-cls.pt", device="cpu")
model.set_classes(["person", "building"])
result = model(SAMPLE_IMAGE)
print(result.probs)
```

Set classes before zero-shot classification. Select `task="embed"` when loading for embeddings. Video embedding samples `clip_frames=8` frames by default, averages their vectors and L2-normalizes the result. Training is not supported. Exported video graphs take direct runtime input.

## Export

| Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Classification | yes | yes | yes | yes | yes |  |  |  |  |  |  |  |
| Embeddings | yes | yes | yes | yes | yes |  |  |  |  |  |  |  |

A "yes" means the export is supported. An empty cell means the exporter refuses that combination.

**Python**

```python
from libreyolo import LibreYOLO, SAMPLE_IMAGE

model = LibreYOLO("LibrePEt16-cls.pt", device="cpu")
model.export(format="onnx")
```

[Export setup](/docs/v1.6.0/export) lists format dependencies and loading exported artifacts.

## Checkpoints

| File | Input (px) | Task | Weights license |
| --- | --- | --- | --- |
| `LibrePEt16-cls.pt` |  | Classification | apache-2.0 |
| `LibrePEs16-cls.pt` |  | Classification | apache-2.0 |
| `LibrePEb16-cls.pt` |  | Classification | apache-2.0 |
| `LibrePEl14-cls.pt` |  | Classification | apache-2.0 |
| `LibrePEg14-cls.pt` |  | Classification | apache-2.0 |

## Licensing

Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice.

- Original work: Perception Encoder, Meta AI
- Upstream license: Apache-2.0
- Upstream source: https://github.com/facebookresearch/perception_models
- LibreYOLO code: MIT
- Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO
- Interpretation: The publisher declares Apache-2.0 for the encoder weights.

## Citation

```bibtex
@article{bolya2025PerceptionEncoder,
  title={Perception Encoder: The best visual embeddings are not at the output of the network},
  author={Daniel Bolya and Po-Yao Huang and Peize Sun and Jang Hyun Cho and Andrea Madotto and Chen Wei and Tengyu Ma and Jiale Zhi and Jathushan Rajasegaran and Hanoona Rasheed and Junke Wang and Marco Monteiro and Hu Xu and Shiyu Dong and Nikhila Ravi and Daniel Li and Piotr Doll{\'a}r and Christoph Feichtenhofer},
  journal={arXiv:2504.13181},
  year={2025}
}
```

Copied from https://raw.githubusercontent.com/facebookresearch/perception_models/main/README.md


## PE Core architecture

- PE Core t16 classify: [SVG](/diagrams/models/pe/t16-classify.svg), [interactive diagram](/diagrams/models/pe/t16-classify.html)
- PE Core t16 embed: [SVG](/diagrams/models/pe/t16-embed.svg), [interactive diagram](/diagrams/models/pe/t16-embed.html)
- PE Core s16 classify: [SVG](/diagrams/models/pe/s16-classify.svg), [interactive diagram](/diagrams/models/pe/s16-classify.html)
- PE Core s16 embed: [SVG](/diagrams/models/pe/s16-embed.svg), [interactive diagram](/diagrams/models/pe/s16-embed.html)
- PE Core b16 classify: [SVG](/diagrams/models/pe/b16-classify.svg), [interactive diagram](/diagrams/models/pe/b16-classify.html)
- PE Core b16 embed: [SVG](/diagrams/models/pe/b16-embed.svg), [interactive diagram](/diagrams/models/pe/b16-embed.html)
- PE Core l14 classify: [SVG](/diagrams/models/pe/l14-classify.svg), [interactive diagram](/diagrams/models/pe/l14-classify.html)
- PE Core l14 embed: [SVG](/diagrams/models/pe/l14-embed.svg), [interactive diagram](/diagrams/models/pe/l14-embed.html)
- PE Core g14 classify: [SVG](/diagrams/models/pe/g14-classify.svg), [interactive diagram](/diagrams/models/pe/g14-classify.html)
- PE Core g14 embed: [SVG](/diagrams/models/pe/g14-embed.svg), [interactive diagram](/diagrams/models/pe/g14-embed.html)
- PE Core family classify: [SVG](/diagrams/models/pe/family-classify.svg), [interactive diagram](/diagrams/models/pe/family-classify.html)
- PE Core family embed: [SVG](/diagrams/models/pe/family-embed.svg), [interactive diagram](/diagrams/models/pe/family-embed.html)