Perception Encoder

Perception Encoder maps images, text and finite video clips into a shared embedding space.

任务: classification, embeddings
安装: pip install "libreyolo[clip]"
支持层级: 兄弟层级，自 v1.6.0 起。一个独立的产品面，有自己的工厂和契约。
上游: Perception Encoder，由 Meta AI 发布，采用 Apache-2.0 许可。论文、源码
许可: 代码采用 MIT，权重采用 Apache-2.0。商用

Install

bash

pip install "libreyolo[clip]"

Predict

Python

from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibrePEt16-cls.pt", device="cpu")model.set_classes(["person", "building"])result = model(SAMPLE_IMAGE)print(result.probs)

Set classes before zero-shot classification. Select task="embed" when loading for embeddings. Video embedding samples clip_frames=8 frames by default, averages their vectors and L2-normalizes the result. Training is not supported. Exported video graphs take direct runtime input.

Export

任务	ONNX	TorchScript	ExecuTorch	TensorRT	OpenVINO	Paddle	MNN	RKNN	ncnn	TFLite	CoreML	Core AI
Classification	Classification to ONNX：支持	Classification to TorchScript：支持	Classification to ExecuTorch：支持	Classification to TensorRT：支持	Classification to OpenVINO：支持	Classification to Paddle：不支持	Classification to MNN：不支持	Classification to RKNN：不支持	Classification to ncnn：不支持	Classification to TFLite：不支持	Classification to CoreML：不支持	Classification to Core AI：不支持
Embeddings	Embeddings to ONNX：支持	Embeddings to TorchScript：支持	Embeddings to ExecuTorch：支持	Embeddings to TensorRT：支持	Embeddings to OpenVINO：支持	Embeddings to Paddle：不支持	Embeddings to MNN：不支持	Embeddings to RKNN：不支持	Embeddings to ncnn：不支持	Embeddings to TFLite：不支持	Embeddings to CoreML：不支持	Embeddings to Core AI：不支持

Python

from libreyolo import LibreYOLO, SAMPLE_IMAGE model = LibreYOLO("LibrePEt16-cls.pt", device="cpu")model.export(format="onnx")

Export setup lists format dependencies and loading exported artifacts.

Checkpoints

文件	权重许可
Classification
LibrePEt16-cls.pt	apache-2.0
LibrePEs16-cls.pt	apache-2.0
LibrePEb16-cls.pt	apache-2.0
LibrePEl14-cls.pt	apache-2.0
LibrePEg14-cls.pt	apache-2.0

上面的每个文件目前都在 LibreYOLO 组织中，并会在首次使用时下载。

Licensing

请检查你所下载的具体权重在 Hugging Face 仓库中的许可。LibreYOLO 组织里的每个检查点都附有许可，同一家族内也不一定相同。该仓库是权威来源；以下摘要说明本页上次验证时适用的情况。

这里只说明涉及的许可证，不构成法律意见。如果答案对商用很重要，请自行阅读许可证并咨询法律顾问。

原始工作: Perception Encoder, Meta AI
上游许可: Apache-2.0
上游来源: github.com/facebookresearch/perception_models
LibreYOLO 代码: MIT
权重: 采用 Apache-2.0 许可，重新发布在 huggingface.co/LibreYOLO
解读: The publisher declares Apache-2.0 for the encoder weights.

Citation

@article{bolya2025PerceptionEncoder,
  title={Perception Encoder: The best visual embeddings are not at the output of the network},
  author={Daniel Bolya and Po-Yao Huang and Peize Sun and Jang Hyun Cho and Andrea Madotto and Chen Wei and Tengyu Ma and Jiale Zhi and Jathushan Rajasegaran and Hanoona Rasheed and Junke Wang and Marco Monteiro and Hu Xu and Shiyu Dong and Nikhila Ravi and Daniel Li and Piotr Doll{\'a}r and Christoph Feichtenhofer},
  journal={arXiv:2504.13181},
  year={2025}
}

复制自作者在 raw.githubusercontent.com/facebookresearch/perception_models/main/README.md 上提供的引用块。