TripoSR
TripoSR is a feed-forward image-to-3D model by Tripo AI and Stability AI. It turns one picture of an object into a triplane NeRF in a single pass and returns the object as a vertex-coloured triangle mesh. LibreYOLO runs it through the LibreGen factory in plain PyTorch.
- Задачи
- 3d generation
- Установка
pip install "libreyolo[gen]"- Уровень поддержки
- Смежный уровень, начиная с v1.8.0. Отдельная часть продукта со своей фабрикой и контрактом.
- Исходный проект
- TripoSR от Tripo AI and Stability AI, MIT. Статья, исходный код
- Лицензии
- Код: MIT and Apache-2.0, веса: MIT. Коммерческое использование
Install
TripoSR runs on the LibreGen extra, which brings scikit-image for marching cubes.
pip install "libreyolo[gen]"Predict
Weights download on first use from LibreYOLO/LibreTripoSR, with nothing gated and no login.
from libreyolo import LibreGen, SAMPLE_IMAGE model = LibreGen("triposr") # weights download on first useresult = model(SAMPLE_IMAGE) result.save("object.obj") # vertex colours as `v x y z r g b`m = result.meshesprint(m.vertices[0].shape, m.faces[0].shape) # (V, 3), (F, 3)# Saves the object as .obj under runs/generate/predict.libreyolo generate source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/guggenheim-bilbao.jpg model=triposrfrom libreyolo import LibreGen, SAMPLE_IMAGE model = LibreGen("triposr")result = model(SAMPLE_IMAGE, resolution=384) # finer marching-cubes gridresult = model(SAMPLE_IMAGE, threshold=25.0) # density at the surfaceA DINO ViT-B/16 encodes the 512 px input, a transformer decodes a triplane
NeRF from it in one pass, and the mesh is the marching-cubes isosurface of the
NeRF's density. There is no sampling, so seed is accepted like every
LibreGen family but changes nothing. It takes about a second on a GPU and runs
in fp32 on NVIDIA GPUs, Apple silicon and CPU.
The source is anything predict takes: a path, URL, PIL image, NumPy array, a folder or a list. An image with real transparency is used as the cutout; anything else has its background removed first with LibreYOLO's BiRefNet port.
result.meshes holds one mesh per object: vertices (V, 3), z-up inside
[-0.5, 0.5]^3, faces (F, 3), and extras["vertex_colors"] (V, 3) RGB in
[0, 1]. The input view looks along -x, so the front of the object faces +x.
save() writes .obj with vertex colours.
Settings
| Setting | Default | Meaning |
|---|---|---|
resolution | 256 | Marching-cubes grid, from 32 to 1024. |
threshold | 25.0 | Density at the surface. |
margin | 1/0.85 | Crop side over object size (upstream's foreground ratio 0.85). |
Marching cubes runs through scikit-image instead of upstream's torchmcubes: the same isosurface, triangulated differently.
Variants
One checkpoint, 0.42B parameters, about 1.7 GB plus 0.4 GB for BiRefNet.
Export
| Задача | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | Axelera | ncnn | TFLite | CoreML | Core AI |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 3D generation | 3D generation to ONNX: не поддерживается | 3D generation to TorchScript: не поддерживается | 3D generation to ExecuTorch: не поддерживается | 3D generation to TensorRT: не поддерживается | 3D generation to OpenVINO: не поддерживается | 3D generation to Paddle: не поддерживается | 3D generation to MNN: не поддерживается | 3D generation to RKNN: не поддерживается | 3D generation to Axelera: не поддерживается | 3D generation to ncnn: не поддерживается | 3D generation to TFLite: не поддерживается | 3D generation to CoreML: не поддерживается | 3D generation to Core AI: не поддерживается |
Export is not implemented for LibreGen 3D families. export(), train() and
val() raise NotImplementedError; their names are reserved by ADR 0031 in
the library.
Licensing
Проверяйте лицензию в репозитории конкретных весов на Hugging Face. Она указана для каждого чекпойнта в организации LibreYOLO и может различаться даже внутри одного семейства. Этот репозиторий считается авторитетным источником, а сводка ниже описывает условия на момент последней проверки страницы.
Это описание соответствующих лицензий, а не юридическая консультация. Если ответ важен для коммерческого использования, прочитайте лицензии самостоятельно и проконсультируйтесь с юристом.
- Оригинальная работа
- TripoSR, Tripo AI and Stability AI
- Лицензия исходного проекта
- MIT
- Исходный код проекта
- github.com/VAST-AI-Research/TripoSR
- Код LibreYOLO
- MIT
- Веса
- MIT, повторно опубликованы на huggingface.co/LibreYOLO
- Толкование
- MIT is a permissive license, so the code and weights can be used in commercial and closed-source products; the only obligation is to keep the copyright and license notice with copies. LibreYOLO's adapter is MIT; the triplane, NeRF and mesh-extraction code it adapts from TripoSR keeps MIT, and the transformer and DINO ViT encoder follow HuggingFace diffusers and transformers code under Apache-2.0, with their notices in libreyolo/models/gen/triposr/NOTICE. LibreYOLO avoids upstream's torchmcubes (MPL-2.0) by running marching cubes through scikit-image (BSD-3-Clause). The stabilityai/TripoSR weights (MIT) are redistributed unchanged as LibreYOLO/LibreTripoSR; Background removal runs on LibreYOLO's BiRefNet port with the ZhengPeng7/BiRefNet weights (MIT), from LibreYOLO/LibreTRELLISl. Both are pinned and not gated.
TripoSR is by Tripo AI and Stability AI. The code
(VAST-AI-Research/TripoSR) and
the stabilityai/TripoSR weights are MIT, so commercial use is fine with the
copyright notice kept. LibreYOLO redistributes the weights unchanged as
LibreYOLO/LibreTripoSR. It avoids upstream's torchmcubes (MPL-2.0) by using
scikit-image (BSD-3-Clause); the BiRefNet background remover weights are MIT.
Citation
@article{TripoSR2024,
title={TripoSR: Fast 3D Object Reconstruction from a Single Image},
author={Tochilkin, Dmitry and Pankratz, David and Liu, Zexiang and Huang, Zixuan and and Letts, Adam and Li, Yangguang and Liang, Ding and Laforte, Christian and Jampani, Varun and Cao, Yan-Pei},
journal={arXiv preprint arXiv:2403.02151},
year={2024}
}Скопировано из блока цитирования авторов на странице github.com/VAST-AI-Research/TripoSR#citation.