TripoSR

TripoSR is a feed-forward image-to-3D model by Tripo AI and Stability AI. It turns one picture of an object into a triplane NeRF in a single pass and returns the object as a vertex-coloured triangle mesh. LibreYOLO runs it through the LibreGen factory in plain PyTorch.

Tareas
3d generation
Instalación
pip install "libreyolo[gen]"
Nivel de compatibilidad
Nivel hermano, desde v1.8.0. Una superficie de producto independiente con su propia factoría y su propio contrato.
Proyecto original
TripoSR de Tripo AI and Stability AI, MIT. Artículo, fuente
Licencias
Código MIT and Apache-2.0, pesos MIT. Uso comercial

Install

TripoSR runs on the LibreGen extra, which brings scikit-image for marching cubes.

bash
pip install "libreyolo[gen]"

Predict

Weights download on first use from LibreYOLO/LibreTripoSR, with nothing gated and no login.

Python
from libreyolo import LibreGen, SAMPLE_IMAGE model = LibreGen("triposr")        # weights download on first useresult = model(SAMPLE_IMAGE) result.save("object.obj")          # vertex colours as `v x y z r g b`m = result.meshesprint(m.vertices[0].shape, m.faces[0].shape)   # (V, 3), (F, 3)
CLI
# Saves the object as .obj under runs/generate/predict.libreyolo generate source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/guggenheim-bilbao.jpg model=triposr
Settings
from libreyolo import LibreGen, SAMPLE_IMAGE model = LibreGen("triposr")result = model(SAMPLE_IMAGE, resolution=384)    # finer marching-cubes gridresult = model(SAMPLE_IMAGE, threshold=25.0)    # density at the surface

A DINO ViT-B/16 encodes the 512 px input, a transformer decodes a triplane NeRF from it in one pass, and the mesh is the marching-cubes isosurface of the NeRF's density. There is no sampling, so seed is accepted like every LibreGen family but changes nothing. It takes about a second on a GPU and runs in fp32 on NVIDIA GPUs, Apple silicon and CPU.

The source is anything predict takes: a path, URL, PIL image, NumPy array, a folder or a list. An image with real transparency is used as the cutout; anything else has its background removed first with LibreYOLO's BiRefNet port.

result.meshes holds one mesh per object: vertices (V, 3), z-up inside [-0.5, 0.5]^3, faces (F, 3), and extras["vertex_colors"] (V, 3) RGB in [0, 1]. The input view looks along -x, so the front of the object faces +x. save() writes .obj with vertex colours.

Settings

SettingDefaultMeaning
resolution256Marching-cubes grid, from 32 to 1024.
threshold25.0Density at the surface.
margin1/0.85Crop side over object size (upstream's foreground ratio 0.85).

Marching cubes runs through scikit-image instead of upstream's torchmcubes: the same isosurface, triangulated differently.

Variants

One checkpoint, 0.42B parameters, about 1.7 GB plus 0.4 GB for BiRefNet.

Export

TareaONNXTorchScriptExecuTorchTensorRTOpenVINOPaddleMNNRKNNAxelerancnnTFLiteCoreMLCore AI
3D generation3D generation to ONNX: no compatible3D generation to TorchScript: no compatible3D generation to ExecuTorch: no compatible3D generation to TensorRT: no compatible3D generation to OpenVINO: no compatible3D generation to Paddle: no compatible3D generation to MNN: no compatible3D generation to RKNN: no compatible3D generation to Axelera: no compatible3D generation to ncnn: no compatible3D generation to TFLite: no compatible3D generation to CoreML: no compatible3D generation to Core AI: no compatible

Export is not implemented for LibreGen 3D families. export(), train() and val() raise NotImplementedError; their names are reserved by ADR 0031 in the library.

Licensing

Comprueba la licencia en el repositorio de Hugging Face de los pesos concretos que descargues. Cada checkpoint de la organización de LibreYOLO incluye una, y no siempre es la misma en toda una familia. Ese repositorio es la fuente autorizada; el resumen siguiente describe qué se aplicaba cuando se verificó esta página por última vez.

Esta es una descripción de las licencias implicadas, no asesoramiento legal. Si la respuesta es importante a nivel comercial, lee las licencias y busca tu propio asesoramiento.

Trabajo original
TripoSR, Tripo AI and Stability AI
Licencia del proyecto original
MIT
Fuente del proyecto original
github.com/VAST-AI-Research/TripoSR
Código de LibreYOLO
MIT
Pesos
MIT, republicados en huggingface.co/LibreYOLO
Interpretación
MIT is a permissive license, so the code and weights can be used in commercial and closed-source products; the only obligation is to keep the copyright and license notice with copies. LibreYOLO's adapter is MIT; the triplane, NeRF and mesh-extraction code it adapts from TripoSR keeps MIT, and the transformer and DINO ViT encoder follow HuggingFace diffusers and transformers code under Apache-2.0, with their notices in libreyolo/models/gen/triposr/NOTICE. LibreYOLO avoids upstream's torchmcubes (MPL-2.0) by running marching cubes through scikit-image (BSD-3-Clause). The stabilityai/TripoSR weights (MIT) are redistributed unchanged as LibreYOLO/LibreTripoSR; Background removal runs on LibreYOLO's BiRefNet port with the ZhengPeng7/BiRefNet weights (MIT), from LibreYOLO/LibreTRELLISl. Both are pinned and not gated.

TripoSR is by Tripo AI and Stability AI. The code (VAST-AI-Research/TripoSR) and the stabilityai/TripoSR weights are MIT, so commercial use is fine with the copyright notice kept. LibreYOLO redistributes the weights unchanged as LibreYOLO/LibreTripoSR. It avoids upstream's torchmcubes (MPL-2.0) by using scikit-image (BSD-3-Clause); the BiRefNet background remover weights are MIT.

Citation

@article{TripoSR2024,
  title={TripoSR: Fast 3D Object Reconstruction from a Single Image},
  author={Tochilkin, Dmitry and Pankratz, David and Liu, Zexiang and Huang, Zixuan and and Letts, Adam and Li, Yangguang and Liang, Ding and Laforte, Christian and Jampani, Varun and Cao, Yan-Pei},
  journal={arXiv preprint arXiv:2403.02151},
  year={2024}
}

Copiado del bloque de cita de los autores en github.com/VAST-AI-Research/TripoSR#citation.

Verificado con LibreYOLO v1.8.0. Las tablas de compatibilidad, los checkpoints y las cifras de rendimiento de esta página se generan a partir de la biblioteca publicada y los pesos publicados; no se escriben a mano.