> This is the documentation for LibreYOLO 1.5.0. Current version: https://www.libreyolo.com/docs/models/segformer

# SegFormer
SegFormer is a semantic segmentation transformer that pairs a hierarchical Mix Transformer (MiT) encoder with a lightweight all-MLP decode head, avoiding the heavy decoders and fixed positional encodings earlier segmentation transformers needed. LibreYOLO supports it for one task, semantic segmentation, across six sizes.
Tasks: semantic. Install: pip install libreyolo.
Verified against LibreYOLO v1.5.0.

## Install

SegFormer needs no optional extra. Everything it imports is in the base install.

```bash
pip install libreyolo
```

## Predict

Weights download from Hugging Face on first use and are cached locally.

**Python**

```python
from libreyolo import LibreYOLO, SAMPLE_IMAGE

model = LibreYOLO("LibreSegformerb0-sem.pt")
result = model(SAMPLE_IMAGE, save=True)

mask = result.semantic_mask
print(mask.data.shape, mask.classes)
```

**CLI**

```bash
libreyolo predict model=LibreSegformerb0-sem.pt source=https://raw.githubusercontent.com/LibreYOLO/libreyolo/release/libreyolo/assets/parkour.jpg save=True
```

`result.semantic_mask` carries the dense class map: `.data` is an `(H, W)`
tensor of class IDs on the original image size, and `.classes` lists the class
IDs actually present. `result.boxes` is `None`, since there are no
per-instance detections. `conf` and `iou` are accepted for API parity but do
not change the output: the model returns one class per pixel, not per-instance
detections to filter or de-duplicate. See [prediction](/docs/v1.5.0/predict) for
sources, streaming and result handling.

## Variants

Six sizes, b0 through b5, widening and deepening the Mix Transformer encoder
at each step while keeping the same all-MLP decode head design.

| File | Input (px) | Task | Weights license |
| --- | --- | --- | --- |
| `LibreSegformerb0-sem.pt` |  | semantic | other |
| `LibreSegformerb1-sem.pt` |  | semantic | other |
| `LibreSegformerb2-sem.pt` |  | semantic | other |
| `LibreSegformerb3-sem.pt` |  | semantic | other |
| `LibreSegformerb4-sem.pt` |  | semantic | other |
| `LibreSegformerb5-sem.pt` |  | semantic | other |

## Train

`train()` fine-tunes a published checkpoint by default. Pass no `model_path`
to `LibreSegformer(...)` instead and it builds with a randomly initialized
encoder and head, training from scratch, the only route to weights that carry
none of the pretrained checkpoints' non-commercial restriction (see
[Licensing](#licensing)).

**Python (fine-tune)**

```python
from libreyolo import LibreYOLO

model = LibreYOLO("LibreSegformerb0-sem.pt")
model.train(data="my-dataset.yaml", epochs=160, imgsz=512, batch=8)
```

**CLI**

```bash
libreyolo train model=LibreSegformerb0-sem.pt data=my-dataset.yaml \
  epochs=160 imgsz=512 batch=8
```

**From scratch**

```python
from libreyolo.models.segformer.model import LibreSegformer

# No model_path: random init, nothing downloaded. The only route to
# weights free of the pretrained checkpoints' non-commercial term.
model = LibreSegformer(size="b0", nb_classes=150)
model.train(data="my-dataset.yaml", epochs=160, imgsz=512, batch=8)
```

**Multi-GPU**

```bash
libreyolo train model=LibreSegformerb0-sem.pt data=my-dataset.yaml \
  epochs=160 device=0,1 batch=16
```

Left alone, the trainer follows the SegFormer paper's ADE20K recipe: AdamW at
a backbone base learning rate with the decode head trained at 10x that rate,
weight decay everywhere except LayerNorm and the Mix-FFN positional
convolution, and a linear decay schedule with a warmup. Convergence for the
larger sizes, b3 through b5, has not been validated end to end.

See [training](/docs/v1.5.0/train) for datasets, augmentation, multi-GPU and loggers.

## Validate

`val()` returns a dictionary of `metrics/` keys: mIoU and pixel accuracy,
measured against any dataset in the format you trained on.

**Python**

```python
from libreyolo import LibreYOLO

model = LibreYOLO("LibreSegformerb0-sem.pt")
metrics = model.val(data="my-dataset.yaml")

print(metrics["metrics/mIoU"])
print(metrics["metrics/pixel_accuracy"])
```

**CLI**

```bash
libreyolo val model=LibreSegformerb0-sem.pt data=my-dataset.yaml
```

## Export

| Task | ONNX | TorchScript | ExecuTorch | TensorRT | OpenVINO | Paddle | MNN | RKNN | ncnn | TFLite | CoreML | Core AI |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| semantic | yes | yes | yes | yes | yes |  |  |  |  |  |  |  |

A "yes" means the export is supported. An empty cell means the exporter refuses that combination.

An exported artifact loads back through `LibreYOLO()` on its file suffix, so a
`.onnx` or `.engine` file behaves like a checkpoint and returns the same
`Results`. [Export](/docs/v1.5.0/export) lists the arguments every format accepts.

**Python**

```python
from libreyolo import LibreYOLO

model = LibreYOLO("LibreSegformerb0-sem.pt")
model.export(format="onnx", imgsz=512)
model.export(format="tensorrt", imgsz=512, half=True)
```

**CLI**

```bash
libreyolo export model=LibreSegformerb0-sem.pt format=onnx imgsz=512
libreyolo export model=LibreSegformerb0-sem.pt format=tensorrt imgsz=512 half=True
```

**Use the exported file**

```python
from libreyolo import LibreYOLO, SAMPLE_IMAGE

# The factory routes on the file suffix, so an exported artifact loads
# like any checkpoint and returns the same Results object.
model = LibreYOLO("LibreSegformerb0-sem.onnx")
result = model(SAMPLE_IMAGE)

print(result.semantic_mask.data.shape)
```

## Checkpoints

Every published weight file for this family.

| File | Input (px) | Task | Weights license |
| --- | --- | --- | --- |
| `LibreSegformerb0-sem.pt` |  | semantic | other |
| `LibreSegformerb1-sem.pt` |  | semantic | other |
| `LibreSegformerb2-sem.pt` |  | semantic | other |
| `LibreSegformerb3-sem.pt` |  | semantic | other |
| `LibreSegformerb4-sem.pt` |  | semantic | other |
| `LibreSegformerb5-sem.pt` |  | semantic | other |

## Licensing

Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice.

- Original work: SegFormer, NVIDIA
- Upstream license: NVIDIA Source Code License (non-commercial, research or evaluation only)
- Upstream source: https://github.com/NVlabs/SegFormer
- LibreYOLO code: Apache-2.0
- Weights: NVIDIA Source Code License (non-commercial, research or evaluation only), republished at https://huggingface.co/LibreYOLO
- Interpretation: The pretrained ADE20K checkpoints LibreYOLO hosts for this family are converted from NVIDIA's official SegFormer release under the NVIDIA Source Code License. That license permits redistributing the weights and derivative works, provided the license text and attribution notices travel with them, but it limits USE to non-commercial research or evaluation, a restriction its Section 3.2 carries forward into every derivative and that cannot be relicensed away: these weights are NOT covered by LibreYOLO's normal permissive terms, and that limitation binds you, not just LibreYOLO. LibreYOLO's own SegFormer implementation is a separate Apache-2.0 port of Hugging Face Transformers' code, unrelated to NVIDIA's repository, so a model you train from scratch with LibreSegformer(...).train(...) carries none of this restriction.

LibreSegformer's encoder and decode head are a PyTorch port of Hugging Face
Transformers' Apache-2.0 SegFormer implementation, not of NVlabs/SegFormer:
NVIDIA's original repository was never read or copied, and is credited here
only for attribution to the paper's authors. Only the pretrained checkpoints
above carry NVIDIA's non-commercial restriction; the architecture and
LibreYOLO's own code stay MIT throughout.

## Citation

```bibtex
@inproceedings{xie2021segformer,
  title={SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers},
  author={Xie, Enze and Wang, Wenhai and Yu, Zhiding and Anandkumar, Anima and Alvarez, Jose M and Luo, Ping},
  booktitle={Neural Information Processing Systems (NeurIPS)},
  year={2021}
}
```

Copied from https://github.com/NVlabs/SegFormer#citation


## SegFormer architecture

- b0: [SVG](/diagrams/models/segformer/b0-semantic.svg), [interactive diagram](/diagrams/models/segformer/b0-semantic.html)
- b1: [SVG](/diagrams/models/segformer/b1-semantic.svg), [interactive diagram](/diagrams/models/segformer/b1-semantic.html)
- b2: [SVG](/diagrams/models/segformer/b2-semantic.svg), [interactive diagram](/diagrams/models/segformer/b2-semantic.html)
- b3: [SVG](/diagrams/models/segformer/b3-semantic.svg), [interactive diagram](/diagrams/models/segformer/b3-semantic.html)
- b4: [SVG](/diagrams/models/segformer/b4-semantic.svg), [interactive diagram](/diagrams/models/segformer/b4-semantic.html)
- b5: [SVG](/diagrams/models/segformer/b5-semantic.svg), [interactive diagram](/diagrams/models/segformer/b5-semantic.html)
- Shared B0-B5 topology: [SVG](/diagrams/models/segformer/family.svg), [interactive diagram](/diagrams/models/segformer/family.html)