# SAM
SAM (Segment Anything) turns a point or box click into an object mask. LibreYOLO loads it through a dedicated LibreSAM factory, separate from the LibreYOLO() detector factory, because a promptable model needs a different call shape.
Tasks: Instance segmentation. Install: pip install "libreyolo[sam]".
Verified against LibreYOLO v1.6.0.

## Install

SAM needs the `sam` extra, which pulls in `transformers` and `timm`.

```bash
pip install "libreyolo[sam]"
```

## Predict

`LibreSAM(...)` is a separate entry point from `LibreYOLO(...)`: it returns a
promptable segmenter rather than a detector, because a forward pass here is
meaningless without a spatial prompt. There is no `libreyolo predict` CLI
command for this family; use the Python API.

**Point and box prompts**

```python
from libreyolo import LibreSAM, SAMPLE_IMAGE

# "base" autodownloads facebook/sam-vit-base on first use.
# Other sizes: "large", "huge" (also "b"/"l"/"h").
model = LibreSAM("base")

# A point prompt: [x, y] in pixel coordinates, label 1 = foreground.
result = model.predict(SAMPLE_IMAGE, points=[640, 420], labels=[1])
print(result.masks.xy)      # polygon per mask
print(result.boxes.xyxy)    # tight box derived from the mask

# A box prompt instead of a point.
result = model.predict(SAMPLE_IMAGE, bboxes=[300, 200, 900, 700])

# No prompt at all segments the whole image (a simplified automatic
# mask generator, not the exhaustive reference one).
result = model.predict(SAMPLE_IMAGE)
```

**Encode once, prompt many**

```python
from libreyolo import LibreSAM, SAMPLE_IMAGE

model = LibreSAM("base")

# The image encoder is the expensive part. set_image() runs it once;
# every predict() call after that reuses the cached embedding.
model.set_image(SAMPLE_IMAGE)
a = model.predict(points=[640, 420], labels=[1])
b = model.predict(bboxes=[300, 200, 900, 700])
model.reset_image()
```

A point prompt accepts `[x, y]` for one object, `[[x, y], ...]` for several, or
numpy arrays; `labels` marks each point `1` (foreground) or `0` (background)
and defaults to all foreground. A box prompt takes `[x1, y1, x2, y2]` or a list
of boxes, one mask per box. Omitting both prompts segments the whole image by
prompting a dense grid and keeping the confident, non-overlapping masks; this
"segment everything" mode is simplified against the reference automatic mask
generator and can under-segment crowded scenes, so a real point or box prompt
is the precise path. `conf` filters by predicted mask quality (IoU), not a
detection confidence: pass `0.0` to keep every candidate. `multimask=True`
returns all three of SAM's whole-versus-part ambiguity masks per prompt
instead of the single best one. `device=` moves the model and, if a
`set_image()` session is active, its cached embedding. Every mask carries
class id `0`, named `"object"`, since a promptable mask has no fixed class
set. `train()`, `val()`, `export()` and `track()` all raise
`NotImplementedError` for this family: SAM is predict-only in LibreYOLO, and
video tracking is out of scope. See [prediction](/docs/predict) for source
types.

## Variants

Three ViT image-encoder sizes: base, large and huge, all at a fixed 1024 px
input. No accuracy or latency benchmark is published for this family yet, so
choosing a size trades encoder weight for mask quality directly: base is the
fastest to encode, huge the heaviest.

## Licensing

Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice.

- Original work: SAM (Segment Anything), Meta AI Research (FAIR)
- Upstream license: Apache-2.0
- Upstream source: https://github.com/facebookresearch/segment-anything
- LibreYOLO code: MIT
- Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO
- Interpretation: SAM-1 base, large and huge use LibreYOLO mirrors of the publisher artifacts under Apache-2.0.

`LibreSAM("base")`, `"large"` and `"huge"` use LibreYOLO mirrors of the SAM-1 artifacts. The weights retain Meta's Apache-2.0 declaration.

## Citation

```bibtex
@article{kirillov2023segany,
  title={Segment Anything},
  author={Kirillov, Alexander and Mintun, Eric and Ravi, Nikhila and Mao, Hanzi and Rolland, Chloe and Gustafson, Laura and Xiao, Tete and Whitehead, Spencer and Berg, Alexander C. and Lo, Wan-Yen and Doll{\'a}r, Piotr and Girshick, Ross},
  journal={arXiv:2304.02643},
  year={2023}
}
```

Copied from https://github.com/facebookresearch/segment-anything#citing-segment-anything


## SAM architecture

- SAM base: [SVG](/diagrams/models/sam/base-box-segment.svg), [interactive diagram](/diagrams/models/sam/base-box-segment.html)
- SAM large: [SVG](/diagrams/models/sam/large-box-segment.svg), [interactive diagram](/diagrams/models/sam/large-box-segment.html)
- SAM huge: [SVG](/diagrams/models/sam/huge-box-segment.svg), [interactive diagram](/diagrams/models/sam/huge-box-segment.html)
- SAM family: [SVG](/diagrams/models/sam/family-box-segment.svg), [interactive diagram](/diagrams/models/sam/family-box-segment.html)