SAM

Install

SAM needs the sam extra, which pulls in transformers and timm.

pip install "libreyolo[sam]"

Predict

LibreSAM(...) is a separate entry point from LibreYOLO(...): it returns a promptable segmenter rather than a detector, because a forward pass here is meaningless without a spatial prompt. There is no libreyolo predict CLI command for this family; use the Python API.

from libreyolo import LibreSAM, SAMPLE_IMAGE # "base" autodownloads facebook/sam-vit-base on first use.# Other sizes: "large", "huge" (also "b"/"l"/"h").model = LibreSAM("base") # A point prompt: [x, y] in pixel coordinates, label 1 = foreground.result = model.predict(SAMPLE_IMAGE, points=[640, 420], labels=[1])print(result.masks.xy)      # polygon per maskprint(result.boxes.xyxy)    # tight box derived from the mask # A box prompt instead of a point.result = model.predict(SAMPLE_IMAGE, bboxes=[300, 200, 900, 700]) # No prompt at all segments the whole image (a simplified automatic# mask generator, not the exhaustive reference one).result = model.predict(SAMPLE_IMAGE)

from libreyolo import LibreSAM, SAMPLE_IMAGE model = LibreSAM("base") # The image encoder is the expensive part. set_image() runs it once;# every predict() call after that reuses the cached embedding.model.set_image(SAMPLE_IMAGE)a = model.predict(points=[640, 420], labels=[1])b = model.predict(bboxes=[300, 200, 900, 700])model.reset_image()

A point prompt accepts [x, y] for one object, [[x, y], ...] for several, or numpy arrays; labels marks each point 1 (foreground) or 0 (background) and defaults to all foreground. A box prompt takes [x1, y1, x2, y2] or a list of boxes, one mask per box. Omitting both prompts segments the whole image by prompting a dense grid and keeping the confident, non-overlapping masks; this "segment everything" mode is simplified against the reference automatic mask generator and can under-segment crowded scenes, so a real point or box prompt is the precise path. conf filters by predicted mask quality (IoU), not a detection confidence: pass 0.0 to keep every candidate. multimask=True returns all three of SAM's whole-versus-part ambiguity masks per prompt instead of the single best one. device= moves the model and, if a set_image() session is active, its cached embedding. Every mask carries class id 0, named "object", since a promptable mask has no fixed class set. train(), val(), export() and track() all raise NotImplementedError for this family: SAM is predict-only in LibreYOLO, and video tracking is out of scope. See prediction for source types.

Variants

Three ViT image-encoder sizes: base, large and huge, all at a fixed 1024 px input. No accuracy or latency benchmark is published for this family yet, so choosing a size trades encoder weight for mask quality directly: base is the fastest to encode, huge the heaviest.

Licensing

Citation

@article{kirillov2023segany,
  title={Segment Anything},
  author={Kirillov, Alexander and Mintun, Eric and Ravi, Nikhila and Mao, Hanzi and Rolland, Chloe and Gustafson, Laura and Xiao, Tete and Whitehead, Spencer and Berg, Alexander C. and Lo, Wan-Yen and Doll{\'a}r, Piotr and Girshick, Ross},
  journal={arXiv:2304.02643},
  year={2023}
}