Validation and metrics

Run a validation

val() takes the dataset and returns the metrics.

from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt")metrics = model.val(data="coco8.yaml") print(metrics["metrics/mAP50-95"])print(metrics["metrics/mAP50"])print(metrics["speed/total_ms"])

libreyolo val model=LibreYOLO9s.pt data=coco8.yaml

from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt")metrics = model.val(data="coco8.yaml", split="train", batch=4) print(metrics)

The return value is a plain dict[str, float]. Every key is literal, so read it by name rather than by position.

The main arguments are data, split, batch, imgsz, conf, iou, workers, device, augment, save_json and verbose. conf defaults to 0.001 and iou to 0.6, both far looser than prediction defaults, because a mAP sweep needs the low-confidence tail. imgsz defaults to the model's own input size rather than a fixed number. split accepts val, test or train and nothing else.

Any other field of the validation config passes through as a keyword argument, including save_dir, max_det, eval_max_det, half, amp_dtype, cache and save_plots.

Metric keys per task

Detection returns the COCO family of numbers:

metrics/mAP50-95   metrics/mAP50    metrics/mAP75metrics/mAP_small  metrics/mAP_medium  metrics/mAP_largemetrics/AR1  metrics/AR10  metrics/AR100  metrics/AR_max_detmetrics/AR_small  metrics/AR_medium  metrics/AR_largemetrics/precision  metrics/recallmetrics/precision(B)  metrics/recall(B)  metrics/mAP50(B)  metrics/mAP50-95(B)

Two of those are traps. metrics/precision and metrics/recall are aliases held for backward compatibility: they carry the mAP 50-95 and AR@100 values, not a precision and recall pair. Use the named keys.

Instance segmentation returns the mAP and AR figures above as mask numbers under the unsuffixed keys, with the box versions under a (B) suffix and the mask versions repeated under (M). Precision and recall exist only in suffixed form for this task, as metrics/precision(B)/metrics/recall(B) and metrics/precision(M)/metrics/recall(M), and both pairs carry the same alias values as detect's: the (B) pair is box mAP50-95 and box AR@100, the (M) pair is mask mAP50-95 and mask AR@100.

OBB's metrics/precision and metrics/recall are not aliases: they are the real precision and recall at IoU 0.50, taken at the loosest operating point (every prediction that survives conf, default 0.001). The (OBB)-suffixed copies repeat the same four values under a task-specific name, the same convention as (B) and (M) above.

accuracy_top5 is really top-min(5, num_classes), so on a three-class dataset it is top-3, which every sample satisfies and which therefore reads 1.0.

The point task's sweep key is built from the distance thresholds, so with the defaults it reads metrics/mAP@[0.01:0.10] and the single-threshold key reads metrics/mAP@0.01. Passing dist_thresholds changes both strings.

Most tasks also return a fitness key, the single number best-checkpoint selection uses by default. Detection, segmentation and OBB do not carry one; their families are selected on metrics/mAP50-95, which their dicts do return. Pose returns neither fitness nor metrics/mAP50-95; its trainers set best_metric_key to metrics/keypoints_mAP50-95 instead.

Speed keys

Every validator adds timing:

speed/preprocess_ms   speed/inference_ms   speed/postprocess_msspeed/total_ms        speed/total_s        speed/images_seen

These are per-image milliseconds averaged over the run. They describe the machine and settings you ran on, so a figure taken from them is only meaningful reported with its hardware, batch size and precision.

Evaluation backend

Detection and segmentation metrics are computed through a COCO evaluator, and faster_coco_eval=True, the default, selects the C++ backend when the faster-coco-eval package is installed. When it is not, the run falls back to pycocotools with one warning per process:

faster_coco_eval requested but not installed; falling back to pycocotools.Install with: pip install faster-coco-eval

Which backend actually ran is recorded on the model as last_eval_backend, and the CLI reports it in its output for detection-style tasks. Set LIBREYOLO_FASTER_COCO_EVAL to override the config value from the environment.

iou_thresholds is honored only on the OBB path. The COCO path evaluates through its own fixed 0.50 to 0.95 sweep and ignores the value.

Validation loss

By default validation reports accuracy only. val_loss=True also computes the family's training objective on validation batches.

from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt")model.train(data="coco8.yaml", epochs=10, val_loss=True)

It emits metrics/loss plus one metrics/loss/<component> per term, weighted exactly as training weights them, so the components sum to the total. Through a logger they appear as val/loss and val/loss/<component>, and libreyolo monitor overlays metrics/loss with train/loss.

The components are the family's own:

It is off by default because target assignment adds time and memory to validation. The validator reuses the model output already produced for the accuracy metric rather than running a second forward pass, it runs under no_grad on the evaluation or EMA model, and under multi-GPU training it is computed locally on rank 0 with no collectives. Best-checkpoint selection stays on the accuracy metric.

Three things it deliberately does not do. It never includes contrastive-denoising terms, because those need the ground truth at forward time and validation forwards without it. It reports the evaluation-mode model, so where a family's train and eval forwards genuinely differ, in BatchNorm statistics or stochastic depth, the number reflects eval mode; that is the intended comparison. And a task a family has not implemented it for raises a configuration error at setup rather than quietly skipping:

val_loss=True currently supports RF-DETR detection only; segment, pose, OBB,classify, and semantic tasks are not supported

FOMO is the exception that changes nothing: its validator always computed this loss, and val_loss=True only affects which keys it is published under.

Augmented validation and validation loss cannot be combined, and asking for both raises.

Files a validation writes

val() always writes config.yaml into its save directory, defaulting to runs/val/<model>_<size>_<timestamp> when save_dir is not given.

from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt")model.val(data="coco8.yaml", save_json=True, save_dir="runs/val/exp")

save_json=True writes predictions.json for detection, and predictions_bbox.json plus predictions_masks.json for segmentation. OBB does not support it and says so.

save_plots=True writes into a plots/ subdirectory. Detection gets box_metrics.png, per-class AP and recall charts, precision-recall and confidence curves, a confusion matrix, and annotated sample images when OpenCV is installed. Segmentation adds the mask-side copies of each, and pose gets its own metric and curve set. The other validators do not implement plots; classification, semantic, panoptic, depth, normal, edge, restore, matte, OCR, OBB and point all write nothing there. A plotting failure warns and never aborts the run.

Validation during training

Training validates every eval_interval epochs against the dataset's val split, and the metrics it produces are what drives best.pt selection, the patience early stop, and the val/ keys in every logger. The validation runs on the EMA weights when EMA is on.

See Hyperparameters for eval_interval, patience and save_plots, and Experiment loggers for where the numbers go.

Related

• Datasets for the split keys and formats validators read.