RF100-VL is a benchmark of 100 real-world object detection datasets collected from Roboflow Universe, spanning seven domains: aerial, document, flora and fauna, industrial, medical, sports, and a misc category. It was introduced by Roboflow to measure how well detectors generalize beyond COCO. It is a dataset benchmark, not a model: any detector can be evaluated on it.

What is the RF100-VL protocol in this report?

For each model family we fine-tune the COCO-pretrained checkpoint for 100 epochs on each dataset's train split, then score that dataset's test split with pycocotools at maxDets 500. We report the unweighted mean of mAP50 and mAP50-95 across the 100 datasets, plus the median training time per dataset. A run only counts as a result when all 100 datasets completed.

Which models have completed the RF100-VL sweep so far?

Two: YOLOv9-S at 0.559 mAP50-95 and 0.814 mAP50, and YOLOv9-T at 0.540 mAP50-95 and 0.796 mAP50. Both trained and scored all 100 datasets with none skipped, at a median of roughly 33 minutes per dataset. The other families are still running and are not listed until they finish a full 100.

Where can I check the RF100-VL numbers myself?

Every artifact is published at huggingface.co/datasets/LibreYOLO/rf100-vl-results: the per-dataset training configs, per-epoch metrics, logs, GPU telemetry, scoring inputs and the submission JSON. Each run carries a manifest.json pinning the exact LibreYOLO and harness commits that produced it.

RF100-VL: benchmarking detectors on one hundred real datasets

COCO has 80 classes and a decade of overfitting behind it. If you want to know whether a detector actually works in the real world, you need a harder test. That test is RF100-VL: 100 datasets pulled from Roboflow Universe, crowdsourced from real projects, and grouped into seven domains: aerial, document, flora and fauna, industrial, medical, sports, and everything else.

We are running it across 15 detection families in LibreYOLO: one representative model per family, fine-tuned for 100 epochs on each dataset's train split, then evaluated on its test split. 15 families x 100 datasets means 1,500 training runs. This article is the living report.

The 100 datasets

Chest x-rays next to conveyor belts. Coral reefs next to circuit boards. Aerial sheep counting next to invoice parsing. The spread is the point: a model that wins here did not memorize a domain.

Every planet orbiting at the top of this page is one of the 100 datasets: the thumbnail is a real annotated sample, the orbit is its domain. Hover a planet (or a domain in the legend) to explore. The rings are sized by domain: industrial and flora/fauna carry the most datasets, sports the fewest.

A few things that make RF100-VL genuinely hard:

Tiny training sets. Many datasets have a few hundred images. Transfer learning quality matters more than architecture tricks.
Dense scenes. Some datasets average over 100 objects per image (circuit elements reach ~255), which stresses query-based detectors and GPU memory alike.
Wild class imbalance and odd aspect ratios. Drone footage, thermal cameras, microscopy, documents, screenshots.
No room for per-dataset tuning. With 100 datasets you pick one recipe per family and live with it.

The protocol

For each family we take the COCO-pretrained checkpoint at the smallest practical size (S for most, R18 for RT-DETR, T for YOLOv9 E2E) and fine-tune 100 epochs at 640 px (576 for RF-DETR). Then model.val(split="test"), scored with pycocotools at maxDets 500. We track three numbers per family: mAP50, mAP50-95, and the median wall-clock training time per dataset, because accuracy you cannot afford to train is not accuracy.

One rule governs what appears below: a family is listed only after it has trained and scored all 100 datasets from a clean state under one set of commits. A partial sweep is a debugging artifact, not a result, so it is not shown here at any confidence level.

Verified runs

The first two complete campaigns

Fine-tuned for 100 epochs on each of the 100 datasets, then scored on each dataset's test split with pycocotools at maxDets 500. The reported figure is the unweighted mean across the 100 scores.

100

datasets

models complete

200

runs, none skipped

180 h

GPU time

Modelparamsok/100mAP⁵⁰mAP^50-95median train

YOLOv9SmAP^50-95 0.559 · ok 100/100 · ~32.5 min7.2M1000.8140.559~32.5 min

YOLOv9TmAP^50-95 0.540 · ok 100/100 · ~32.9 min2.0M1000.7960.540~32.9 min

Both runs are complete: 100 of 100 datasets trained and scored, none skipped, all protocol-conformant. Single seed, 640 px, one RTX 5060 Ti. Raw artifacts, per-epoch metrics and the manifest pinning the exact commit are published at LibreYOLO/rf100-vl-results. The remaining families are still running and will be added as each one finishes a full 100.

Reading the results

Two families have finished a full 100 so far, and both are YOLOv9. That is not a ranking yet, so read it as two reference points rather than a podium.

Capacity buys less than you would guess. YOLOv9-S carries 3.6 times the parameters of YOLOv9-T and converts that into 0.019 mAP50-95, from 0.540 to 0.559. On 100 diverse datasets, architecture and scale move the needle by hundredths, not percentage points.
Training cost is flat between them. Both sit near 33 minutes per dataset in the median, so the larger model is close to free here. Fine-tuning on a few hundred images is dominated by fixed per-dataset overhead, not by model size.
The spread across datasets dwarfs the spread across models. The gap between the two models is far smaller than the gap between the easiest and hardest datasets for either one. Which datasets resemble your problem matters more than which of these two you pick.

Every number above is traceable to a published run, and a family appears only once it has trained and scored all 100 datasets. The full per-dataset breakdown, failure analysis and training curves land in the final report; until then this page grows as each sweep finishes.