> This is the documentation for LibreYOLO 1.6.0. Current version: https://www.libreyolo.com/docs/train/datasets # Datasets A LibreYOLO dataset is a YAML file naming a root, its splits and its class names. Everything else, including where the label files live, is derived from that file by convention. Verified against LibreYOLO v1.6.0. ## Point train at a dataset `data=` takes a YAML path or the name of a config that ships with the package. **Python** ```python from libreyolo import LibreYOLO model = LibreYOLO("LibreYOLO9s.pt") # A bundled name, a relative path or an absolute path all work. model.train(data="coco8.yaml", epochs=10) ``` **CLI** ```bash libreyolo train model=LibreYOLO9s.pt data=coco8.yaml epochs=10 ``` The name is resolved in a fixed order: an absolute path that exists, then the name as given relative to the working directory, then the same name with `.yaml` appended, then the bundled config directory. When nothing matches, the error names every directory that was searched and lists the bundled configs. ## Bundled configs Thirteen dataset configs ship inside the package, under `libreyolo/config/datasets/`. | Config | Task | Notes | |---|---|---| | `coco8.yaml` | detect | 8 images, downloads from a plain URL | | `coco128.yaml` | detect | 128 images | | `coco1000.yaml` | detect | 800 train, 200 val | | `coco5000.yaml` | detect | 4000 train, 1000 val | | `coco.yaml` | detect | full COCO 2017 | | `coco-val-only.yaml` | detect | val2017 only | | `coco8-pose.yaml` | pose | 8 images, COCO-17 keypoints | | `coco-pose.yaml` | pose | COCO 2017 keypoints | | `ade20k.yaml` | semantic | 150 classes | | `cityscapes.yaml` | semantic | 19 classes, download by hand | | `cocostuff.yaml` | semantic | 182 classes, download by hand | | `gopro.yaml` | restore | deblurring pairs | | `sr8.yaml` | restore | super-resolution pairs | Only `coco8.yaml` and `coco128.yaml` carry a plain download URL. The rest either carry a Python download block, which needs the opt-in described below, or expect the data to already be on disk. ## Where a dataset lives on disk The YAML `path` key names the dataset root. An absolute `path` is used as written. A relative one is looked for under the datasets directory first, then beside the YAML file itself, and a dataset that is about to be downloaded goes under the datasets directory. That directory is `~/datasets`, overridden by the `LIBREYOLO_DATASETS_DIR` environment variable. There is no settings file for it. ## The YAML keys ```yaml path: my-dataset # dataset root train: images/train # required to train val: images/val # required to validate test: images/test # optional nc: 3 # optional; must agree with names names: 0: person 1: helmet 2: vest download: https://example.com/my-dataset.zip # optional ``` `train`, `val` and `test` each accept an image directory, a `.txt` file listing one image path per line, or a list mixing both. Lines in a `.txt` list may be relative, in which case they resolve against the list file's own directory, and lines starting with `#` are skipped. `names` may be a list or an integer-keyed mapping. `nc` is optional; when both are present and disagree, the doctor reports it as an error. RF-DETR pose reads `kpt_names` keyed by class ID or name. It keeps the first named keypoint rows per class; an empty list denotes a box-only class. Multi-class pose requires `names` and at least one keypoint-bearing class. ## Directory layout and label files Detection, segmentation, pose and oriented boxes all share one layout. The label path is derived from the image path by rewriting an `images` directory component to `labels` and changing the extension to `.txt`: ```text my-dataset/ images/train/0001.jpg -> labels/train/0001.txt images/val/0002.jpg -> labels/val/0002.txt ``` Only a whole `images` path component is rewritten, so a directory named `images_old` is left alone. A detection row is five fields, all normalized to `[0, 1]` against the original image width and height: ```text ``` A missing or empty label file means the image has no objects, and it trains as background rather than raising. A row with more than five fields is read as a polygon and its box becomes the polygon's extent, so a segmentation export used for detection training loads without complaint. The doctor reports how many rows took that path. Finite boxes crossing the image border are clipped consistently for training and validation. Boxes with no visible area, non-finite coordinates and malformed polygons are dropped. Out-of-range class IDs are reported before target construction. `train(classes=[...])` filters supervision using original class IDs; see [hyperparameters](/docs/v1.6.0/train/hyperparameters). ## Other tasks Segmentation keeps the same layout with polygon rows, ` ... `, at least three points. A five-field detection row is accepted and means a rectangular instance. Pose adds `kpt_shape: [K, D]` and an optional `flip_idx` permutation to the YAML. Each row is exactly `5 + K * D` fields: the box, then `K` keypoints of `x y` or `x y v`, with visibility `0`, `1` or `2`. Oriented boxes use exactly nine fields, the class followed by four corner points in normalized coordinates. No angle is stored in the file. Semantic segmentation pairs each image with a single-channel mask of the same resolution, resolved by substituting `masks_dir` (default `masks`) for `images`. Pixel value `255` means ignore. `label_mapping` remaps source ids to train ids at load time. Classification uses an ImageFolder tree instead of label files, with `train/` and `val/` each containing one directory per class. The class-to-index mapping is the sorted folder name order. Restoration pairs a degraded input with a clean target of identical resolution through `input_dir` and `target_dir`. Depth, surface normals and edges each pair an image with a dense map through their own directory key. The full per-task contract, including the depth scale conventions and the panoptic segment-id PNG encoding, is `docs/dataset_schema.md` in the library repository. ## Native COCO JSON A COCO JSON annotation file can be used directly. Add an `annotations` mapping, and the split path becomes the image root: ```yaml path: my-dataset train: images/train val: images/val annotations: train: annotations/train.json val: annotations/val.json ``` When `names` is present, the JSON category names must match it, and `names` defines the label ids the model predicts. Without `names`, COCO category ids are sorted and mapped densely to `0..N-1`. This path expects one image directory per split. A list of paths or a `.txt` image list raises rather than silently loading a different set. ## Autodownload A dataset counts as present when its `train` or `val` path resolves to a non-empty directory or an existing file. When it does not, and the YAML has a `download` key, the value decides what happens next. An `http` or `https` URL is fetched and, if it is a zip, extracted into the dataset root. Anything else is treated as an embedded Python script and runs only when `allow_download_scripts=True`. Without that, the script is skipped with a warning and training continues against whatever is on disk. ```bash libreyolo train model=LibreYOLO9s.pt data=coco.yaml allow_download_scripts=true ``` The flag is a code-execution gate, not a network gate. URL downloads happen either way; it is the `download: |` blocks that need it. The CLI prints a warning when the flag is on, and the doctor never enables it. ## Check the dataset before you train `libreyolo doctor` reads a detection dataset and reports what would go wrong before a GPU is involved. It exits 1 when it finds errors, so it works as a CI gate. **Check a dataset** ```bash libreyolo doctor my-dataset.yaml ``` **Fail a CI job on warnings too** ```bash libreyolo doctor my-dataset.yaml strict=true json=true ``` **Skip the image decode pass** ```bash # Reads labels and YAML only. Corruption, duplicate and split-leakage # checks all need the pixels, so they are skipped. libreyolo doctor my-dataset.yaml fast=true ``` **Python** ```python from libreyolo import doctor report = doctor.diagnose("my-dataset.yaml", imgsz=640) for finding in report.findings: print(finding.severity.value, finding.check_id, finding.message) raise SystemExit(report.exit_code(strict=False)) ``` The checks come in six families: | Family | Looks for | |---|---| | `config` | missing `names`, `nc` that disagrees with `names`, missing or empty splits, duplicate class names | | `files` | images with no label file, labels with no image, missing images listed in a split, stem collisions | | `labels` | malformed rows, class ids outside `[0, nc)`, coordinates outside `[0, 1]`, zero-area boxes, tiny or huge boxes, duplicate boxes, byte-identical label files | | `balance` | classes with zero or few instances, class imbalance ratio, classes present in one split only, background image share | | `images` | undecodable files, EXIF rotation, odd channel layouts, uniform images, exact and near duplicates | | `splits` | the same image appearing in two splits, exactly or near-identically | `--only` and `--skip` take a check id or a family prefix, so `skip=images,labels.tiny_object` is valid. `--fast` drops every check that needs to decode pixels, which is the `images` and `splits` families. Two behaviors are worth knowing. `--strict` makes warnings fail the exit code as well as errors. And the doctor covers detection datasets only: a pose, segment or oriented-box dataset is rejected with a message naming what it detected, rather than being checked against the wrong contract. ## Related - [Hyperparameters](/docs/v1.6.0/train/hyperparameters) for the arguments `train()` takes once the data is in place. - [Validation and metrics](/docs/v1.6.0/train/validation) for evaluating on the `val` or `test` split.