# Falcon OCR
Falcon OCR is a 0.27B early-fusion document OCR model by the Technology Innovation Institute (TII). One transformer reads image patches and text together and writes the page as text, with LaTeX for formulas and HTML for tables. LibreYOLO runs a PyTorch port of Falcon OCR 1.5 on the ocr task.
Tasks: OCR. Install: pip install "libreyolo[vlm]".
Verified against LibreYOLO v1.8.0.

## Install

Falcon OCR belongs to LibreYOLO's VLM tier and needs the `vlm` extra.

```bash
pip install "libreyolo[vlm]"
```

## Predict

Weights download on first use from `LibreYOLO/LibreFalconOCR270m`, a
byte-identical mirror of `tiiuae/Falcon-OCR` pinned to a revision. The release
runs on FlexAttention and a Triton kernel through remote code; LibreYOLO runs a
plain PyTorch port with neither.

**Python**

```python
from libreyolo import LibreVLM

model = LibreVLM("falcon-ocr")         # weights download on first use
result = model.predict("page.png")      # a scanned or rendered page

print(result.ocr.markdown)              # the page text
```

**With layout**

```python
from libreyolo import LibreVLM

# PP-DocLayoutV3 finds regions in reading order; each crop is read
# with its category prompt.
model = LibreVLM("falcon-ocr", layout=True)
result = model.predict("page.png")

for label, text in zip(result.ocr.labels, result.ocr.texts):
    print(label, text[:60])
```

**One element**

```python
from libreyolo import LibreVLM

model = LibreVLM("falcon-ocr", prompt="table")
result = model.predict("table.png")     # a cropped table
print(result.ocr.markdown)              # HTML table
```

The family loads through `LibreVLM()`, or the `LibreFalconOCR` class. There
are two modes, as upstream:

- Default: the whole page is read in one pass. `Results.ocr` holds one region
  covering the image, label `plain`, and `Results.ocr.markdown` is the same
  text.
- `layout=True`: LibreYOLO's [PP-DocLayoutV3](/docs/models/pp-doclayoutv3) port
  finds regions in reading order at `layout_threshold=0.3`, nested boxes are
  dropped, and each crop is read with its category prompt. One region per crop,
  `labels` holds the layout class, `det_conf` the layout score, and `markdown`
  joins the texts with blank lines. Pictures and classes without an upstream
  prompt category are skipped.

`prompt=` takes a category (`plain`, `text`, `table`, `formula`, `caption`,
`footnote`, `list-item`, `page-footer`, `page-header`, `section-header`,
`title`) or a custom instruction. Pages are resized so both sides fall between
64 and 1536 px. Decoding is greedy with a budget of 8192 new tokens. A
generative model emits no score, so `conf` is a constant 1.0.

## Variants

One size, `270m`. CUDA runs the bf16 weights under `torch.autocast`; CPU and
MPS run float32, about 1.1 GB of weights. Batches of one. On CPU in float32 the
port reproduces the release token for token on six OCR fixtures and a
synthetic report page; CUDA bf16 and the layout mode were not compared against
upstream.

## Validate

`val()` runs the shared OCR validator.

**Python**

```python
from libreyolo import LibreVLM

model = LibreVLM("falcon-ocr")
metrics = model.val(data="ocr_dataset/")
```

## Export

Inference only: `train()` and `export()` raise.

## Licensing

Check the license on the Hugging Face repository of the specific weights you download. That repository is authoritative and licenses are not always uniform across a family. This is a description of the licenses involved, not legal advice.

- Original work: Falcon OCR, Technology Innovation Institute (TII)
- Upstream license: Apache-2.0
- Upstream source: https://github.com/tiiuae/Falcon-Perception
- LibreYOLO code: Apache-2.0
- Weights: Apache-2.0, republished at https://huggingface.co/LibreYOLO
- Interpretation: Apache-2.0 is a permissive license, so the Falcon OCR weights and code can be used in commercial and closed-source products. Redistributing them means keeping the license text and TII's NOTICE, and the license grants a patent license. LibreYOLO ports the inference code of the Hugging Face release tiiuae/Falcon-OCR at revision 0bc28f89c90fbe8bdb063097fab144d4ce4c19b5 (Falcon OCR 1.5), which shares its architecture with TII's Falcon Perception repository (commit c457916c9974efbacfa91f0f6ecc2c49c6543e56, Apache-2.0, whose NOTICE covers the weights); see libreyolo/models/falconocr/NOTICE. The adapter is MIT, and the optional layout stage uses LibreYOLO's Apache-2.0 PP-DocLayoutV3 port. The weights are an unmodified, ungated mirror at LibreYOLO/LibreFalconOCR270m that ships the LICENSE and NOTICE.

Falcon OCR is Apache-2.0, code and weights, by the
[Technology Innovation Institute](https://github.com/tiiuae/Falcon-Perception).
Commercial use is allowed. The weights are an ungated, byte-identical mirror.
The optional layout stage uses LibreYOLO's Apache-2.0 PP-DocLayoutV3 port.

## Citation

```bibtex
@article{bevli2026falcon,
  title   = {Falcon Perception},
  author  = {Bevli, Aviraj and Chaybouti, Sofian and Dahou, Yasser and Hacid, Hakim and Huynh, Ngoc Dung and Le Khac, Phuc H. and Narayan, Sanath and Para, Wamiq Reyaz and Singh, Ankit},
  journal = {arXiv preprint arXiv:2603.27365},
  year    = {2026},
  url     = {https://arxiv.org/abs/2603.27365}
}
```

Copied from https://huggingface.co/tiiuae/Falcon-OCR#citation
