Hallucinations are a product of how transformer models are trained, and how they generate text. If you’re casually chatting with a model, the occasional made-up detail is tolerable. But when you’re processing invoices, contracts, lab reports, or financial statements, a single fabricated number can carry real legal and financial consequences.
Over the last few months we’ve invested heavily in keeping our hallucination rate as low as possible, and in building tools to enable manual review of any remaining potential hallucinations. Together, our systems achieve a near-zero hallucination rate on our internal benchmark.
This post walks through the system that makes that possible. It’s several independent layers, each catching what the previous one misses.
It starts with the model
The cheapest hallucination to handle is the one that never happens. So the first line of defense is the OCR model itself — we train ours specifically to minimize hallucination risk, and we do research on architecture and training methods that make rare or ambiguous text easier to read correctly.
One example we’ve published openly: character-level tokenization. By having the model generate one character at a time, we let it spend a constant, predictable amount of compute on every character, without the strong “autocomplete-the-common-word” bias that trips up BPE models. The payoff is that rare sequences — an unusual part number, a foreign name — get transcribed far more faithfully. Some of this work is open source in Surya OCR 2.
Still, no model is perfect. On the hardest inputs — dense handwriting, degraded scans, unusual layouts — even a strong model occasionally fails. On an internal benchmark set of our most degeneration-prone pages, raw model output contains a hallucination on roughly 6 in 10 pages. Next, we’ll go into more detail on how we mitigate those.
Page-level checks
After a page is OCRed, we score it on several quality axes that are good at exposing the signatures of a hallucinated page. If a page scores below threshold, we regenerate and re-check, repeating until it clears or we escalate to the next layer.
In the benchmark set, the page-level layer flags about 67% of pages as suspect, and resolves a quarter of those on its own. The rest get escalated for closer inspection.
Block-level checks
When a page can’t be cleared at the page level, we zoom in. We split it into its individual blocks — a table, a paragraph, a signature block — and examine each one independently. Working at this granularity matters: a single bad region on an otherwise-perfect page is easy to miss at the page level but obvious up close, and re-reading just that region is both more accurate and far cheaper than redoing the whole page.
For each suspect region we attempt to heal it — re-read it until we get a stable, confident transcription. When we can heal it, the corrected text flows through transparently. When we can’t — when the region is genuinely illegible and we can’t produce a reading we’re confident in, we mark it illegible instead.
A wrong number that looks right is the most dangerous output a document system can produce. An explicit “we couldn’t read this” is something your team — or your downstream automation — can catch, route, and handle.
Word confidence
The final layer puts you in control. We recently shipped word bounding boxes and confidence scores — for every word on the page, you get its exact location and how confident the model is in the prediction. You can see at a glance where the model was confident and where it wasn’t, and review the uncertain spots before they ever reach your downstream work.
Every word boxed and scored in the playground — the single low-confidence word is flagged in red for review.
Seeing it in action
It’s easiest to understand on real handwriting. Here are three samples from our stress set, run through the live system.
Recovering difficult cursive. A handwritten letter from 1847. The script is hard, but it’s legible, so we read it faithfully:
Refusing to guess. And a page where the handwriting is genuinely unreadable — a dense, struck-through manuscript draft. Rather than fabricate a plausible-looking transcription, the system flags it:
How we measure it
We maintain an internal benchmark built specifically around failure: 121 single-page samples mined from real-world documents, each one a page we’ve actually seen push OCR into a hallucination. It’s adversarial by design, not a representative sample of everyday work (where hallucinations are rare).
We run every page multiple times and measure the fraction of runs that come back degenerate. Lower is better.
| Configuration | Pages that degenerate |
|---|---|
| Raw model, single pass | ~61% |
| + one sampling-level safeguard | ~41% |
| Full multi-stage system | ≈ 0% |
Each safeguard we add moves the number down, and they compound. Running the full system end-to-end over 60 of these worst-case pages, we found no genuine surviving hallucinations — and across the 920 regions on those pages, 97.5% were recovered as confident text while 2.5% were flagged illegible.
The result
Put together, these layers turn an unavoidable property of LLMs into something you can actually rely on. On our benchmark set of the hardest, most failure-prone pages:
- Raw model output degenerates on roughly 6 in 10 pages.
- After the full system, the surviving hallucination rate is ≈ 0%.
- Of everything we flag, 97.5% is recovered automatically and the rest is marked illegible.
On real customer documents — including handwritten records that broke earlier versions of our pipeline — the same system eliminates blanked pages and runaway repetition entirely, recovering legible content and clearly marking the rest.
Hallucinations will always be a property of how these models work. Our job is to make sure they never quietly become your problem.
If you’re processing mission-critical documents and want OCR you can trust — in the cloud or fully on-prem — reach out to our team or try it yourself.