The OCR with the highest average score may not be able to read your table

Akshay Pachaar ran three OCR solutions through the same 150-page document: DeepSeek-OCR, PaddleOCR-VL, and Datalab Surya. The pages ranged from handwritten notes and rotated scans to dense tables—plus a filled-out U.S. 1040 tax form.

Why test this way? A single overall score only shows who comes out ahead on average. Your document is not an average document. Heavy handwriting, complex tables, skewed scans—each scenario can make a particular parser fail, and an overall average won't reveal it.

So every page was checked with yes/no questions: Did this line appear? Is the cell under the correct header? Does A come before B?

Results:

• No parser won on every type of page
• DeepSeek-OCR was best on tables, 87.2
• On handwritten notes, one parser passed 5/6 checks; the other two scored 0
• On the filled-out 1040 tax form, PaddleOCR-VL returned only the title

Only the title. A tax form, reduced to nothing but a title in its eyes. That's what a specific failure looks like—and what an average score can't see.

Behind this logic is OmniParse Bench, Datalab's open-source benchmarking tool. 16,288 tests, 2,937 pages, 92 languages. Each page is tagged with the difficulties it contains: handwriting, rotation, small text, forms, tables. Scoring uses open code, with no LLM judge, so it's reproducible.

The way to judge is close to engineering intuition: don't chase a pretty overall score; look only at which specific actions work. When choosing a solution, you shouldn't pick the one with the highest overall average—you should pick the one that doesn't fail on your documents.

One commenter raised a direct objection: Datalab's own benchmark, with Datalab ranked first—they wanted a third party to run it too. Fair. The open data and code at least leave a way to verify. Someone else tried a range of options and ultimately chose Datalab's Chandra-2. Others suggested fixing the benchmark version for every evaluation, so you can rerun comparisons after switching solutions.

Around the same time, Baidu open-sourced Unlimited OCR 2, also aimed at long documents. AGTP did a quick overview. Baidu's core is SR-SWA, Sparse Reference Sliding Window Attention, which compresses text-vision attention interactions into a fixed-size reference set. Combined with Unlimited-Horizon Decoding, it frees KV cache that is no longer needed. The result: per-step decoding latency and memory usage barely grow with document length.

Numbers: peak KV cache dropped from 47 GB to 1.77 GB; decoding is 7× faster per step; all 7 Harry Potter books—3,600+ pages—in one run; the Three-Body trilogy—1,200+ pages of Chinese—in one run. 3.34B parameters, MIT license, available on Hugging Face, supports Transformers and SGLang.

The model was just released; whether it can compete, you'll know after one run. Long-document OCR has long been a pain point, and this direction is worth watching.