Unlimited-OCR: the 3B model that reads 40-page documents in one shot
Baidu's MIT-licensed OCR model uses Reference Sliding Window Attention to parse dozens of document pages in a single forward pass. SOTA on OmniDocBench at 93.92%.
My document parsing pipeline had three stages. Convert PDF to images. Feed images one by one into an OCR model. Stitch the results back together. Every page reset the model’s state, so context across pages was zero. Tables that spanned pages broke. Reading order got confused at every boundary.
I assumed this was just how OCR worked.
Then Baidu dropped Unlimited-OCR. A 3B model under MIT license that accepts 40+ pages in a single forward pass and outputs the full transcription without pagination. No for loop. No state resets. One shot.
TL;DR: Baidu’s Unlimited-OCR is a 3B MIT-licensed model that does one-shot long-horizon document parsing. Its Reference Sliding Window Attention keeps KV cache constant, achieving SOTA on OmniDocBench (93.92%) and maintaining a steady 7,800+ tokens/second regardless of document length. It reads 40+ pages in one pass with 96% Distinct-20 accuracy.
Key takeaways:
- Unlimited-OCR achieves SOTA on OmniDocBench at 93.92% Overall, beating DeepSeek OCR by +6.22 points
- Reference Sliding Window Attention keeps KV cache and inference speed constant regardless of output length
- Reads 40+ document pages in a single forward pass. No page-by-page for-loop needed
- MIT-licensed, 3B params (0.5B active), fits on a single consumer GPU (6.78 GB at bf16)
- 35% faster than DeepSeek OCR at 6K output tokens (7,847 vs 5,822 TPS)
What problem does this solve?
Every end-to-end OCR model before Unlimited-OCR had the same flaw. Standard attention accumulates KV cache linearly with each generated token. A 10-page document creates a 10x larger KV cache than a 1-page document. A 40-page document blows past GPU memory on any consumer GPU.
The workaround is the for-loop. Parse page 1, reset state, parse page 2, reset state. It works but it fragments context. Tables that cross pages break. The model doesn’t know what it wrote on the previous page, so repeated content appears. Reading order errors compound at every boundary.
The paper describes this problem in a single sentence: “When humans perform such tasks, they maintain a continuous cognitive state in which distant outputs fade softly from memory, while nearby context is used to track progress.”
Current models do the opposite. They maintain exact memory of every token (full attention) at linear cost, or they reset entirely at page boundaries (for-loop). Neither matches how humans transcribe.
What is Reference Sliding Window Attention?
R-SWA is the mechanism that makes this work. Each generated token attends to all reference tokens: the visual tokens from the document images plus the prompt, while only attending to the preceding 128 output tokens for the generated text.
The KV cache stays constant. It doesn’t grow with output length.
Here is what this looks like in practice from the paper’s efficiency analysis:
STANDARD ATTENTION (DeepSeek OCR) | R-SWA (Unlimited-OCR)
─────────────────────────────────────|─────────────────────────────────
Token 256: 7,229 tokens/sec | Token 256: 7,229 tokens/sec
Token 1K: 7,422 tokens/sec | Token 1K: 7,840 tokens/sec
Token 4K: 6,430 tokens/sec, slowing | Token 4K: 7,905 tokens/sec
Token 6K: 5,822 tokens/sec, caching | Token 6K: 7,847 tokens/sec
KV cache at 6K: linear growth | KV cache at 6K: constant (512+128)
Result: model slows down 19% | Result: model stays flat at 7,800+The contrast is clean. Standard attention slows down as the output grows. R-SWA stays flat. At 6K tokens, Unlimited-OCR runs 35% faster than DeepSeek OCR.
The paper also reports that at 512 concurrency on the OmniDocBench benchmark, Unlimited-OCR achieves 5,580 TPS vs DeepSeek OCR’s 4,951 TPS — a 12.7% speedup even on short documents. The gap widens with longer outputs.
How does the benchmark compare?
The paper tests on OmniDocBench, the standard OCR evaluation suite. Both v1.5 and v1.6 versions. The v1.5 table includes 17 end-to-end models for comparison. The technical report has the full data.
| Model | Size | Overall | Text Edit | Formula CDM | Table TEDS | Read-Order |
|---|---|---|---|---|---|---|
| OCRFlux | 3B | 74.82 | 0.193 | 68.03 | 75.75 | 0.202 |
| GPT-4o | — | 75.02 | 0.217 | 79.70 | 67.07 | 0.148 |
| olmOCR | 7B | 81.79 | 0.096 | 86.04 | 68.92 | 0.121 |
| MinerU2-VLM | 0.9B | 85.56 | 0.078 | 80.95 | 83.54 | 0.086 |
| Qwen2.5-VL | 72B | 87.02 | 0.094 | 88.27 | 82.15 | 0.102 |
| DeepSeek-OCR 2 | 3B | 89.17 | 0.049 | 86.85 | 85.60 | 0.060 |
| DeepSeek OCR | 3B | 87.01 | 0.073 | 83.37 | 84.97 | 0.086 |
| Unlimited-OCR | 3B | 93.23 | 0.038 | 92.61 | 90.93 | 0.045 |
| vs DeepSeek OCR | +6.22 | -0.035 | +9.24 | +5.96 | -0.041 |
On OmniDocBench v1.6 (the latest version):
| Model | Size | Overall |
|---|---|---|
| DeepSeek-OCR 2 | 3B | 90.25 |
| dots.ocr | 3B | 90.77 |
| FireRed-OCR | 2B | 93.26 |
| Qianfan-OCR (Baidu) | 4B | 93.90 |
| Unlimited-OCR | 3B | 93.92 |
Unlimited-OCR ties Qianfan-OCR (a 4B model from Baidu released in March 2026) while using 25% fewer parameters. It beats every other model at every size.
The formula recognition improvement (+9.24 CDM over DeepSeek OCR) is the biggest surprise. Mathematical notation in documents is notoriously hard for OCR models because spacing and positioning carry semantic meaning. The improvement suggests R-SWA’s constant KV cache eliminates the attention drift that standard attention exhibits on long-formula outputs.
What does multi-page performance look like?
This is the headline feature. The paper tests on an in-house dataset of books at 2, 5, 10, 20, and 40+ pages.
| Pages | Distinct-20 | Distinct-35 | Edit Distance |
|---|---|---|---|
| 2 | 99.76% | 99.87% | 0.036 |
| 5 | 99.78% | 99.98% | 0.045 |
| 10 | 97.49% | 99.83% | 0.053 |
| 20 | 98.73% | 99.89% | 0.057 |
| 40+ | 96.08% | 96.90% | 0.107 |
The key observation: edit distance degrades gracefully. At 2 pages it is 0.036. At 40+ pages it is 0.107. There is no cliff at 10 pages where the model forgets what it is doing.
The paper notes that errors at 40+ pages come from small text rendered at 1024x1024 resolution (the DeepEncoder’s “Base” mode), not from R-SWA losing track of position. Switching to higher resolution or the “Gundam” mode (640x1024 with cropping) would likely improve this.
What is the architecture?
Under the hood, Unlimited-OCR is a DeepSeekV2 MoE model with a dual vision encoder.
| Component | Spec |
|---|---|
| Total params | ~3B |
| Active params | ~0.5B (64 experts, 6 active per token) |
| Architecture | DeepSeekV2 MoE, 12 layers |
| Hidden size | 1280, 10 heads |
| Vision encoder | CLIP-L-14 (224px) + SAM ViT-B |
| Context | 32K (128K planned) |
| Weights | 6.78 GB bfloat16 |
| License | MIT |
The vision encoder compresses image tokens aggressively (DeepEncoder from DeepSeek OCR). The decoder uses R-SWA across all layers. Training was 4,000 steps on 2M document samples, continuing from the DeepSeek OCR checkpoint with the encoder frozen. Done on 8x16 A800 GPUs (128 GPUs) using Megatron-LM and DeepEP for expert parallelism.
How do you run it?
Three paths, all documented on the GitHub repo:
Transformers (quick test):
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("baidu/Unlimited-OCR",
trust_remote_code=True, torch_dtype="bfloat16").cuda()
tokenizer = AutoTokenizer.from_pretrained("baidu/Unlimited-OCR",
trust_remote_code=True)
model.infer(tokenizer, prompt="<image>document parsing.",
image_file="doc.pdf", output_path="./output",
base_size=1024, image_size=640, crop_mode=True)SGLang (production):
python -m sglang.launch_server \
--model baidu/Unlimited-OCR \
--context-length 32768 \
--host 0.0.0.0 --port 10000Batch inference via infer.py:
python infer.py --image_dir ./docs/ --output_dir ./outputs \
--concurrency 8 --image_mode gundamWhere does it fall short?
The paper is upfront about three limitations.
Prefill still grows with page count. R-SWA solves the decode memory problem but not the prefill memory problem. The vision encoder compresses aggressively, but at 40+ pages the prefill still uses significant memory. The 32K context window sets a hard limit.
Currently document-only. Unlimited-OCR is trained on 2M PDF document samples. It handles books, papers, reports, and forms. It has not been trained on handwriting, natural scene text, or mixed-media documents. For those use cases, PaddleOCR or a general VLM is still the right tool.
Resolution limits on multi-page. The Base mode (1024x1024) is used for multi-page to keep prefill manageable. Small text below a certain size becomes hard to read. This is a tradeoff the paper explicitly acknowledges and plans to address with higher-resolution modes.
The 128K context and “prefill pool” mechanism (where the model learns to fetch KV chunks on demand, like a human flipping pages) are planned but not yet shipped.
What is the verdict?
Unlimited-OCR is the first MIT-licensed model that does real one-shot long-horizon document parsing. The R-SWA mechanism is simple in concept and effective in practice. The benchmark numbers are clean — SOTA across every metric on OmniDocBench, with the widest margins on formula and table extraction.
The limitations are honest. The prefill problem means “unlimited” still has limits. But for anyone processing 5-30 page documents in a pipeline, this model eliminates the page-by-page for loop entirely. No more context fragmentation. No more reading order errors at page boundaries. No more stitched-together results with repeated content at seams.
For a 3B model under MIT license that runs on a single GPU, that’s a genuine shift in what is possible with open-source OCR.
FAQ
How does this compare to running GPT-4o for OCR? GPT-4o scores 75.02 on OmniDocBench. Unlimited-OCR scores 93.23. The gap is 18 points. For document OCR specifically, the specialized model beats the generalist by a wide margin. GPT-4o costs about Rs 36 ($0.435) per 1M input tokens. Unlimited-OCR costs zero in inference — just the GPU electricity.
Can I use this for handwritten text? The model is trained on printed document data only. Handwriting recognition isn’t supported. PaddleOCR (also from Baidu) has handwriting support.
Does it preserve table structure? Yes. Table TEDS is 90.93 on v1.5, the highest among all models tested. The paper attributes this to R-SWA maintaining reading order coherence across long outputs.
What GPU do I need? The model is 6.78 GB at bfloat16. Any GPU with 16GB+ VRAM works. An RTX 4090 or A10 runs it comfortably. A Mac with 32GB+ unified memory can run it via SGLang or Transformers.
Related Posts
- The open-source AI model landscape, June 2026. Where OCR models fit in the broader open-source model ecosystem.
- Nanbeige4.1-3B: How a 3B model sustains 600 tool calls and beats 30B models. Another example of a small MoE model punching above its weight class.
- The year small models ate the benchmarks. The broader trend of small specialized models outperforming large generalists on specific tasks.
This article was published on Agentic Up (https://agenticup.dev): practical guides for developers and founders building with AI agents. Reach me at [email protected]
