How to process 1,000-page documents without hitting context limits
Unlimited-OCR extracts text from any document. Recursive Language Models analyze it in chunks. Together they handle documents no single model can read.
TL;DR: Document processing has two independent bottlenecks. Models can’t read scanned PDFs (format ceiling). And even when they can, context rot degrades quality beyond ~50 pages (context ceiling). Unlimited-OCR solves the first — 40+ pages in one pass. Recursive Language Models solve the second — sub-RLMs with fresh contexts for each section. Together they process 200-page documents for ~$0.035.
My document pipeline had two ceilings. I hit the first one six months ago. The second one I hit today.
The first ceiling was format. I needed to extract text from 50-page scanned PDFs — contracts, invoices, research papers — and send them to a language model for analysis. The model couldn’t read the PDFs. Every OCR tool I tried broke at page boundaries. Tables spanning two pages lost their structure. Reading order got confused. I ended up stitching page outputs together manually, introducing errors at every seam.
Unlimited-OCR solved that. A 3B MIT-licensed model that reads 40+ pages in one forward pass. Its Reference Sliding Window Attention keeps KV cache constant. No page-by-page for loop. No state resets. One shot. I wrote about it here.
The second ceiling was context. Unlimited-OCR gave me clean text from 200-page documents. But no language model could hold 200 pages in its context window without degrading. Even models that fit the whole text showed context rot — answers got worse past 50 pages, not better. I was extracting text I couldn’t analyze.
That is the hole Recursive Language Models fill.
Key takeaways:
- Document processing has two independent ceilings: format (models can’t read scanned content) and context (models degrade past ~50 pages of text)
- Unlimited-OCR solves the first: 40+ pages in one pass, SOTA on OmniDocBench (93.92%), MIT-licensed
- RLM solves the second: recursively decomposes analysis into sub-RLMs with fresh contexts, handles 10M+ tokens at $0.99/pass
- Together they form a two-model pipeline that processes documents no single model can handle end-to-end
- The Vertical Agent Method maps naturally: one agent for extraction, one for analysis, a clean handoff between them
What does each ceiling look like?
Format ceiling. Your input is a scanned contract, a photographed invoice, a handwritten form. The model needs text, but the document is an image. Traditional OCR handles one page at a time — every page boundary resets the model’s state, tables break, reading order scrambles. Unlimited-OCR eliminates the page boundary entirely by processing the full document in one shot.
Context ceiling. Your input is now clean text from 200 pages. Even models with long context windows degrade as text grows. The paper calls this “context rot” — the phenomenon where model quality drops consistently past a certain input length. An RLM avoids this by treating the text as an external environment. It writes Python code to search, chunk, and recursively analyze relevant sections. Each sub-RLM gets a fresh context window with only the relevant snippet. Rot doesn’t accumulate because each call is clean.
The pipeline
Scanned document (PDF, image)
↓
Unlimited-OCR — 3B MoE, R-SWA attention
One-shot parsing, no page boundaries
↓
Clean structured text (from 40+ pages)
↓
RLM (predict-rlm) — recursive decomposition
Chunks text, spawns sub-RLMs per section
↓
Structured output (JSON, spreadsheet, report)Stage 1: Unlimited-OCR — takes the document, outputs clean text with preserved layout. A single forward pass through the R-SWA attention mechanism means no context reset between pages. Tables that cross pages maintain their structure. The model is 3B parameters with 0.5B active, fits on any 16GB+ GPU, and costs nothing to run (MIT license, open weights).
Stage 2: RLM — takes the text, treats it as an external environment, and recursively decomposes the analysis. The model writes Python to read relevant sections, spawns sub-RLMs for deep dives, and assembles results. Each sub-RLM gets its own context and its own tools. The parent RLM only keeps the summary.
How the RLM decomposition works
The predict-rlm runtime makes this concrete. You define the task as a DSPy signature:
class AnalyzeDocument(dspy.Signature):
"""Extract key dates, entities, and financials from the document."""
document: str = dspy.InputField()
report: str = dspy.OutputField()
rlm = PredictRLM(
AnalyzeDocument,
lm="deepseek/deepseek-v4-pro",
sub_lm="deepseek/deepseek-v4-flash",
)The RLM gets the full document text. But it doesn’t load the entire thing into a single prompt. Instead it writes Python code — loops, function calls, recursive sub-RLM invocations — to work through the document section by section.
For a 200-page contract analysis:
- The RLM reads the table of contents, identifies sections
- It spawns a sub-RLM per section: “Analyze section 3: payment terms”
- Each sub-RLM gets that section’s text in a fresh context
- Results flow back up: sub-RLM → main RLM → formatted report
The cost for this: roughly $0.01 per 8 decomposition turns on DeepSeek V4-Flash. That’s absurd — cheaper than a single API call to a frontier model, for a 200-page analysis.
When the pipeline breaks
Three cases where this is the wrong solution.
Short plain-text documents. If your input is a clean text file under 10 pages that fits in a standard model’s context window, use the model directly. The OCR + RLM pipeline adds latency and complexity for no benefit.
Latency-sensitive workflows. This pipeline takes 2-5 minutes per document. The OCR pass is fast (Unlimited-OCR processes 40 pages at roughly 7,800 tokens/second), but the RLM decomposition involves multiple LLM calls. Each sub-RLM call takes 2-10 seconds. For a 200-page document with 10 sections, expect 3-5 minutes total.
Documents where the analysis is faster than the setup. If you need a one-line answer from a 2-page memo, the overhead of setting up the pipeline dwarfs the time a human would take. Use the tool when depth matters, not speed.
The architecture pattern is vertical agents
This is the Vertical Agent Method applied to document processing. Two agents, each owning one workflow.
Agent 1: Extraction (Unlimited-OCR). One job: convert any document format to clean text. It doesn’t analyze. It doesn’t summarize. It reads.
Agent 2: Analysis (RLM). One job: decompose the text into analysis tasks and assemble results. It doesn’t extract. It doesn’t format-shift. It analyzes.
The handoff between them is a text file on disk. Clean. Testable. Replaceable.
The paper that started this conversation — Recursive Language Models — shows that RLM(GPT-5) on BrowseComp+ (6-11M token inputs) scores 91.33% at $0.99 per pass. Standard GPT-5 can’t even process those inputs. A summary agent gets 58% at $1.31 per pass. The RLM is both better and cheaper.
That kind of gap doesn’t come from prompting better. It comes from using the right architecture for the right sub-problem.
The cost math
| Step | What it costs | Time |
|---|---|---|
| Unlimited-OCR pass (40 pages) | Free (MIT, local GPU) | ~5 seconds |
| RLM decomposition (8 turns, V4-Flash) | ~$0.01 | ~2 minutes |
| Sub-RLM per section (5 sections) | ~$0.005 each | ~30 seconds each |
| Total for 200-page analysis | ~$0.035 | ~5 minutes |
Compare this to a human analyst reading 200 pages: 4-6 hours at roughly $50-100/hour. Even a single API call to Claude Opus or GPT-5.5 for a full document run costs more ($1-3 per run) and hits context rot past ~50 pages.
FAQ
Does Unlimited-OCR work with handwriting? The model is trained on document OCR data — printed text, tables, layouts. Not handwriting or natural scene text. For handwritten documents, pair it with a handwriting-specific model before passing to the RLM.
What document formats does the pipeline support? Unlimited-OCR accepts scanned PDFs, images (PNG, JPG), and document photos. The RLM stage works with any text input. For native digital PDFs (not scanned), skip the OCR stage and feed text directly to the RLM.
Can the RLM stage run locally? It depends. predict-rlm supports Ollama as a backend, so you can run smaller models locally. Sub-RLMs on a local 7B model will be slower but free. The main RLM typically needs a stronger model for decomposition reasoning.
Does this replace traditional RAG? They serve different needs. RAG is for retrieval over a large corpus — find the relevant 3 pages out of 10,000. This pipeline is for deep analysis of a single long document — understand all 200 pages thoroughly. Use both in the same system if needed.
What is the maximum document length? Unlimited-OCR currently handles 40+ pages in one pass with slight quality degradation at the upper end (96.08% Distinct-20 at 40 pages vs 99.76% at 2 pages). The RLM has no theoretical limit — each sub-RLM processes a chunk sized for its context window.
Related Posts
- Unlimited-OCR: the 3B model that reads 40-page documents in one shot. The architecture and benchmarks behind the extraction half of this pipeline.
- Loop engineering: the production agent loop nobody talks about. RLM is a different approach to the agent loop — model-controlled instead of harness-controlled.
- The Vertical Agent Method framework. The methodology behind building agents that own one workflow each and hand off cleanly.
- llama.cpp: the local inference engine that changed how developers ship AI. Running Unlimited-OCR and a local RLM on the same machine without cloud costs.
This article was published on Agentic Up (https://agenticup.dev): practical guides for developers and founders building with AI agents. Reach me at [email protected]
