Baidu’s Unlimited OCR: read dozens of scanned pages in one inference pass by bounding the decoder’s memory
Baidu’s Unlimited OCR lets a single model read many scanned pages in one inference pass. It caps the decoder’s output‑history cache at 128 tokens while keeping visual encodings global. The authors say this preserves accuracy and keeps memory use roughly constant as document length grows. The paper and accompanying code were published by Baidu’s research team and reported on July 5, 2026.
The researchers put the idea simply: the model keeps the source in view and keeps only a short, rolling memory of what it just generated, “a kind of soft forgetting.”
“Someone copying a book doesn’t re-read everything they’ve already written. They keep their eyes on the source, the last few characters they wrote, and the next one to put down. Older passages fade through a kind of soft forgetting.”, Baidu researchers
The bottleneck they solved
Transformer decoders used for end‑to‑end OCR normally keep a growing key‑value (KV) cache of past generated tokens so future tokens can attend to history. That cache grows linearly with output length and pushes memory use and latency up. Common workarounds process multi‑page documents page‑by‑page and reset the cache each page, which limits single‑pass work to roughly ten pages in many pipelines, according to Baidu’s report.
Unlimited OCR instead limits that growth by changing the attention mask for generated outputs while leaving the visual and reference tokens globally available.
Reference Sliding Window Attention (R‑SWA)
The authors call the mechanism Reference Sliding Window Attention (R‑SWA). Under R‑SWA each newly generated token can fully attend to all reference tokens and to the visual tokens produced by the encoder, but it only attends to the last 128 previously generated output tokens. Visual tokens are encoded once and treated as stable references instead of being repeatedly transformed.
That combination, global attention to the document representation with a short rolling window over recent outputs, keeps the KV cache bounded. The paper reports this lets the model generate tens of pages in one pass while keeping throughput and memory use roughly flat as documents grow longer.
How the system is built
- Encoder: the team builds on the open‑source Deepseek OCR encoder (DeepEncoder). A 1024×1024 PDF page is compressed to 256 visual tokens. The encoder was kept frozen during training.
- Decoder: mixture‑of‑experts (MoE) architecture with a total of 3, 000, 000, 000 parameters and roughly 500, 000, 000 active parameters during inference (the router selects a sparse subset per input).
- Attention window: rolling output window = last 128 generated tokens; reference and visual tokens remain globally visible.
- Training data: ~2 million document samples, split 9:1 between single‑page and multi‑page. Multi‑page documents were synthetically stitched from single pages (2-50 pages). Single‑page annotations used Paddle OCR.
- Sequences & compute: training sequences were packed into 32, 000 tokens and trained for 4, 000 steps on a cluster described as “8 × 16 Nvidia A800 GPUs” (the paper’s phrasing suggests a 128‑GPU setup). Only the language model parameters were updated. DeepEncoder stayed frozen.
Reported results and what they mean
- OmniDocBench v1.5: Unlimited OCR scores 93.00% overall. By comparison, Deepseek OCR 2 scores 91.09% on the same benchmark, so the gain is about 1.91 percentage points over that baseline (both figures reported by the authors).
- OmniDocBench v1.6: 93.92%, which the authors report as topping end‑to‑end system rankings on that version.
- Long‑horizon error: normalized edit distance (as reported) remains below 0.11 even past 40 pages on the synthetic long documents used in evaluation.
- Diversity metric (Distinct‑35): reported at 97% for 40+ page inputs (a metric the authors use to track output uniqueness across long sequences).
- Throughput (Base mode): Unlimited OCR = 5, 580 tokens/sec vs Deepseek OCR = 4, 951 tokens/sec (a 12.7% increase on the authors’ evaluation setup). The paper also provides a theoretical upper‑bound scenario where Unlimited OCR leads the baseline by ~35% at ~6, 000 output tokens under ideal parallelism.
- Context ceiling: the experiments use a 32, 000‑token context, so practical single‑pass limits are bounded by that token budget and the number of visual tokens per page. The authors plan to scale to 128, 000‑token models and prototype a “prefill pool” to fetch relevant KV blocks on demand.
Modes, trade‑offs, and practical limits
Unlimited OCR exposes two operating modes. “Base” mode is tuned for multi‑page parsing and throughput. “Gundam” mode raises dynamic resolution for single‑page high‑fidelity reads. The authors report that restricting the generated‑output window to 128 tokens does not degrade accuracy on their benchmarks and can slightly improve stability, which is notable because naive sliding windows can hurt multimodal coherence if visual tokens are repeatedly transformed.
“Unlimited” is a practical label, not literal infinity. The approach shifts the bottleneck from unbounded KV growth to the fixed context length and the per‑page visual‑token budget. How many pages that buys depends on layout, font size, and how many visual tokens the encoder emits per page.
Where this fits and why it matters
Two production pressures collide in long‑document OCR: memory and latency balloon if a decoder stores every generated token, and token counts drive compute and cost. Unlimited OCR addresses the first by bounding the decoder’s KV growth and addresses the second by pairing that with a compact visual encoder. Encoding pages as a few hundred visual tokens is far cheaper than tokenizing every character as LM tokens for many long‑document tasks.
The team published code and model weights to public repositories and provide a demo (see the paper and project pages referenced by the authors). Runtime engines the authors list for experimentation include ModelScope, vLLM, and SGLang.
Risks, unanswered questions, and what to test before adoption
The technical contributions are clear, but several practical questions remain and should be checked before production adoption:
- Context scaling: how many real pages does a 32, 000‑token budget buy for your document layouts? Measure visual‑token density per page for your templates and extrapolate.
- Robustness to niche inputs: handwriting, tiny fonts, dense tables and forms, authors note Mistral OCR 3 as a strong performer for handwriting and complex tables, but direct head‑to‑head benchmarks are missing. Run targeted tests: handwriting suites, table benchmarks, and small‑font scans at 300-600 dpi across 5-50 pages.
- Latency & cost: the reported throughput gains are meaningful, but run your own latency and cost experiments on your target hardware and mode (Base vs Gundam), and validate tokenization, batching, and fp16/bfloat16 configs.
- Data provenance & copyright: large OCR corpora often include scanned books and public documents. Prior research has shown language models can reproduce memorized copyrighted passages verbatim in some settings, so verify training data licenses, apply redaction where required, and maintain audit trails when processing customer or sensitive documents.
- Privacy & governance: processing many pages in one pass means more sensitive content flows through a single inference call. Use secure runtimes, encryption at rest and in transit, and strict access controls.
Actionable evaluation plan for engineering teams:
- Measure pages per context: sample representative pages, count resulting visual tokens, and estimate pages per 32k/128k context.
- Accuracy suites: run character‑error and structured extraction tests on handwriting (IAM or equivalent), table parsing (ICDAR or internal labeled tables), and tiny‑font scans.
- Operational tests: measure end‑to‑end latency, memory usage, and cost on your deployment stack (GPU type, batch size, mixed precision).
- Governance checks: confirm training data provenance, and instrument inference logs and data retention policies for compliance.
Key questions you might be asking
- How does Unlimited OCR stop memory from ballooning as it generates dozens of pages?
The authors implement Reference Sliding Window Attention (R‑SWA): the decoder retains global access to reference and visual tokens but only attends to the last 128 generated output tokens, keeping the KV cache size bounded.
- Does that restricted output window hurt accuracy on long documents?
According to Baidu’s reported benchmarks, no, the model scores 93.00% on OmniDocBench v1.5 (vs 91.09% for Deepseek OCR 2) and 93.92% on v1.6, with normalized edit distance reported below 0.11 past 40 pages on synthetic long documents.
- What are the model and compute details I should know?
DeepEncoder compresses a 1024×1024 page into 256 visual tokens. Decoder is an MoE with ~3B parameters and ~500M active during inference. Training used sequences packed into 32, 000 tokens for 4, 000 steps on a cluster described as “8 × 16 Nvidia A800 GPUs” (paper phrasing suggests a 128‑GPU setup). The encoder stayed frozen during training.
- Is it truly unlimited, how many pages can I feed it?
Not literally. Practical limits are determined by context length and visual‑token density per page. The reported experiments used a 32, 000‑token context; the authors plan 128, 000‑token models and a “prefill pool” mechanism to extend reach.
- Can I try or run it now?
The authors published code, model weights, and a public demo; they also list runtime support on engines such as ModelScope, vLLM, and SGLang for experimentation and deployment. Run the same benchmark suites you care about before committing to production.
There’s practical elegance in deliberate forgetting: keep the document representation globally available and let the decoder maintain only a short, actionable memory of recent outputs. For teams building long‑document pipelines, legal discovery, audits, regulatory filings, multi‑page forms, R‑SWA is a concrete pattern to prototype. It moves the engine room from unbounded KV growth toward context budgeting and smarter retrieval, and that trade matters when accuracy, latency, and governance compete for attention.
Try a focused pilot: measure pages‑per‑context on real layouts, run handwriting, table, and tiny‑font suites, and validate latency and cost on your target hardware. If the numbers hold, Unlimited OCR’s bounded‑memory approach is a practical lever for scaling multi‑page document understanding without linear memory growth.