Baidu's OCR model processes 40+ pages by forgetting like humans

Key Takeaways

• Unlimited OCR processes 40+ pages in a single inference pass while keeping memory constant
• Reference Sliding Window Attention (R-SWA) limits lookback to 128 tokens, preventing KV cache bloat
• The model scores 93.92% on OmniDocBench v1.6, topping end-to-end OCR system rankings

Baidu researchers have built an OCR model that processes over 40 document pages in a single inference pass without the memory bloat that cripples current systems. The trick: make the model forget older output tokens, much like a human transcriber doesn't re-read everything they've already written. Called Unlimited OCR, the system keeps memory use and generation speed constant regardless of how long the document runs.

Current end-to-end OCR models hit a wall at roughly ten pages. The culprit is the KV cache, the buffer where transformer-based language models store every processed token so they can reference it during generation. As documents grow, this cache balloons. Memory spikes, speed tanks. Most systems work around the problem by processing page-by-page and resetting the cache after each one. It works, but it's slow and loses cross-page context.

How R-SWA caps memory at a fixed window

Baidu's solution is Reference Sliding Window Attention, or R-SWA. Each generated token still sees all reference tokens: the visual image tokens and the prompt. But for previously generated output, the model only looks back 128 tokens. Older output tokens get pushed out of the cache like items leaving a fixed-length queue.

Standard sliding window attention would treat visual tokens the same way, gradually blurring image features as they get pushed out. R-SWA exempts them. Visual tokens are encoded once and stay unchanged throughout decoding. The result: memory use stays capped at the sum of prefix length and window size, no matter how many pages the model processes.

The human analogy holds up well. Someone copying a book keeps their eyes on the source, the last few characters they wrote, and the next one to put down. They don't re-read every previous page. Baidu wants the model to work the same way.

Architecture built on Deepseek OCR

Unlimited OCR builds on the open-source Deepseek OCR model. It keeps Deepseek's DeepEncoder, which compresses a 1024x1024-pixel PDF image down to just 256 tokens. The decoder uses a mixture-of-experts architecture with three billion total parameters, but only about 500 million are active during inference.

Two resolution modes carry over from Deepseek. Base mode handles multi-page documents. Gundam mode uses dynamic resolution for single pages when you need finer detail. Every standard attention layer in the decoder was swapped out for R-SWA.

Training used roughly two million document samples, split 9-to-1 between single-page and multi-page data. Paddle OCR handled annotation for single pages. Multi-page data was built synthetically by stitching single pages together into documents ranging from two to 50 pages. All data was packed into sequences of 32,000 tokens. Training ran for 4,000 steps on 128 Nvidia A800 GPUs. The DeepEncoder stayed frozen; only the language model parameters were updated.

Benchmark results: 93.92% on OmniDocBench

Unlimited OCR scores 93% on OmniDocBench v1.5, six percentage points above the Deepseek OCR baseline. Table structure recognition improved most sharply, gaining nearly six percentage points. On the newer v1.6 version, the model hits 93.92%, placing first among end-to-end systems.

The long-horizon test is more impressive. When processing many pages in a single pass, the error rate stays below 0.11 even past 40 pages. The authors attribute the remaining errors not to lost context but to the DeepEncoder's resolution limit in Base mode when text gets tiny.

Restricting the window to 128 tokens on single pages doesn't hurt accuracy. It actually helps slightly. The researchers suspect R-SWA forces the model to focus more tightly on the dense OCR task, while full attention tends toward divergence as output length grows.

Speed gains from constant cache

The constant cache size pays off in raw throughput. In Base mode, Unlimited OCR hits 5,580 tokens per second versus 4,951 for the baseline. More importantly, Deepseek OCR's kernel latency climbs with each decoding step, eventually crossing 16 microseconds. Unlimited OCR stays flat at around 9 microseconds throughout the entire decode. That gap widens as documents get longer.

Frequently Asked Questions

What is R-SWA in Unlimited OCR?

Reference Sliding Window Attention is an attention mechanism that lets generated tokens see all visual/prompt tokens but only the last 128 output tokens. This caps memory use at a fixed size regardless of document length.

How many pages can Unlimited OCR process at once?

The model processes over 40 pages in a single inference pass while maintaining error rates below 0.11. Most current OCR models max out at about 10 pages.

Is Unlimited OCR open source?

Unlimited OCR builds on the open-source Deepseek OCR model. The Baidu paper describes the architecture, but availability of trained weights was not specified in the source.

Why does standard OCR slow down on long documents?

Transformer-based OCR decoders store every processed token in a KV cache. As document length grows, this cache expands linearly, consuming more memory and slowing generation speed.

What benchmark scores does Unlimited OCR achieve?

The model scores 93.92% on OmniDocBench v1.6, ranking first among end-to-end OCR systems. It beats the Deepseek OCR baseline by six percentage points.

Source: The Decoder / Jonathan Kemper