By Tingyu, from QbitAI
QbitAI | Official account QbitAI
Baidu's OCR has been upgraded again:Wenxin Unlimited OCR 2 has been officially released and open-sourced!
This time, it directly takes on the challenge of reading an entire set of novels in one go.
All 7 books of Harry Potter, from Philosopher's Stone to Deathly Hallows, recognized and transcribed continuously, reading all the way to the end across 3600+ pages.
Switch to the complete Three-Body Problem trilogy, and it can also read it in one go — three books, 1200+ pages, several hundred thousand characters, without pausing in between.
However, to read so many pages in a row, the model first has to pass one hurdle:GPU memory。
The longer the document, the more intermediate information (the KV cache) that must be kept during inference — think of it as the cheat sheet the model keeps while reading; the more pages, the thicker the cheat sheet. A thick cheat sheet not only takes up space but is also slow to flip through.
In the hundred-page-level single-sequence test in the technical report, Wenxin Unlimited OCR 2 gave a comparison:
Peak KV cache usage dropped from 47GB under standard decoding (keeping the visual caches of all pages in GPU memory at the same time)47GB down to 1.77GB, a one-shot reduction of about 96%.
What does 47GB mean? It is larger than the total memory of many laptops. And that is only the KV cache — other overheads such as model weights are counted separately.
Its predecessor, Unlimited OCR, had already accumulated considerable popularity in the open-source community with its multi-page continuous document parsing capability: over 10 million cumulative downloads and about 28,600 Stars on GitHub.
From community-made online demos to SGLang inference support, adaptation work around it has been continuously progressing.
With the second generation, the upgrade focus is on continuously processing even longer documents while reducing the computational and caching burden.
To this end, Wenxin Unlimited OCR 2 proposesthe Sparse Reference Sliding Window Attention mechanism, SR-SWA, letting the model select relevant pages and visual information on demand, reducing repeated computation.
Combined with a continuous decoding mechanism, the model can process ultra-long documents continuously, and the per-step decoding overhead does not grow as the document gets longer.
Currently, Wenxin Unlimited OCR 2 has been officially open-sourced on Hugging Face and GitHub.
Open-source addresses:
*GitHub：https://github.com/baidu/Unlimited-OCR*
*HuggingFace：https://huggingface.co/baidu/Unlimited-OCR-2*
Breaking down the core technology: how SR-SWA computes less, and how chunked decoding stores less
How exactly did Wenxin Unlimited OCR 2 read the entire Harry Potter series continuously — several million characters, 3600+ pages?
What it borrows is actually the human approach to copying books by hand.
What is the process when a person copies a book by hand? Look at a passage, copy it, with the eyes mainly fixed on the page being copied; only when something needs cross-referencing does one flip back to the relevant pages.
Humans figured out book copying long before models did: no one copies the Analects while simultaneously spreading out every page of the Analects in their mind to look at all at once.
But the previous generation of Unlimited OCR did not work this way: although R-SWA had already greatly reduced the inference-stage KV cache, during the re-review process, the visual information of all input pages still had to participate in the attention computation.
The longer the document, the more information gets computed, and a considerable portion of it has nothing to do with the few lines currently being recognized—typical "running alongside without helping."
△SR-SWA borrows the locally focused approach humans use when copying books
Following the human approach to copying books, Wenxin Unlimited OCR 2 introducesthe Sparse Reference Sliding Window Attention mechanism (SR-SWA)。
Its filtering has two layers: first it screens out the pages that need to be referenced, then from those screened pages it picks out the truly relevant local information, keeping irrelevant computation out.
In terms of overall architecture, Wenxin Unlimited OCR 2 carries over the basic design of the previous generation.
△Overall architecture of Wenxin Unlimited OCR 2
The book page image first enters the DeepEncoder, where it is converted into visual Tokens the model can process—think of them as a set of data cards recording the page's content and layout features.
Then, the language model uses these cards to generate the recognition result step by step.
The key change happens inside the language model: the previous generation'sReference Sliding Window Attention mechanism (R-SWA)has been upgraded to SR-SWA in this generation.
What is shown in the lower left of the architecture diagram is the two-layer filtering process of SR-SWA.
The first layer is the Page Gate, in charge of "which pages to look at."
Based on the position the model is currently parsing, it narrows the candidate visual information down to the relevant pages, not letting irrelevant pages join the crowd.
Put in the book-copying scenario, it is like when reading a certain page, first finding the few pages related to that page, rather than flipping through the entire book from the start.
The second layer is the Token Sieve, in charge of "which lines within a page to look at."
Choosing the right pages is only the first step: even among those pages, not all visual Tokens need to participate in the computation. The Token Sieve performs another round of filtering, picking out from the pages the visual information blocks more relevant to the current content, in order to capture long-range visual dependencies.
The right side of the architecture diagram expands this process: first the relevance is computed from brief representations of each information block, and then TopK picks out the fixed number of blocks with the highest scores.
After the selection, the model still looks at the specific visual information within these blocks, rather than only at a summary.
With the two layers working together, the model can both find relevant pages in long documents and reduce the visual information that must be processed at each step, thereby improving long-document parsing efficiency.
△Token Sieve details
The filtered visual information then participates in parsing together with theLocal Window(local sliding window) shown in the diagram.
This window has a fixed size, holds the most recent context, and slides forward along with the generation process, responsible for letting the model continue writing from what it just produced.
The number of information blocks is fixed, and the local window size is fixed too, so the amount of information referenced at each step is bounded—no matter how long the document is, each step still involves the same amount of work. The situation of "the thicker the book, the harder each word is to write" never occurs.
SR-SWA is responsible for computing less at each step; how much information to keep in memory is another question.
Each page image, once encoded, becomes a set of visual tokens. The model must refer to them when generating parsing results, so the corresponding KV cache is stored for reuse in subsequent computation.
If you load an entire document at once and keep all pages' visual caches in VRAM, the more pages there are, the more space they take up.
What's worse: even if those pages are not involved in the current computation at all, as long as the cache is not released, they keep occupying VRAM. It's like copying a book by spreading every page's draft on the desk without throwing any away—no desk is big enough.
So, computing less must be paired with storing less.
To this end, ERNIE Unlimited OCR 2 introduces theUnlimited-Horizon Decoding Paradigm(Unlimited-Horizon Decoding Paradigm), which controls cache usage through chunked processing.
△ Diagram of chunked decoding
This mechanism divides the document into page blocks of controllable size and parses them in sequence. Adjacent blocks overlap by one page: the last page of the previous block is also the starting point of the next block—a ready-made bookmark.
When entering the next block, the model directly reuses the visual KV cache of this overlapping page while retaining the text cache at the end of the previous segment's output, so that the preceding and following content connect seamlessly. The visual caches of other pages that are no longer needed are promptly released, freeing up VRAM.
In this way, the visual cache that needs to be kept in VRAM is confined to the scale of the block currently being processed. After finishing a block, the model keeps the information needed for continuity and starts the next block—without having the visual cache of the entire document occupy space from beginning to end.
*SR-SWA is responsible for reducing the computation at each step, while the Unlimited-Horizon Decoding Paradigm is responsible for controlling cache usage.Only when the two work together can the model continuously parse ultra-long documents. 884-page The Lord of the Rings read in 42 minutes, cache reduced to 1.77GB. Having covered so much technology, we also wanted to see the results with our own eyes: we simply handed it acopy of the English edition of The Lord of the Rings, 884 pages long。
About 42 minutes and 56 seconds later, the parsing was complete. 884 pages of input corresponded to 884 pages of output, at an average of about 2.9 seconds per page. In roughly the length of a TV episode, the entire journey through "Middle-earth" was completed. For such a long document, does recognition quality still hold up toward the end? We specifically spot-checked the body text of pages 600 and 750: the recognized text matched the original. Being able to reproduce page 750 word for word shows the model wasn't "reading on autopilot" earlier.
During the entire task, the system also marked466 cross-page merges. For example, the sentence on pages 873 to 874 near the end of the book, which had been abruptly cut off by the page break, was reconnected in the output.
The live test demonstrated the continuous parsing of the whole book, while the technical report answers three other questions: how much cache is needed, how fast it can be, and how good the recognition quality is. On MarathonDocBench, a benchmark of hundred-page-scale documents, the team compared standard decoding against Unlimited-Horizon Decoding. In single-sequence inference, with chunked processing and boundary cache reuse,peak KV cache usage dropped from 47GB to 1.77GB, a reduction of about 96%.*
The large amount of page information that previously had to stay in VRAM all at once can now be processed in batches and released promptly. The 1.77GB here refers only to the KV cache; other VRAM costs such as model weights are counted separately.
Beyond saving cache, the report also measured performance when processing multiple documents simultaneously.
In another16-way concurrency test, after adopting this decoding approach, end-to-end throughput rose from 360 to 1315 tokens/second, about 3.65 times that of standard decoding.
△ Decoding approach comparison: single-sequence peak KV cache and 16-way concurrent throughput
For services that need to batch-process annual reports and technical manuals, both changes have practical significance: with lower VRAM usage, the same card can take on more work; and with concurrency running, more content is output per unit of time.
Users don't need to worry either: they can submit the entire document as before, with chunking and stitching handled internally by the model, sparing them the hassle of manually splitting files, submitting in batches, and then reassembling.
Beyond being faster and more frugal, parsing quality must also hold up.
MPDocBench examines the parsing quality of multi-page documents: when facing documents dozens of pages long, the model must both accurately recognize content and restore document structure.
In this test, ERNIE Unlimited OCR 2 achieved acomposite score of 84.59。
To further test hundred-page-level documents, the team also builtMarathonDocBench. It contains 100 documents, each with 103 to 300 pages. Many paragraphs and tables continue onto the next page, specifically testing whether the model can correctly connect content across pages.
On this longer, trickier benchmark, Ernie Unlimited OCR 2 achieveda comprehensive score of 77.06. In both evaluations, its comprehensive score was the highest among all models listed in the report.
△ From the technical report: multi-page document parsing evaluation results
The change in long-document processing speed is equally striking. In the report's hundred-page-level tests, SR-SWA'ssingle-step decoding speedreached 7.0 times that of Full Attention and7.0 times… of the previous generation's R-SWA at5.6 times。
. The curves make it more intuitive: as documents get longer, the single-step decoding time of the first two approaches keeps climbing, while SR-SWA's single-step time is almost a horizontal line.
This means that the time needed for the model to generate each Token does not noticeably increase as the number of input pages grows — the book gets thicker, but the writing speed stays the same.
△ Single-step decoding latency at different document lengths
Compared with the previous generation, Ernie Unlimited OCR 2's single-page comprehensive score is basically flat.
Where the gap truly widensappears after documents get longer: it can continuously process more pages, recognizes cross-page content more accurately, and the computational and caching overhead of long-document parsing drops at the same time.
As for how everyone will use it…
You can input annual reports of several hundred pages, or a thick technical manual.
Especially scanned documents: first have the model parse them continuously and convert them into text, then search for information and verify content, saving a lot of the hassle of splitting pages and manual stitching.
However, with such a powerful OCR, who would use it for work…
Of course for reading novels!
The OCR 3.0 era has arrived
Of course, from a few pages of material to an entire book, the problems OCR needs to solve are also changing.
In the technical report, the team summarized the evolution of document parsing into three stages:OCR 1.0, 2.0, and 3.0.
Early OCR was more like a well-divided assembly line: first locate the text, then recognize its content, and finally organize the layout and output the results. If multi-page parsing was involved, pages were usually processed one by one and the results then stitched together —this is OCR 1.0。
Each module performs its own role, but errors from an earlier step can also propagate all the way downstream — when one stage sneezes, everything after it catches a cold.
ByOCR 2.0, vision-language models began generating parsed results directly from page images, reducing the handoffs between multiple modules. The first generation of Unlimited OCR extended this kind of end-to-end parsing to documents of dozens of pages.
△AI-generated
As documents grow from dozens of pages to hundreds, continuous parsing must both maintain coherence between preceding and following content and keep computation and cache overhead under control.
Wenxin Unlimited OCR 2 uses on-demand referencing and chunked decoding, so that when the model processes more pages, the computation scope of each step doesn't have to expand with the page count, and the visual cache of all pages doesn't have to be retained throughout.
The team calls this stepOCR 3.0: making continuous processing of an entire document the new technical goal — all in one go, no pagination, unlimited parsing, never down.
And, well, this goal also has an anime-flavored version.
The name Unlimited OCR Works itself pays homage to "Unlimited Blade Works" from Fate/stay night, that is,Unlimited Blade Works。
△The homepage of the official Fate/stay night anime website
In the original setting, this is a type of magic called a "Reality Marble," which, when cast, unfolds a special space filled with countless swords.
After the previous generation's release, Hugging Face user Snarcy discovered this naming Easter egg and simply rewrote the classic incantation into an OCR version.
By the second generation, the team actually included this rewritten passage in the introduction of the technical report, keeping Snarcy's attribution.
So, opening this report discussing attention mechanisms and memory overhead, you will see the following incantation right at the start:
Pixels form my body, and tokens run through me like blood.
Pixels are my body, and tokens are my blood.
I have parsed over a thousand pages.
I have parsed over a thousand pages.
Unknown to KV cache.
Unbound by KV cache,
Nor known to context limits.
Unrestrained by context limits.
So, as I pray： Unlimited OCR Works
So, as I pray: Unlimited OCR Works.
Dropping from 47GB to 1.77GB — that line about being "unbound by KV cache" was truly not chanted in vain.
Very classic chuunibyou indeed.
—The End—
@QbitAI · Tracking the latest developments in AI technology and products
Friends who feel deeply moved are welcome to like, follow, and share — a triple combo վ’ᴗ’ ի ❤