We evaluated seven production extraction systems on the same 225 documents: four dedicated document-extraction platforms (Reducto Deep Extract, Extend - MAX, LlamaExtract - Agentic, Datalab Extract - Balanced) plus three frontier LLMs called directly (GPT-5.5, Claude Opus 4.8, Gemini 3.1 Pro). The corpus is deliberately hard: documents average 358 pages and roughly 88,700 ground-truth fields each. Every system was run in its strongest available configuration: The frontier models were run with maximum thinking/reasoning enabled across the board, and Reducto, Extend, LlamaExtract, and Datalab Extract in their highest-accuracy extraction modes.

The headline is not a single accuracy number; it is a gap in who can finish the job at all. Results are reported along four separate dimensions, kept deliberately distinct: success performance (accuracy on completed documents), failure metrics (accepted then could not finish), incompatibility (refused up front as unsupported in kind, independent of size), and latency.

Much of the highest-value work in enterprise data extraction lives in long, content-rich, table-heavy documents: statistical releases, census tabulations, financial and regulatory filings, healthcare datasets, scientific reference tables. These documents routinely run from hundreds to thousands of pages and pack tens of thousands of individual values into dense, repeating tables, and the numbers locked inside them feed real downstream workflows, from financial models to regulatory reporting to operational decisions. Extracting them is a substantial challenge: the length alone strains context windows and output limits, tabular structure has to be held together across page breaks and shifting layouts, and dropped or misread rows can quietly corrupt an entire analysis. This is exactly the environment where extraction systems degrade or fail outright the capability this benchmark measures.

The question this benchmark answers is simple: which platform performs best when the documents are dense, across accuracy, robustness, and latency, rather than on a curated simple set. We measure not just "how accurate is it when it works," but "how often does it work at all, and how does it fail when it doesn't."

Ground truth was created independently from the raw documents, without any extraction vendor or product in the loop, and was human-reviewed and reconciled before acceptance.

The candidate labels were drafted by frontier models reading each source document directly: GPT-5.5 and Claude Opus 4.7. No document-extraction product, and no OCR, parsed text, or layout from any extraction system, was part of the process. Human annotators then reconciled the disagreements between the two models and additionally reviewed a sample of the labels the models agreed on, confirming that agreement reflected correctness rather than a shared mistake, before the result was accepted as ground truth.

The dataset is a mixture of short releases and documents in the thousands of pages. Density is the defining property: a few dozen schema fields expand into tens of thousands of ground-truth values per document because those fields repeat across long tables. The corpus spans government and public-sector statistics, census and demographic tabulations, labor series, financial filings and asset-backed-securities reports, healthcare and Medicare datasets, regulatory/permit filings, and scientific reference tables; it is predominantly English-language public documents.

The grader is deterministic: no LLM, no fuzzy matching beyond an explicit cosmetic normalizer. Each document yields three numbers in [0, 100]:

Row matching is key-based, never positional. For each ground-truth array the grader infers the field(s) that most uniquely identify a row, then matches predicted rows to ground-truth rows by that key. Documents are score and then averaged with equal weight.

1. Recall is the greatest differentiator. Precision and leaf accuracy are high almost everywhere; recall (getting all the rows out of a dense document) is what separates the field.
2. Completeness is a real challenge. Most systems are reasonably accurate on what they return. The benchmark is decided by how much of a dense corpus a system can return at all.
3. In this benchmark, direct frontier LLM calls were less robust than dedicated extraction systems on long, dense documents. Called directly, they break on the defining property of this corpus (size) via truncation, context limits, page caps, and schema rejections. Their strong accuracy on the documents they finish does not carry to the larger scale documents.
4. The strongest dedicated extraction platforms add real robustness over raw LLMs (Extend and LlamaExtract finish 90%+ of the corpus vs. ~50% for the lowest coverage frontier models).

This benchmark aims to be a fair, reproducible comparison, but its scope and provenance carry caveats that should be read alongside the results:

• Sponsorship. Reducto commissioned this benchmark. Reducto is also one of the systems under test and the top performer on these documents.
• Methodology provenance. Reducto authored parts of the methodology and the ground truth generation harness.
• Model-assisted ground truth. Ground truth was drafted with frontier models and then reconciled by hand; it was not written from scratch by humans for every document.
• Label bias. AI-drafted labels can carry the biases of the models that produced them, including blind spots shared with the systems being graded - a model may be scored generously where it errs in the same way the labeler did.
• Different classes of system. Direct frontier-LLM API calls are not equivalent to dedicated extraction platforms. This compares two different classes of system, with different interfaces, defaults, and intended use, on a single task formulation.
• Scope. Results are specific to this corpus, schema setup, provider configurations, and run date. Provider models and services change over time; numbers may not reproduce on a later run.
• Sampled label checks. Agreement-field sampling provides evidence that the labels are sound, not a mathematical guarantee that the full corpus is correct. Unsampled documents could contain label errors that this process would not surface.
• Reproducibility. The evaluation harness, grading methodology, provider adapters, and benchmark implementation are available in the accompanying GitHub repository. Due to licensing and commercial restrictions, the full benchmark corpus is not publicly released. A representative subset of 50 benchmark tasks is provided to illustrate the methodology and enable inspection of the evaluation pipeline.