MissionReportBench

Key findings

Six findings shape how these results should be read. Each is stated first, then supported in the sections below.

1. Why we built MissionReportBench

Writing a status report well is hard, and the difficulty is not in the prose. A battalion commander reviews reports from six companies and passes a few items to brigade, so a company’s report has to lead with the one item that needs the commander’s help, even when another item looks more urgent. A program status report has to separate what is known from what is not: a test result still under review is not a finding, and an overall rating that no input provides is not the writer’s to assign. An award citation follows a format set by the awards manual, down to its opening and closing lines, and a promotion brief has to fit the boxes on its form. A detail can be accurate and still not belong in the report, and dropping one qualifier can turn a tentative finding into a settled one.

These judgments come from years of government experience. Experienced staff officers know which facts their reader needs, what their commander already knows, how their organization’s report should look, and which acronyms to spell out. Some of that knowledge is written down in regulations, local guidance and prior reports kept on internal drives. Much of it is not written down at all. Models have had little chance to learn it from the open web.

MissionReportBench measures that judgment directly. Each task is an assignment written by an expert who has done the job: a role, a reader, the working files and the organization’s template. The model must produce a finished Word document, the format government offices work in. The expert’s rubric then scores each decision the report makes, with partial credit, so the results show which decisions models get right and which they miss.

2. Methodology

2.1 Task construction

The 20 tasks cover four report families and five settings: Army, Air Force, Navy, Marine Corps, and Department of Defense offices and agencies. Readers range from a battalion commander to an Under Secretary and a promotion board. Each task was written by an expert with reporting experience in that organization, who assembled the source files and wrote the prompt, a golden report and a weighted rubric. Every task passed micro1’s quality review before sign-off.

Tasks carry 1 to 15 source files, 163 in total, and most include the organization’s template, prior reports or the governing regulation. Source text runs from under a thousand words to over a hundred thousand. Nineteen tasks set a length rule: fourteen prompts give a word limit, three a page limit, and in two the limit comes from the form the report must use.

2.2 Agent’s environment

Each attempt runs in a fresh Linux sandbox with the source files under /app/attachments/. The model must save a Word document under /app/outputs/, the only file the grader reads. The expert’s report and the rubric are never in the sandbox. The model works through a shell and file tools in the OpenCode harness, with pandoc, pdftotext, LibreOffice and Python document libraries available.

2.3 Evaluation protocol

Run settings. Every model ran every task three times using default reasoning effort, with a 60-minute limit and no feedback between attempts. All models ran using the same harness (OpenCode 1.18.31), sandbox image and grader.

Scoring. A report’s score is the positive points it earns minus the penalty points it triggers, divided by the task’s positive points, bounded to 0 to 100%. The 20 rubrics hold 302 positive criteria worth 535 points and 63 penalties worth 88 points. A model’s score is the mean over tasks of its three-report mean, and intervals resample tasks. Each criterion carries one of five category labels (Table 1).

Judge. An agentic judge, Claude Opus 5.5 in OpenCode, reads each report, can render its pages, and compares it with the expert’s report. It decides each criterion true or false with a written reason and never sees the weights, the grading notes or the source files. As a check, we graded each expert’s golden report against its own rubric to ensure it scores 100%.

3. Overall performance

Gemini 4 Argon leads at 64.1%, followed by Claude Sonnet 5.5 (63.8%), Claude Opus 5.5 (63.5%) and Claude Fable 5.1 (63.3%). Muse Wonder, GPT-6 Astra and GPT-6.1 Sol follow within three points, then Grok 4.7 at 59.5% and Gemini 3.8 Flash at 54.3%. The intervals of the top seven overlap.

Full credit is rare. Three of the 540 reports meet every criterion and trigger no penalty, one each from Opus 5.5, Sonnet 5.5 and Gemini 3.8 Flash.

A second attempt changes the score more than it changes what is missed. Taking the best of three with hindsight adds 5.2 to 9.0 points, and criteria missed in all three attempts hold 62 to 82% of each model’s missed points.

Models with similar scores fail on different criteria. A model’s result on a single criterion agrees with its own repeat attempt at a median r of 0.71 and with another model at 0.54. On 105 of 365 criteria, one model succeeds in all three attempts where another fails in all three. Some difficulty is shared: 29 to 43% of each model’s missed points are criteria that every other model also misses.

4. Where models lose points

Scored the same way as a full report, earned points minus penalty points, format is the easiest category at 71.1% and synthesis, stating what several facts mean together, is the hardest at 50.0%. Judgment, deciding what goes in and what leads, is the largest category with 42% of positive points and nets 63.3%: models earn 70.7% of its points and lose 7.4% to penalties for material the reader does not need. Format loses the most to penalties, 10.4%.

Basic formatting is close to solved: 32 of the 48 format criteria, such as word limits, reporting periods, unit names and classification markings, are met in at least 90% of reports. Most of the remaining format points are lost on a few form conventions, such as an award citation’s closing line, a performance brief’s extra sections and a manual’s cover page, which section 6.4 covers.

Judgment and synthesis together account for 67 to 77% of every model’s lost points. That is the difficulty the benchmark was built to measure: a short report has to choose what goes in and state it at the level a senior reader needs.

5. Writing more adds penalties, not points

A rubric that credits each item a report covers invites a simple strategy: write more and cover more. On MissionReportBench that strategy does not pay. Within a task, longer reports trigger more penalties (r = +0.27) and earn no more positive points (r = +0.03). The extra length adds unnecessary details to the reports and includes material the reader does not need.

The same pattern exists across models. Models that write longer reports trigger far more penalties than models that write shorter reports (r = +0.84 between median words per report and penalties triggered). Claude Fable 5.1 writes the longest reports and, with Gemini 4 Argon, triggers the most penalties: 29.1% more than the nine-model average. Conversely, GPT-6.1 Sol and GPT-6 Astra write the shortest reports and trigger 35.5% fewer penalties than the average. Most of these penalties catch material from the sources that the expert left out because the reader does not need it.

6. Failure modes

We audited failed criteria drawn evenly across models, reading each model’s report and the full record of its work. In nearly every case the model had read the facts it needed, usually within its first few steps. The failures happen when the model decides what to write.

6.1 Choosing what leads the report

An Army airborne infantry company’s weekly report has to flag that its only certified range safety officer leaves on 1 March, before the company’s 9 March rifle range, and the next certification class is not until 16 March. The expert leads with that gap and asks the battalion for help. All 27 reports miss the criterion for leading with it, though most mention the gap somewhere. Every model opens with a cold-weather injury, the first item in the battalion commander’s priority memo. GPT-6 Astra listed the gap among “three priority items” at step 4 and then put it last: “Need certified RSO support to protect 31 March qualification deadline; range packet due 20 Feb.”

A prompt can pull models the same way. A Pentagon executive office’s weekly report must summarize senior executive issues that could draw negative press. The sources contain none, and all 27 reports fill the section with an item about a different workforce. Grok 4.7 noted at step 2 that the item “does not meet the requirement for SES-specific negative press items” and included it anyway.

6.2 Summarizing: listing the facts without saying what they mean

A UN Command security battalion’s 115-word weekly report has to present a North Korean guard’s ten-second line crossing as part of a week of unusual guard activity at the same two buildings, which the operations log records on five days. Every model reports the crossing alone, and 26 of 27 reports miss the pattern. Claude Sonnet 5.5 had the whole log in its first tool output and wrote “13 Nov, 1352: KPA guard crossed MDL approximately one meter near T2/T3, returning after ten seconds.” The one report that earns the criterion, by Fable 5.1, adds four words: “Atypical KPA guard activity all week.”

Sometimes the model tells the user what it should have told the reader. An Army battalion’s readiness report has to explain that 55 Soldiers with passing fitness records do not count as ready because they are on temporary medical profile. Fable 5.1 computed the 55, put it in its first draft, and cut it at step 13 to meet the 400-word limit. Its closing message to the user still made the point; the commander’s report did not.

6.3 Reporting incomplete information

Government reports often have to describe things that are not fully known: a cause still under investigation, a test not yet run, a record that cannot be found. A good report says what is known, what is not, and what will settle it. Models have trouble with this in two ways.

First, they leave out what is not yet known. Outright errors are rare: several penalties for misstating a fact, such as presenting a conditional ship date as firm, never fire in any report. But a Navy program office’s status report keeps two test results and drops the fact that the cause of the gap between them is still under review. An Army acquisition report gives a milestone slip without saying that its effect on later milestones has not been assessed. Each is missed in 26 of 27 reports.

Second, some models fill the gap with a conclusion of their own. The Navy program office’s sources never give an overall program rating. The three Claude models and Gemini 3.8 Flash assign one in every attempt, and Opus 5.5 said so itself: “none of the source documents gives one. I chose AMBER because four risks are AMBER and two items are open ahead of DT-1.” An Air Force wing’s inspection report prompt tells the model not to treat missing records as proof that something did not happen, and the evaluator “could not establish that the outstanding location had been surveyed.” Fable 5.1, Gemini 3.8 Flash and Gemini 4 Argon write in every attempt that the area was never surveyed. GPT-6 Astra, GPT-6.1 Sol and Grok 4.7 never state an unsettled matter as settled in any of the audited cases.

6.4 The Word document: forms and templates

Where the prompt states a word limit, median length lands within 0.96 to 1.27 times the expert’s, and the rubric’s length checks pass 89 to 100% of the time. Where the form sets the limit, models run long: the performance brief and the award citation come in at 5.0 and 6.2 times the expert’s length.

The performance brief comes with a blank form of five boxes. In 25 of 27 reports, models add a section the form does not have; Gemini 3.8 Flash’s report runs to 20 pages, and its closing message claims “Exact visual alignment” with the blank form. On the award citation, GPT-6 Astra, GPT-6.1 Sol, Grok 4.7 and Gemini 3.8 Flash write a median of 127 to 162 words against the expert’s 125, while the Claude models and Gemini 4 Argon write 775 to 1,988 and Muse Wonder 5,339. An Army after-action report that names the Army’s FM 6-0 manual as its reference leaves out the manual’s cover page in 26 of 27 reports, and most closing messages say the report follows the manual.

Templates fare the same way. On the 11 tasks that supply one, 27% of reports were rebuilt from a blank python-docx document instead, from 15 of 33 for Grok 4.7 to none for Gemini 4 Argon. A rebuilt document keeps the template’s words and loses its numbering, page numbers, header and footer markings and margins. Opus 5.5 rebuilt the security battalion’s report from blank and closed with “It follows the battalion commander’s template.”

7. How models approach the problem

The nine models work very differently on the same tasks. Gemini 3.8 Flash makes a median of 61 tool calls per report and Claude Sonnet 5.5 makes 9. Across models, more tool calls go with lower scores (r = −0.81), but that rests mostly on Gemini 3.8 Flash, which makes the most calls and scores lowest; without it, r = −0.31. Within each model, the number of tool calls on an attempt barely relates to its score (r between −0.26 and +0.24).

8. Report families

Weekly activity reports are the easiest family, at 64.2% averaged over all models; the other three sit together at 58.7% to 59.9%.

The model order changes by family. The Claude models score 68 to 70% on weekly activity reports against 59% for the GPT-6 models, and the order reverses on program status reports, where GPT-6 Astra scores 66.5% and Opus 5.5 53.7%. Families hold three to nine tasks, so these describe this task set rather than general rankings.

9. Conclusion

MissionReportBench measures a task government staff do every week: turn a pile of working files into a short report, in the office’s form, for a reader who will act on it. Frontier models produce readable Word documents and carry over what the sources state. They average about 60%, and 3 of 540 reports meet every criterion.

1. The losses sit in the decisions experienced staff make without thinking.Which item leads, what the details mean, how to report what is not yet known, and what to leave out.
2. The highest scorers get there by including more.Longer reports trigger more penalties and earn no more points; the extra material is what a stricter reader would penalize.
3. The headroom is in knowing what a senior reader needs.Putting only that on the page, in the form the office uses.

Appendix A. Example reports side by side

Each exhibit shows an expert page beside page 1 of a model’s report on the same task, rendered with Word’s fonts. The model report is that model’s middle-scoring attempt unless noted.