LongHarness keeps evidence and conclusions in sync for 100M-token corpora

What is LongHarness?

LongHarness is a model-agnostic agent harness for questions whose answers sit across document collections larger than a model’s context window. The 9 October 2026 arXiv preprint (v1) comes from Songtao Huang, Shiyang Feng and co-authors at Shanghai Artificial Intelligence Laboratory, Fudan, Peking University and other Chinese institutions. The abs page uses arXiv’s nonexclusive-distrib license, so this article relies on the authors’ text and AI Lookout charts rather than copying paper figures.

The core idea is Evidence–Conclusion Coevolution: unresolved conclusions steer what to search and read next, while newly admitted evidence revises those conclusions. Evidence Lifecycle Management keeps an EvidenceBank of source-linked statements that can be upserted, revised or retracted with registered read offsets.

That is a different bet from stuffing more tokens into one prompt, and from coding-agent harnesses such as Claude Code or Codex-style shells that keep state mostly inside transcripts and files. Related harness work on our site includes DeepSeek’s desktop harness and SpatialHarness-style scaffolding, but LongHarness is aimed specifically at multi-document corpus reasoning.

What failure modes is it trying to fix?

The paper names two coupled failure modes. Unbounded exploration is retrieval that widens the interaction history without closing decision-relevant gaps, burning the turn or read budget. Evidence-state drift is when an earlier fact or a later correction stops informing the current conclusion, so the final answer rests on stale or incomplete evidence.

Those failures reinforce each other: drift hides which needs are already solved, so the agent searches again, the history grows, and earlier evidence becomes harder to recover. The authors’ case study of a Treasury geometric-mean question shows Codex locating a table, continuing long backward reads, correcting a May 1942 value from 4100 to 4400, then still submitting the old computation.

LongHarness’s answer in that vignette is not magic math—it is recording the correction in the EvidenceBank, tracking cross-issue consistency as an open gap, and only then calling compute.

How do EvidenceBank and Conclusion State work?

The EvidenceBank holds source-linked statements extracted from passages the agent has actually read. Admission requires registered source references from prior `read` actions; the controller checks reference validity. Revisions create a new active entry and mark the old one superseded; retractions entries leave the active set but remain in the revision record.

The Conclusion State stores established claims, supported hypotheses and unresolved gaps, each linked to evidence IDs. Conclusion updates after EvidenceBank mutations wait until the agent observes the bank result, so claims are not rewritten against rejected upserts.

The action space mixes `search`, bounded `read`, EvidenceBank `upsert`/`revise`, deterministic `compute` with declared evidence references, and `final`. Runtime limits in the reported setup include 200 turns, 60 searches, a 1,000,000-token cumulative read budget and 16,384 tokens per read. Codex and Claude Code were capped at 200 model turns and 200 tool calls in the comparison.

What did the OfficeQA Pro numbers show?

OfficeQA Pro V1 has 133 questions over 697 US Treasury documents (~100M tokens). V2 expands to 1,435 documents and harder compositional numeric questions. Agents see the full corpus without gold document hints. Direct baselines get BM25 passages into a 256K-token window.

With GLM-5.3-Flash, the main text reports LongHarness exact match of 60.2% on V1 and 61.1% on V2, versus 15.8% and 4.4% for the same backbone without the harness. The authors say that beats Claude Code by 12.0 points on V1 and Codex by 18.9 points on V2. With Qwen3.8-Flash, LongHarness reaches 61.7% on V1.

Appendix D’s three-run means for GLM-5.3-Flash put LongHarness at 59.90% / 59.63% on V1/V2, Claude Code at 49.37% / 32.59%, Codex at 46.62% / 41.11% and RLM at 45.36% / 36.30%. Those means are what our chart plots. Main-table comparisons are mostly one run per setting because of cost; variance is non-trivial on V2 for some baselines.

Do the ablations support the design?

Table 2 ablates the design on the same GLM-5.3-Flash backbone. Removing persistent states drops V2 exact match by 7.8 points; removing Evidence–Conclusion Coevolution drops 9.4; append-only evidence without revision drops 10.0; removing explicit conclusion-guided exploration instructions drops 7.8; replacing structured conclusions with free-text notes drops 7.8.

All five ablations hurt more on V2 than V1, which matches the claim that maintaining and reconciling evidence matters more when answers need values from multiple sources. The authors also audit trajectories into retrieval, evidence editing and transformation stages; on V2, LongHarness’s stage success rates stay high while Claude Code’s median turns explode.

How does interaction length change on harder tasks?

Median model turns with GLM-5.3-Flash rise from 22 to 30 for LongHarness between V1 and V2, from 21 to 116 for Claude Code, and from 19 to 43 for Codex. Codex exhausted its recorded turn budget on 3.8% of V1 questions and 21.1% of V2 questions.

The paper’s explanation is that unresolved gaps and retained evidence let LongHarness reuse facts instead of re-finding them as cross-source checks multiply. That is an author interpretation of turn histograms, not an independent replication, but it is consistent with the ablation pattern.

How does it compare on AA-LCR and LongBench v2?

On AA-LCR 1.1 (provided document sets) and the LongBench v2 Long subset, LongHarness also leads the harness comparison in the authors’ tables, with GLM-5.3-Flash exact scores of 84.0% and 75.9% in the main single-run summary, and three-run means of 84.0% and 75.31% in Appendix D. Those tasks still matter for long context, but they are less about hunting a 100M-token warehouse than OfficeQA Pro is.

Separately, teams shipping long-context decision models—such as SubQ Snap’s 384k decision model—are attacking a related bottleneck from the model side. LongHarness argues the harness layer still decides whether found evidence stays actionable.

What are the limits?

This is a preprint. Main harness comparisons are often one run; Appendix D’s three runs show LongHarness is stable, but Claude Code and Codex move several points on V2. AA-LCR harness answers were graded with GPT-5.6-Luna, introducing judge variance.

There is no public code repository linked from the abs page we inspected, so practitioners cannot yet drop LongHarness into production the way they can clone Codex CLI or Claude Code. Backbones, tool versions (Codex CLI 0.135.0, Claude Code 2.1.216) and read budgets are specific; transferring the recipe to other stacks will take engineering.

Finally, author-reported wins against Claude Code and Codex are matched-backbone harness comparisons, not claims that LongHarness beats those products’ default frontier models. Read the paper’s limits before treating 60% OfficeQA exact match as a general enterprise SLA.