Reason Agent 1.1 beats Codex on GPT-6 Astra in Reason Machines' coding benchmark runs

What did Reason Machines announce?

Reason Machines, a San Francisco company whose GitHub organization dates from February 2026, sells Reason, an agent that takes a software task from start to finish on its own computer. You ask from a Mac app, a terminal, the web or Slack, and it returns a pull request with a recording of what it did. On 10 October 2026 the company published a blog post introducing Reason Agent 1.1, a new version of the harness that wraps whichever model the agent is using.

The headline claim is narrow and specific. On GPT-6 Astra, which Artificial Analysis lists as released by OpenAI on 3 September 2026, run at xhigh reasoning effort and scored on the same tasks by the same graders, Reason Agent 1.1 reaches 65.1 on the Coding Agent Index. Codex on the same model scores 62.5 and an unmodified base harness 61.4. Reason says its harness also costs 15 to 23% less per attempt than Codex on every one of the three benchmarks, which puts its cost per solved task 22% below Codex and 10% below the base harness.

The same day, Reason created a public GitHub repository, reason-machines/benchmark, with the runner, scorer and methodology it used, so other people can reproduce the runs with their own Reason account.

How does Reason Agent 1.1 save tokens?

Reason reports that on GPT-6 Astra its harness used 23% fewer input tokens per attempt than Codex, 1.92 million against 2.48 million, and 15% fewer output tokens. It attributes the savings to two principles, while saying plainly that it has not yet tested how much of the saving each one accounts for.

The first is parallel tool execution. Instead of returning to the model after every single tool call, Reason gives the model a codemode tool that runs JavaScript, and that JavaScript can call the other tools. The model can write one short program that runs a search, reads five files and launches a test run, then returns only the parts that matter. In the company's illustration, the same task takes three model calls in Reason and seven in a harness that makes one call per tool.

The second is a deliberately small tool set. The model sees a few general coding tools and a short prompt. Everything else a coding platform needs, such as orchestrating subagents, cloud computers, preview URLs and pull requests with screenshots and video, sits behind deferred tools that codemode finds when needed. Because the prompt and tool list never change between turns and each request only appends to the last, every turn stays eligible for the provider's prompt cache. That is the same lever behind Asana's 76-fold cost cut from keeping a browser agent's cache intact: on long agent runs, re-read input dominates the bill, and Reason cites an analysis in which raising the cache hit rate from 95% to 98% cut one modelled bill by about 17%.

What do the benchmark numbers show?

The Coding Agent Index is a composite maintained by the independent benchmarking firm Artificial Analysis. Version 1.5 averages three benchmarks with equal weight: DeepSWE v1.1, with 113 software-engineering tasks graded by hidden tests; SWE-Atlas Codebase QnA, with 124 questions about real codebases graded against rubrics; and Terminal-Bench 4.0, with 66 command-line tasks. Reason ran all 303 tasks three times and reports pass@1, the average share of attempts that pass.

Reason's lead over Codex is 2.6 points, and its post gives 95% confidence intervals computed over the same tasks. The public Harbor Hub job for the GPT-6 Astra run lists 909 trials, 898 completed and 11 errors, roughly 1.6 billion total tokens at a 95% cache hit rate, and $3,586.30 of inference cost, about $3.95 per finished trial. Reason also ran GPT-6.1 Sol at medium, high and xhigh effort and says its median cost per task there was 34% lower than Codex at the same effort, comparing against Artificial Analysis's own Codex rows.

How can you check or rerun the results?

The repository explains how to rerun the results. You install Harbor 0.22.0, an open framework for evaluating agents in containers, with the reason-machines-harbor adapter, set a Reason API key, and point Harbor at a benchmark dataset and a model. You can bring your own OpenAI key, in which case OpenAI bills the model calls directly. Each attempt starts in a new, empty Reason workspace, and Reason recommends a frozen harness release named runtime/reason-v7-2026-10 so that reruns use exactly the same harness.

The methodology, version 1.6.2, is unusually detailed. Timeouts score zero, provider refusals are redrawn up to ten times, infrastructure errors are retried, and attempts affected by a confirmed harness or environment bug are voided and rerun. Passing Terminal-Bench attempts are screened for reward hacking with Artificial Analysis's judge, and failures score zero. Cost is priced from token counts in each trajectory using a committed price table, so anyone can recompute it.

Why the harness matters as much as the model

Benchmark tables usually rank models, but agent results depend heavily on the scaffold around them. Reason cites the HarnessTax study by Melissa Z. Pan, Ion Stoica, Matei Zaharia and colleagues, which it says found that the same model can cost up to five times more in one harness than another, and that in independent tests a harness other than the model maker's own scored best in nine of twelve comparisons. That supports a practical point for teams choosing tools: switching harness, as products like Together Link, which runs open models inside Claude Code and Codex let you do, can matter as much as switching model.

Reason also argues that labs train models together with their own harness in reinforcement-learning environments, so a lab's harness plays best on its own models, and that a harness built for every model has to stay out of the way instead. Whether or not that holds generally, the claim that an outside harness can beat OpenAI's own on an OpenAI model is notable, and it gives buyers a reason to compare agents on cost per solved task rather than on pass rate alone, as we also stressed in IQuest-Q1's agentic coding results.

What are the limits of these claims?

The most important caveat is that these are Reason's own runs. Artificial Analysis publishes the index and the Codex comparison rows for GPT-6.1 Sol, but the Codex figure on GPT-6 Astra comes from public Harbor jobs rather than an Artificial Analysis listing, and the Reason result has not been independently reproduced. A 2.6-point lead on 303 tasks is real but modest, and Reason's own chart uses rounded scores.

Reason's methodology is candid about gaps. Its October 2026 Harbor jobs ran on the default harness of the time, before the frozen release existed, and they do not pass some of the strict checks it now recommends, including per-response token records and trajectory fidelity. Its server reported only a trial's own session until 9 October, so earlier results under-count the cost of attempts that started subagents. Trajectories mask secret-like words, and OpenAI's encrypted reasoning appears only as summaries. The benchmark repository also had no license file when we checked, and rerunning it needs a Reason account and API key plus model spend: the GPT-6 Astra job alone cost about $3,600 in inference.

Finally, the scores say nothing about day-to-day use in a messy codebase, review quality, or security. Teams evaluating Reason should run it on their own tasks, as with any agent, including tools like Tencent's TeamAI Git harness for shared agent skills.