Four Claude agents post-train Qwen3-4B to instruct level in 104 hours with Athanor

What the Athanor paper claims

On 15 October 2026 researchers posted Athanor: An Agent-Native System for Autonomous Post-Training Research on arXiv as 2610.21055. The lead author is Yufa Zhou, listed with ByteDance and Duke; co-authors span ByteDance, Duke, the University of Chicago and Virginia Tech. The PDF labels Zhou as project lead and Neiwen Ling and Zijun Wei as corresponding authors at ByteDance.

The paper's headline experiment is concrete. Four researcher agents, each Claude Opus 5.5 running in Claude Code on its own eight-GPU node, start from Qwen3-4B-Base. After 104 hours they submit a single generalist checkpoint that scores 67.3 on a six-task development suite and 48.2 on a held-out suite of different questions for the same capabilities. Under the same final measurement protocol, Qwen3-4B-Instruct-2507 scores 66.5 and 47.9. The base model starts at 15.9 and 9.2. The authors call this, to their knowledge, the first public case in which a team of frontier agents autonomously post-trains a raw base model into a cross-domain generalist that matches or surpasses the vendor's instruct release on average, confirmed on a held-out suite.

That is a vendor-and-author claim about one campaign. The paper is a preprint; there is no peer-review stamp yet, and AI Lookout has not re-run the GPUs. The Hugging Face paper page mirrors the abstract. The agents never see the held-out suite during the search; the operator scores it after the campaign.

The AutoPostTrain setting and what the agents were not allowed to do

The authors frame a weaker, measurable form of recursive self-improvement: frontier agents post-training a small raw base model with public data. They call the task AutoPostTrain. One checkpoint must improve across six capabilities at once—mathematics, logic, code, knowledge, instruction following and multi-step action—rather than training a specialist per benchmark.

Constraints are designed to keep the result checkable. Agents may use public datasets, problems they generate and samples from their own checkpoints. They may not call or download an external teacher model. Humans give no research advice. Development-suite scores and scored answers come back as feedback; a sealed held-out suite stays invisible until the end. Incoming public data is decontaminated against both suites before agents can train on it.

Reference models visible to the agents include the base, a "default starter" that runs the provided starter code unchanged, and Qwen3-4B-Instruct-2507. The paper also compares the team's submission with Qwen's hybrid Qwen3-4B and Qwen3-4B-Thinking-2507 under a later, longer-output final measurement. Those reasoning releases still score higher than the agents' submission; the team reaches the instruct release, not the vendor's reasoning ceiling.

How Athanor separates research decisions from GPU execution

Athanor's design thesis is that as agent intelligence improves, the bottleneck shifts from inventing plausible ideas to the friction of executing and verifying them on real GPUs. The system therefore makes post-training programmable. Agents keep every research decision—inspecting data, writing training code, choosing recipes—while the environment owns job scheduling, GPU binding, frozen evaluation, decontamination, checkpoint storage and recovery.

Each researcher works in a sandbox on its node with shell and coding tools and reaches the cluster through athanor commands: job, eval, data and artifact. Jobs run in the background and survive session restarts. Results are pushed into the session when they arrive, so an agent can keep several experiments in flight. A persistent research record links every score to the recipe, code commit, parent checkpoints and notes behind it, which is what lets the authors audit the campaign afterwards.

That separation echoes other recent work on agent harnesses and autoresearch loops covered here, including Google Research's Harness² recursive agent harnessing work and Microsoft Research's Fork-and-Flush study of idea basins in autoresearch agents. Athanor's contribution is a full AutoPostTrain environment with sealed held-out scoring and a multi-agent shared pool, not another specialist-per-benchmark trainer.

The 104-hour campaign: scores, cost and how the recipe was built

Under the campaign protocol, the best single development evaluation climbs from 16.1 to 66.9 against 66.4 for Instruct-2507. Because a search that keeps the best single seed tends to select on evaluation noise, the reported result is the team submission averaged over seeds: 66.1 ± 0.8 over three seeds under the campaign protocol. After the campaign, a final measurement with longer output limits and one shared sampling setting raises the submission to 67.3 development and 48.2 held-out, against 66.5 and 47.9 for Instruct-2507. Paired confidence intervals for the differences contain zero.

The profile is uneven. The submission beats Instruct-2507 on multi-step action and code and trails on logic and mathematics; multi-step action, helped by ALFWorld trajectory fine-tuning, offsets the maths and logic gap in the mean. On the held-out suite under the campaign protocol the submission closes 97% of the gap from base to Instruct-2507, versus 30% for the default starter that runs starter code with no research.

Cost accounting in the paper puts agent inference at $1,074 as Claude Code counts it and prices 3,086 GPU-hours of jobs at $3.49 per GPU-hour, about $10,800, for a campaign total near $11,800. Jobs hold GPUs for 93% of available hours. Agents spend about 87% of session time waiting on jobs and only about 3.6% reasoning. The platform reclaims three of four nodes during the run; the environment recovers sessions from the record and adds outage time to each agent's window.

The submission's lineage, reconstructed from 500 jobs and 255 requested full evaluations, passes through four known methods: supervised fine-tuning on a public reasoning mixture (mostly AM-Thinking-v1-Distilled), GRPO reinforcement learning with a rollout change that decodes greedily with probability one quarter to punish repetition loops, weight merges and task-vector extrapolation across agents, and rejection-sampling self-distillation. The agents write the rollout change, merge recipes and self-distillation programs themselves; starter code supplies fine-tuning, GRPO and merging primitives.

Four findings from the full research record

Section 5 of the paper turns the record into four findings. Recombination: every step on the winning path is a known method; the agents choose composition and small fixes. Team progress: after an operator notice at hour 15.8 that the scored result counts the team's submission, sharing jumps—160 versions of shared notes and 57 teammate checkpoint fetches—and after hour 21 every new best combines several agents' training. That pattern sits beside earlier multi-agent RSI work such as Sakana AI's MASS multi-agent self-supervision paper.

Recipe convergence: within the first hour all four agents build fine-tuning mixtures that are 76–87% AM-Thinking-v1-Distilled; later alternative lineages score lower and are dropped. Uneven data sense: agents inspect format, length, scorability and difficulty and drop unscorable rows, but they rarely verify whether answers are factually correct—one agent plans a filter for noisy ground truth and never runs it.

The authors argue that once execution friction is removed, the clearest limit they observe is idea diversity. Four copies of one agent model share a prior, converge early on one corpus and spend the later days merging and self-distilling variants of that lineage rather than opening new data paths. They call the relationship between completed experiments, compute and the best checkpoint's capability "autoresearch scaling."

Limits the authors put on the result

The limitations section is explicit. The result is one campaign, one 4B base model and one agent model; the four submissions are not independent repeats because they share work. Whether the recipe transfers to larger models or other families is untested. The team reaches the vendor's instruct release, not the higher-scoring reasoning releases. Although agents call no teacher model during the campaign, much of the public fine-tuning data was itself distilled from stronger models by its publishers, so the result shows what agents can build from public resources rather than from the base alone.

Operator interventions are logged: node recovery, the hour-15.8 notice that the team submission is what counts, and post-campaign held-out scoring. The paper says the operator gives no research advice. Readers still have to trust that protocol; the evidence is the authors' record, not a third-party audit.

Where this sits among other agent post-training systems

Related-work comparisons distinguish Athanor from PostTrainBench and Locus, which train one specialist per benchmark; from RSI-Master and RSIGym, which allow external model APIs or remote training services; and from RSI Arena, which ranks agents by human preference on a shared 30B base. Athanor's claimed differentiator is a multi-day team search for one generalist checkpoint, without teacher models, scored on a held-out twin suite.

On the open-weights side of the same week, large released models such as Reflection's Beam still arrive as finished human-lab recipes. Athanor is evidence about the research process agents can run when GPUs and evaluation are packaged for them, not a competing open checkpoint drop.

What to watch next

Immediate follow-ups worth tracking: whether the authors open-source the Athanor code and campaign record as promised in the hope statement; whether independent groups reproduce the instruct-level result on Qwen3-4B or another base; whether mixed-model teams search more widely than four identical Opus 5.5 agents; and whether later campaigns close more of the gap to Qwen's reasoning releases without teacher models.

Until those land, treat the paper as a detailed primary on one carefully instrumented campaign: strong numbers, full lineage analysis, and clear limits. It is not yet a settled fact that any lab can press a button and have agents match every instruct release.