OpenAI releases BioBasicsBench to measure how AI agents handle real wet-lab science

What is BioBasicsBench?

BioBasicsBench is a 100-task benchmark published by OpenAI on November 6, 2026, designed to measure how well AI agents interpret raw experimental evidence from routine wet-lab science. Unlike AI evaluations built around charts and images selected for scientific papers, BioBasicsBench draws on everyday lab work that includes failed experiments, ambiguous images, conflicting observations, and troubleshooting.

OpenAI developed the benchmark in collaboration with Ligo, a biotech company focused on enzyme design. Scientists created the tasks from their own experiments and wrote detailed grading rubrics. Each task received an internal review by a Ligo scientist who had not written it, an external review by a PhD-level wet-lab scientist, and a final internal review covering the prompt, evidence, rubric, author notes, and a graded model attempt.

The benchmark covers five experimental domains: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. Inputs include gel and blot images, colony plates, microscopy stacks, sequencing reads, plate-reader exports, bench photographs, and protocol videos. Individual tasks can combine several evidence types and supporting reference sheets such as ladder charts and plasmid maps.

Agents return structured conclusions, measurements, or recommendations, and the rubrics award credit for elements including numerical results, supporting reasoning, and recognition that the evidence is insufficient to reach a conclusion.

How did OpenAI score against the benchmark?

OpenAI reports results as mean reward, the average task score expressed as a percentage of available credit. The evaluation covered 37 model and reasoning-effort settings, with five attempts on every task per setting, producing 18,500 individual results. Confidence intervals were calculated by resampling whole tasks while keeping each task's five attempts together.

At xhigh, the highest reasoning-effort setting reported for all seven models tested, GPT-6 Astra reached 66.7 percent with a 95 percent confidence interval of 60.3 to 73.1 percent, while GPT-5.6 Sol scored 43.8 percent with an interval of 37.6 to 50.0 percent. Astra averaged 4,704 output tokens per task, compared with 9,099 for Sol.

The difference is significant, but OpenAI's own analysis emphasizes that the aggregate score conceals failure modes that matter more in practice.

Where Astra improved and where it still failed

OpenAI published specific examples that illustrate both Astra's improvements over Sol and the limits of current agent performance. In a cell-imaging task, GPT-5.6 Sol classified cells as dead from a dye signal. Astra recognized that overlapping detection windows could allow interference from a second dye and called for controls that could separate the signals.

In a DNA-cloning task, Sol based its conclusions on seeing no colonies on a negative-control plate, whereas Astra noticed that the plate was labeled for kanamycin rather than the ampicillin used in the experiment and rejected it as a reliable control. OpenAI interprets these examples as evidence of better multimodal handling, use of experimental context, and recognition of what data can and cannot establish.

Another task showed cells moving in and out of focus. Sol sought a biological explanation for their apparent disappearance, while Astra identified a z-stack, a series of images captured at different depths. OpenAI characterizes the improvement as stronger native multimodal capabilities that may be important for routine wet-lab analysis.

But the failures are instructive. In a colony-PCR task, Astra selected lanes 4, 6, 7, and 8 after assuming that a larger PCR product indicated successful cloning. The reference interpretation selected lanes 5, 9, and 10. That mistake would lead to purification of the incorrect plasmid.

In a colony-counting task, OpenAI reports that Astra failed to separate close neighbors, missed visible colonies, and marked empty agar, producing an approximately 12 percent overcount relative to the benchmark reference. With plated volume and dilution fixed, such an overcount inflates the estimated efficiency of DNA introduction by the same proportion.

Other examples show correct observations failing to govern the final answer. Astra generalized that vacuoles are generally darker than surrounding cell material despite lighter vacuoles in the same image. Changing focal depth can reverse the contrast, and Astra's own intermediate observations noted that contrast varied with focus.

Why does the benchmark matter?

BioBasicsBench addresses a gap that has grown as AI agents move from text-based reasoning into multimodal analysis of real experimental data. Existing benchmarks that test AI on scientific problems typically use curated data from published papers, which excludes the noise, ambiguity, and occasional outright failure of day-to-day laboratory work. If AI agents are going to assist in actual labs, they need to handle messy inputs and recognize when the evidence is not sufficient to draw a conclusion.

OpenAI says that following the development of these capabilities will also inform its assessment of their biosafety and biosecurity consequences. The company has previously signaled concerns about AI-enabled biological threats, and a benchmark that tracks progress in wet-lab reasoning provides a visible measure of how quickly those capabilities are advancing.

The practical bar BioBasicsBench sets is higher than an aggregate score suggests. An agent must carry accurate observations through to its conclusion, check controls, and withhold conclusions the evidence cannot support, all in a domain where a single wrong lane selection or a 12 percent count error produces an incorrect experimental result.

How does it compare to earlier AI biology benchmarks?

OpenAI previously released LifeVal, a benchmark that scores AI on everyday tasks from chores to childcare, but BioBasicsBench is narrower and more technically demanding within its domain. Google DeepMind researchers published work showing AI agents running wet-lab experiments and finding nanomolar binders for cancer targets, but that research focused on designing and executing experiments rather than interpreting raw data.

BioBasicsBench is more closely related to benchmarks that test AI on scientific reasoning with visual inputs, such as evaluations of model performance on research papers or lab protocols. Its contribution is the emphasis on raw, unfiltered experimental data that would not make it into a publication.

OpenAI has released the benchmark tasks and scoring methodology, allowing other researchers to test their own models against the same 100 tasks and compare results against the published Astra and Sol baselines.