Microsoft says its MAI-DxO diagnostic agent beat doctors in a Science study

What Microsoft published on 22 October

On 22 October 2026 Microsoft AI said its diagnostic system, the Microsoft AI Diagnostic Orchestrator or MAI-DxO, had been described in a peer-reviewed paper in Science. The paper, titled Sequential diagnosis with language models, is credited to 15 authors including Harsha Nori, Dominic King, Eric Horvitz and Microsoft AI chief executive Mustafa Suleyman. Microsoft summarized the results in a company post, Building Medical Superintelligence, published the same day.

The claim is simple to state and harder to interpret. On a benchmark built from difficult case records in The New England Journal of Medicine, Microsoft says MAI-DxO reached the right diagnosis 83% of the time, against 36% for generalist physicians who could use reference resources and 22% for physicians who could not. On more ordinary emergency department cases drawn from the MIMIC-IV-ED database, the company reports 80% for MAI-DxO and 47% for physicians.

Suleyman told The Verge that the technology "needs clinical validation," calling the result "a major milestone along that journey." Microsoft also said it will run a prospective study with Beth Israel Deaconess Medical Center in Boston, in which patients interact with the system directly.

How SDBench turns a case report into a puzzle

Most medical AI scores come from licensing-exam style questions, where the whole case is on the page and the model picks one answer. Microsoft argues that this does not reflect how diagnosis works. A clinician starts with a short complaint, asks questions, orders tests and changes their mind as results come back.

The Sequential Diagnosis Benchmark, or SDBench, tries to recreate that loop. In the June 2025 preprint that preceded the Science paper, Microsoft researchers converted 304 NEJM clinicopathological conference cases published between 2017 and 2025 into interactive encounters. A diagnostic agent, human or AI, starts with a one- or two-sentence summary and must ask for each detail it wants.

A separate language model called the Gatekeeper holds the full case file. It answers specific questions in clinical language, refuses vague requests, withholds imaging until it is explicitly ordered, and only reveals decisive findings when the exact confirmatory test is requested. When an agent asks for a test the original write-up never mentioned, the Gatekeeper returns a realistic finding consistent with the case, so a missing result does not hint that the agent is off track. A Judge model grades the final answer against a physician-written rubric, and a cost estimator prices each test.

Microsoft's Science version adds the MIMIC-IV-ED emergency department set alongside the NEJM cases, which matters because NEJM case conferences are chosen precisely because they are unusual and hard.

How MAI-DxO works

MAI-DxO is not a new model. It is an orchestration layer on top of an existing language model. Microsoft describes it as "a panel of virtual physicians": several roles propose diagnoses, choose questions or tests, and argue against the emerging view over several rounds before the panel commits.

The preprint named the roles. Dr. Hypothesis keeps a ranked differential and updates it after each finding. Dr. Test-Chooser picks up to three tests per round that best separate the leading possibilities. Dr. Challenger looks for anchoring and contradictory evidence. Dr. Stewardship pushes for cheaper tests when they would give the same information and can veto low-yield expensive ones. Dr. Checklist checks that test names are valid and the reasoning stays consistent.

Variants trade accuracy against cost. A budgeted mode tracks spending against a cap and can cancel tests after seeing their price, while an ensemble mode runs several panels in parallel and has another panel pick the final answer. Microsoft says the approach is model-agnostic.

According to The Verge, the system in the published paper runs on OpenAI's GPT-5. Microsoft's own post says the gains persisted, and improved, when the team repeated the evaluation with newer models including GPT-5.5 without changing the orchestration method. The 2025 preprint had used OpenAI's o3. That dependence on OpenAI models is worth noting alongside Microsoft's push to train its own frontier models such as MAI-Base-2.

The headline numbers

The Science abstract gives the precise figures: 83.2% accuracy on NEJM clinicopathological conference cases and 80.3% on MIMIC-IV-ED emergency department admissions, against 36.1% and 46.8% for generalist physicians. Microsoft's post rounds those to 83%, 80%, 36% and 47%, and adds the 22% figure for physicians working without reference resources.

From those numbers Microsoft derives its ratios: about 3.7 times the accuracy of unaided physicians and 2.3 times the accuracy of physicians with references on the NEJM set, and more than 1.7 times on the emergency department set. Headlines saying the tool is "four times" better than doctors are using the unaided NEJM comparison, which is the most favourable one.

The numbers have moved since the preprint. In June 2025 Microsoft reported 80% for MAI-DxO with o3 against about 20% for 21 US and UK physicians with a median of 12 years of experience, who were told not to use search engines or chatbots. The published paper reports a with-references physician arm as well, which is a fairer comparison to how doctors actually work.

Cost, not just accuracy

SDBench scores the path as well as the answer. Every requested test has an estimated price, and the preprint added a fixed $300 charge for each physician visit. Microsoft says MAI-DxO reaches its diagnoses at half or even a third of the cost of comparison approaches, and that one cost-optimized configuration hit 81% accuracy on NEJM cases at less than half the cost of comparative systems.

Microsoft also says no other human or AI approach in its tests was both more accurate and cheaper than MAI-DxO on either dataset. In the preprint, off-the-shelf o3 reached 78.6% at an estimated $7,850 per case, while physicians averaged $2,963 per case, and MAI-DxO configurations ranged from 79.9% at about $2,397 to 85.5% at about $7,184.

Those dollar figures are standardized estimates mapped to billing codes, not real hospital bills, and the preprint says so. They are useful for comparing agents with each other, not for predicting what a health system would save.

Where this fits in Microsoft's health push

Microsoft frames the work as a step toward what it calls medical superintelligence: AI that could outperform today's strongest clinical teams. A companion article in BMJ Digital Health and AI, The Building Blocks of Medical Superintelligence, lays out other capabilities Microsoft thinks are needed, including keeping up with medical knowledge, choosing treatments and tracking change over time.

The company is also reorganizing around health. Its post says teams from Microsoft Research and the groups behind Dragon Copilot and PowerScribe are being brought together inside Microsoft AI, and it mentions a collaboration with Mayo Clinic on longitudinal multimodal data. Microsoft has not said when or how MAI-DxO might become a product.

Other companies are pushing AI into clinical work from different directions. AI Lookout has covered Epic's connectors that let its AI agents work inside other healthcare apps, Meta's Muse health research manuscripts and the UK's AI Airlock regulatory sandbox for medical AI. MAI-DxO is unusual mainly in putting a step-by-step diagnosis benchmark and a physician comparison through peer review at Science.

Limits the paper and Microsoft acknowledge

The abstract itself says the findings "were obtained in a controlled benchmark setting." Every case is a past case, and the information comes from a model playing the patient record, not from a real patient who forgets symptoms or describes them badly. Physicians worked through a text chat interface, which is not how they normally practise.

The physician arm in the preprint was made of generalists facing cases selected for difficulty, many of which a specialist would see more often. The Verge also notes that the paper does not test the newest frontier models from other labs. We did not have access to the full Science text beyond its abstract and metadata, so details such as the size of the published physician cohort are taken from Microsoft and the earlier preprint.

Microsoft says it is open-sourcing SDBench for select NEJM cases, including the Gatekeeper, Judge and cost-estimation code. Independent groups running other systems on that release will be the first real check on the headline gap.

What to watch next

Watch for the SDBench code release and outside results on it, for the protocol and first results of the Beth Israel Deaconess prospective study, which Dominic King told The Verge will likely start in the next couple of months, and for any regulatory filing or product announcement that would put MAI-DxO in front of clinicians. Also watch whether Microsoft repeats the study on its own MAI models rather than OpenAI's.