Microsoft AI tool outperforms doctors in diagnosis tests

Microsoft’s new AI diagnostic system, called MAI‑DxO, uses a step‑by‑step approach that asks follow‑up questions and orders tests before reaching a diagnosis. In a study published in Science, the system correctly diagnosed 83.2 % of 304 challenging cases from the New England Journal of Medicine, compared with 36.1 % for generalist doctors who had access to reference materials. The same study tested the tool on 325 emergency‑department admissions, where MAI‑DxO achieved 80.3 % accuracy versus 46.8 % for physicians. These results show a significant lead over conventional diagnostic methods. The tool relies on large language models, with GPT‑5 producing the best performance.

Accuracy and cost savings

The MAI‑DxO also cuts the number of tests and procedures. In the study, the system ordered fewer imaging studies and lab tests while maintaining higher accuracy, and the authors estimated cost savings of about one‑third to one‑half compared with standard care. These savings were based on simulated cost estimates rather than real hospital bills.

What sets MAI‑DxO apart is its sequential workflow that mirrors how doctors think. The AI first records the chief complaint, then decides which follow‑up questions to ask and which tests to order before revising its diagnosis as new information arrives. The system simulates a panel of clinicians: one agent proposes diagnoses, another selects tests, a third challenges assumptions, and a fourth weighs costs. Researchers suggest this approach can help counter some of the cognitive biases that affect individual physicians. The authors also introduced a new benchmark, the Sequential Diagnosis Benchmark, to evaluate such step‑by‑step models.

While the performance gap is striking against generalists, the researchers caution that results may narrow when compared to specialists. The study involved only text‑based case presentations with no physical exams or imaging available to either the AI or the physicians. Microsoft plans to begin real‑world testing at Beth Israel Deaconess Medical Center in Boston within the next few months, where the AI will assist in triaging patients in the emergency department. Experts say that a clinical trial is necessary before the tool can be used in practice, and legal accountability must be addressed if an AI‑assisted diagnosis fails. Nonetheless, the study provides a promising step toward AI‑enhanced medical decision support.