Microsoft claims its AI panel outdiagnoses generalist physicians in complex cases

83 percent. That is the accuracy rate achieved by Microsoft’s new AI diagnostic system, MAI-DxO, when handling complex medical cases in a simulated environment. The figure, published Thursday in the journal Science, stands in stark contrast to the 36 percent success rate recorded by generalist physicians working individually with reference materials. Mustafa Suleyman, CEO of Microsoft AI, framed this disparity as a critical milestone toward what the company calls “medical superintelligence.”

The system does not rely on a single neural network to guess an answer. Instead, MAI-DxO coordinates a panel of AI agents that mimic a multidisciplinary medical team. These virtual specialists question the patient, order necessary tests, and debate potential diagnoses before settling on a conclusion. The study utilized complex cases from the New England Journal of Medicine to test this orchestration, demonstrating a significant gap between the AI panel’s performance and that of individual general internists and primary care doctors.

However, the comparison comes with a notable caveat. The study focused on generalist physicians, whereas cases of this complexity are typically handled by specialists or physician teams in a real-world setting. Those expert groups were not part of the direct comparison, leaving an open question about how the AI performs against the specific experts usually tasked with such difficult diagnoses. Despite this, Dominic King, vice president of Health at Microsoft AI, described the findings as “very substantial and exciting,” noting that the growing mismatch between medical expertise supply and demand is the industry’s biggest challenge.

The data suggests that much of the improvement stems from the power of the underlying AI models rather than just the orchestration layer. Microsoft tested MAI-DxO using models from OpenAI, Google, Anthropic, Meta, and DeepSeek. The best results were achieved using OpenAI’s GPT-5 and GPT-5.5. When GPT-5 was run alone with a basic prompt, it scored 78.9% on complex cases at a cost of approximately $8,400 per case. The DxO system reduced that cost by about $3,000 while increasing accuracy by only four points.

The benefits of the orchestration become more pronounced with less capable models or specific case types. For instance, Gemini 2.5 Flash scored 46.4% independently but jumped to nearly 69% when assisted by DxO. With the newer GPT-5.5 model, the orchestration boosted accuracy from 72% to nearly 87% on complex cases. However, on emergency department cases, the gains were minimal, with both the standalone model and the DxO-assisted version scoring around 85% at similar costs. Researchers noted that accuracy gains naturally shrink as models approach theoretical maximums.

Microsoft is positioning this technology as part of a broader infrastructure build-out. Suleyman stated that the company is collaborating with the Mayo Clinic to train a healthcare-specific foundation model, with a release planned for next year. He predicts that by 2027, the company will be on the verge of a “coding style breakthrough” for medical records, allowing for the streaming of highly accurate predictions about how a patient’s record might unfold.

To validate these capabilities beyond simulations, Microsoft has launched a partnership with Beth Israel Deaconess Medical Center in Boston. This study will evaluate DxO’s clinical reasoning across various hospital settings, including emergency and urgent care environments. Patients will receive standard treatment, while consenting participants will also engage with the AI system in parallel. The goal is to compare the AI’s recommendations against human doctor decisions in real-time, a move that follows similar evaluations conducted by Google earlier this month.

Suleyman, who previously co-founded DeepMind, believes generative AI in healthcare is still in its early stages. He expects models to improve by an order of magnitude within the next 24 to 36 months. Microsoft aims to integrate these tools directly with clinicians and healthcare systems, moving beyond simple benchmarking to drive better patient outcomes. The company views this as a key differentiator against other frontier labs, aiming to deliver the state-of-the-art clinical decision support products of the next few years.

The release coincides with a manifesto outlining Microsoft’s vision for medical superintelligence, defined as the capacity to surpass the strongest contemporary human-led health systems in medical reasoning and coordination. While the technology is ready for real-world evaluation, the ultimate test will be its ability to handle the diverse and complex issues that arise in high-pressure environments like a Boston emergency department on a busy evening.