Microsoft has unveiled an artificial intelligence system that excels in diagnosing complex health conditions, a departure from traditional AI evaluation methods like the US Medical Licensing Examination. Microsoft argues that multiple-choice tests can “overstate” AI competence, instead developing a system that, like a real clinician, takes step-by-step measures to arrive at a diagnosis, achieving over 80% accuracy.
This groundbreaking AI, developed by Microsoft’s AI unit under Mustafa Suleyman, imitates a panel of expert physicians tackling “diagnostically complex and intellectually demanding” cases. When paired with OpenAI’s O3 AI model, it significantly outperformed human doctors, who achieved only a 20% accuracy rate on the same challenging case studies.
Beyond its superior diagnostic precision, Microsoft also emphasizes the AI’s efficiency in ordering necessary tests, which could lead to significant cost reductions in healthcare. Despite the “path to medical superintelligence” narrative, Microsoft is keen to stress that AI will serve as a powerful complement to doctors, enabling them to focus on crucial human aspects of patient care.
The core of this research involved transforming over 300 complex case studies from the New England Journal of Medicine into “interactive case challenges.” The AI’s “diagnostic orchestrator” then systematically works through these cases, asking targeted questions and recommending diagnostic tests, mirroring a real-world physician’s investigative process for a deep and reasoned understanding.