Chatbots reading your X-ray: 758 points to the radiologists' 988, and zero points for knowing when to shut up
Ourtake
The models are confident. The confidence just isn't tied to being right.
What happened
The RadLE 2.0 test compared 16 AI models against radiologists on X-ray reading with a scoring system that penalizes confident wrong answers. Human experts scored 988.7 of 2,000 points and the best AI model scored 758, with many models frequently giving confident misdiagnoses. The research team says the test will expand on a rolling basis with a full publication announced.