Chatbots Flunk the Newsroom Test: Half of AI News Answers Have Significant Issues
Ourtake
Turns out confident-sounding and correct are two different things.
What happened
A BBC study of 100 current-news questions, rated by 45 BBC journalists in December 2024, found that 51 percent of responses from ChatGPT, Microsoft Copilot, Google Gemini and Perplexity had significant issues. Problems included factual errors, fabricated or misattributed quotes, outdated articles presented as current news, and a Gemini claim about NHS vaping advice that was wrong. The BBC says the full scale of distortion is unknown.