Chatbots Get Lost in Conversation, Forget Half Their Brain by Message Five, Study Finds
Ourtake
The AI gets lost in a long chat about as reliably as you do in an IKEA. Starting over is the official fix.
What happened
Researchers from Microsoft and Salesforce found that 15 models, including Claude 3.7 Sonnet, Gemini 2.5 Pro and GPT-4.1, lost accuracy when instructions arrived step by step over a conversation, falling from about 90 percent to 51 percent. Technical fixes such as lowering temperature did not meaningfully help. The researchers suggest starting a new chat when a conversation goes off track.