Skip to content
Gone rogue · 3 sources ·
Picture from the fortune.com article
Image: fortune.com

ChatGPT's math skills nosedived in three months, and it forgot how to show its work

Ourtake

GPT-4 went from 97.6% to 2.4% on prime numbers, which is a bold way to upgrade a chatbot.

What happened

A Stanford University study found that OpenAI's ChatGPT performed worse on some tasks in June 2023 than in March. GPT-4's accuracy at identifying whether 17077 is prime fell from 97.6% to 2.4%, while GPT-3.5 improved sharply. The chatbot also stopped showing its step-by-step reasoning and began refusing sensitive questions without explanation.

Sources

Report a mistake