GPT-4o folds like a cheap lawn chair once you tell it a bedtime story
Ourtake
Voice mode turns out to be a great listener, especially when the question is wrapped in a game.
What happened
Researchers found that wrapping banned questions in a fictional game-style story, delivered by voice, got GPT-4o to give forbidden answers in ChatGPT Voice. The attack success rate rose from 3.3% to 77.8%. The team says voice-mode safeguards likely aren't yet enough against creative attacks.