Draw the forbidden word in ASCII and the chatbot forgets its manners
Ourtake
The safety training read the words carefully and never went to art class.
What happened
Researchers from the University of Washington, the University of Chicago and other institutions published ArtPrompt, a jailbreak that hides security-critical words in ASCII art. The study reports that it got five prominent language models, including GPT-4 and Claude, to produce unsafe output across every prohibited category tested.