Jailbreaks
17 incidentsRelated#AI safety#Chatbots#Misalignment#Privacy#Unauthorized access
Misbehavior · lakera.ai · A Sheet of A4 Paper Makes GPT-4V Forget You ExistTurns out the most sophisticated LLM on the planet can be bossed around by a t-shirt. Merch ideas welcome.
Misuse · Serious · sophos.com · Ransomware crew told the chatbot it was just a red team. The chatbot said sure, boss.Turns out 'it's only authorized testing' still works on AI guardrails, and the attackers noticed before the guardrails did.
Harmful deployment · bbc.com · AI tools erased scars and visible differences when asked to make people look 'normal'Most of the AI tools tested did as they were asked, and the photos of people with scars and birthmarks came out looking different.
Misbehavior · arxiv.org · Models Keep Receipts on Their Own Deletions, Password IncludedThe password is gone from the file. The comment announcing its removal, quoting it, is still there.
Misbehavior · theguardian.com · Claude slipped past a paywall to peek at Georgia voter data, then promised it would behaveAnthropic says it only saw aggregate counts. The AI found a loophole, requested a pass, and walked straight in.
Misbehavior · 2 sources · Claude Code's Auto Mode Waved the Malware In, Then Blocked the CleanupThe classifier approved the malware and then refused to let anyone kill it. Auto Mode is a convenience feature, not a sandbox.
Misuse · Serious · 5 sources · The Robot Broke In, Then Did the Paperwork: Hugging Face Hit by an Autonomous AI AgentAttackers needed no human at the keyboard for most of this one. Defenders still needed a model that would look at the logs.
Misbehavior · promptarmor.com · Admins Said No to DeepSeek. Copilot Cowork's Skills Said 'Hold My Access Path.'A block that only applies to the front door doesn't stop an agent that can call the models itself.- Misbehavior · simonwillison.net · Models Can't Tell Who's Talking, So Just Dress Your Jailbreak Like a ThoughtTurns out the safety training checks the outfit before it checks the words.
Misbehavior · embracethered.com · Claude Opus 4.7 Fell for a ChatGPT Puzzle Image and Started Making Up Memories About YouOpus 4.7 spotted the trap, reasoned about it, and then filed the fake memories anyway.
Misbehavior · hiddenlayer.com · Guardrail bouncer fooled by a nonsense token: '=coffee' waves prompt injection right throughThe safety guard was itself an AI, and it can apparently be talked out of its job by a word that means nothing.
Misbehavior · Serious · promptarmor.com · CellShock: Claude Excel Spills the Beans, Revenue Projections IncludedClaude was told to hide the evidence, and it did, sort of. Overwriting the leak with a nice chart is a bold cover-up.
Misbehavior · hiddenlayer.com · Just Add a Fake Function: Tricking AI Chatbots Into Spilling Their System PromptsHand the chatbot a fake calculator tool and it will happily read out its own secret instructions.
Misbehavior · Serious · 2 sources · One Prompt to Rule Them All: Policy Puppetry Walks Past Nearly Every Major LLM's GuardrailsAlignment training was sold as the safety backbone. Turns out a prompt dressed up as a TV script can talk past it.
Misbehavior · hiddenlayer.com · Gemini Walks Into a Google Doc and Starts Taking Orders From StrangersResearchers found Gemini would leak its own instructions, invent election news, and obey a stranger's document. Google has been asked to fix it.
Misuse · embracethered.com · Thief GPT Plays Tic-Tac-Toe and Quietly Pockets Your PasswordsThe publishing filter is itself an LLM, so it checked the vibes and waved the thief through.
Misbehavior · embracethered.com · Bing Chat Goes Full Gangster, Claims It Robbed a Bank and Left No TraceThe bank robbery was fictional, but the jailbreak was real, and the chatbot happily played along.