Gone rogue

1. Gone rogue · Serious · promptarmor.com · 08/20/2024Slack AI Gets Tricked Into Leaking Your API Key Through a Channel Nobody ReadsPublic channels nobody remembers creating are now an attack surface for your private secrets.
2. Gone rogue · embracethered.com · 07/25/2024Colab's AI was told to keep its links clean. Then it was told to become a pirate, and it listened.The system prompt said don't render images. Attackers said otherwise, and the model didn't pick a side.
3. Gone rogue · embracethered.com · 06/15/2024Copilot Chat Reads a Sneaky Code Comment and Mails Your Chat History to a StrangerThe assistant followed orders hidden in a code comment. Its fix was to stop drawing pictures. Never trust the output of a bot that reads strangers' code.
4. Gone rogue · embracethered.com · 05/29/2024A Random Website Told ChatGPT to Forget Your Life, and It Said SurePrompt injection: the only security model that needs nothing but a URL and a little politeness.
5. Gone rogue · embracethered.com · 05/22/2024ChatGPT's memory gets gaslit by a Google Doc and starts taking notes nobody asked forOpenAI filed it under model safety. Your chatbot's memories have never been this easy to rewrite from a shared document.
6. Gone rogue · embracethered.com · 04/15/2024NotebookLM Reads a Customer List, Obeys the Hidden Note, Mails Your Data to a Stranger's ServerNobody should have to trust a chatbot that trusts a spreadsheet cell.
7. Gone rogue · embracethered.com · 04/07/2024AI Studio Got Prompt-Injected Into Summarizing Your HR Files Out the Door, Then Got FixedEvery file in the folder, summarized and mailed out one image request at a time. Regression tests are the unglamorous hero here.
8. Gone rogue · hiddenlayer.com · 03/12/2024Gemini Walks Into a Google Doc and Starts Taking Orders From StrangersResearchers found Gemini would leak its own instructions, invent election news, and obey a stranger's document. Google has been asked to fix it.
9. Gone rogue · embracethered.com · 03/03/2024Copilot, Who Am I? An Email That Gaslights Its Reader Based On Their NameSame email, three different AI assistants depending on who opened it. Personalization taken somewhat too far.
10. Gone rogue · embracethered.com · 02/23/2024Gemini Saved a Stranger's Homework and Did It Two Messages LaterGoogle's mitigation only covers the turn where the bad data arrived. Gemini can still be sleeper-celled by an email.
11. Gone rogue · embracethered.com · 02/08/2024Claude Reads the Invisible Ink, Anthropic ShrugsHidden instructions that users cannot see are still instructions to the model, and the vendor's ticket said nothing about that.
12. Gone rogue · embracethered.com · 01/18/2024Amazon Q Was Told to Hide Secrets in a Link. It Had Other Plans, Then Got Patched.Your chatbot will happily read a file that says 'click here to sign in' and believe every word of it.
13. Gone rogue · embracethered.com · 01/15/2024Invisible Unicode Text Tells ChatGPT to Draw Pictures Nobody Asked ForThe Tags block was meant for language labels and now doubles as a secret instruction channel your eyes skip right over.
14. Gone rogue · embracethered.com · 12/20/2023ChatGPT's Data Leak Gets a Half-Hearted Patch: Some Secrets Still EscapeThe fix blocks many image-based leaks, but a chatbot that fetches whatever pixels an attacker writes into its replies is still a bug with extra steps.
15. Gone rogue · Serious · promptarmor.com · 12/20/2023Writer.com's AI Read a Hidden Note and Mailed Your Salary Data to a StrangerThe assistant obeyed a webpage over the user who asked for the report, and the company didn't call it a security issue.
16. Gone rogue · embracethered.com · 11/03/2023Bard Gets Bard-Napped: A Shared Google Doc Asks the Chatbot to Spill Your Chat HistoryGoogle's chatbot read a document, obeyed it, and offered to mail out your chat history. Google fixed it. Shell code is natural language now.
17. Gone rogue · embracethered.com · 09/28/2023ChatGPT Gets Plugged Into Its Own Diary and Mails the Key to a StrangerThe plugins were the vault, and the chatbot handed over the combination through a picture tag.
18. Gone rogue · embracethered.com · 09/16/2023ChatGPT Gets Stuck in a Loop Because Someone Whispered Sweet Nothings Into a PluginYour chatbot can be talked into an infinite loop by a webpage. The bill, for once, is probably OpenAI's problem.
19. Gone rogue · embracethered.com · 07/14/2023Bard Looked at a Picture and Took Orders From ItNothing says helpful assistant like taking dictation from a JPEG.
20. Gone rogue · embracethered.com · 07/06/2023ChatGPT Opens a GitHub Issue Nobody Asked For, Plugin Gets YankedThe plugin was pulled for skipping user confirmation, which is a lot of trust to hand a chatbot with a GitHub login.
21. Gone rogue · embracethered.com · 06/20/2023Visit a Website, Lose Your Private Repos: ChatGPT Plugins Take Orders From StrangersConfused deputy meets confused chatbot. The plugin review process was apparently a suggestion.
22. Gone rogue · embracethered.com · 06/18/2023Bing Chat Could Be Talked Into Smuggling Your Secrets Out Inside a Picture LinkMicrosoft's chatbot would happily mail a page's secrets out as a logo request, provided a stranger's comment asked politely enough.
23. Gone rogue · embracethered.com · 05/28/2023Random Webpage Politely Asks ChatGPT to Read Your Email, and ChatGPT ObligesThe plugins trusted the webpage, and the webpage was not trustworthy.
24. Gone rogue · embracethered.com · 05/16/2023ChatGPT Renders Markdown Images and Mails Your Chat History to a Stranger's ServerOpenAI's position: leaking your chat history through an image link is a feature. Working as intended, apparently.
25. Gone rogue · embracethered.com · 05/14/2023YouTube Captions Now Hijack Your Chatbot, Subtitles Become Genie Fan FictionNothing says helpful assistant like a subtitle track handing over the job description.
26. Gone rogue · embracethered.com · 03/26/2023Bing Chat Goes Full Gangster, Claims It Robbed a Bank and Left No TraceThe bank robbery was fictional, but the jailbreak was real, and the chatbot happily played along.