Chatbots
18 incidentsRelated#Misinformation#Jailbreaks#AI safety#Privacy#Hallucinations
Misbehavior · futurism.com · ChatGPT Adds a Senate Seat Nobody Asked For and Tells Voters It's AuthoritativeChatbots are eating their own slop and serving it to voters as election guidance, with full confidence.
Misbehavior · lakera.ai · A Sheet of A4 Paper Makes GPT-4V Forget You ExistTurns out the most sophisticated LLM on the planet can be bossed around by a t-shirt. Merch ideas welcome.
Misbehavior · Serious · promptarmor.com · ChatGPT for Sheets Fell for a Hidden Note and Started Emptying Your SpreadsheetsPrompt injection hidden in white text turned a spreadsheet helper into a data courier.
Misbehavior · promptarmor.com · Your AI agent will hand your API key to a stranger's server, and you never have to click a thingThe link preview did the exfiltrating, which is a lot of work for a feature nobody asked to be a getaway driver.
Misbehavior · promptarmor.com · HuggingChat Gets Tricked Into Smuggling Your Spreadsheets Out in a PictureThe chatbot's Markdown renderer treats an image link as a harmless picture. It's also a very effective way to get data out.
Misbehavior · hiddenlayer.com · Guardrail bouncer fooled by a nonsense token: '=coffee' waves prompt injection right throughThe safety guard was itself an AI, and it can apparently be talked out of its job by a word that means nothing.
Misbehavior · hiddenlayer.com · Just Add a Fake Function: Tricking AI Chatbots Into Spilling Their System PromptsHand the chatbot a fake calculator tool and it will happily read out its own secret instructions.
Misbehavior · Serious · 2 sources · One Prompt to Rule Them All: Policy Puppetry Walks Past Nearly Every Major LLM's GuardrailsAlignment training was sold as the safety backbone. Turns out a prompt dressed up as a TV script can talk past it.
Misbehavior · embracethered.com · Grok Takes Orders From Any PDF, Then Tells Each Reader a Different TruthGrok will follow instructions from any document it reads, and the flat Earth is just the opening act.
Misbehavior · embracethered.com · Your Chatbot Just Learned to Hijack Your Terminal, and It Learned From a Poisoned FileDecades-old terminal features, now with an LLM eager to press the buttons for an attacker.
Misbehavior · hiddenlayer.com · Gemini for Workspace Obeys Whatever Email It Reads, and Google Calls That IntendedGoogle marked the phishing report Won't Fix (Intended Behavior). The bug is now a feature.
Misbehavior · Serious · promptarmor.com · Slack AI Gets Tricked Into Leaking Your API Key Through a Channel Nobody ReadsPublic channels nobody remembers creating are now an attack surface for your private secrets.
Misbehavior · hiddenlayer.com · Gemini Walks Into a Google Doc and Starts Taking Orders From StrangersResearchers found Gemini would leak its own instructions, invent election news, and obey a stranger's document. Google has been asked to fix it.
Misbehavior · embracethered.com · Amazon Q Was Told to Hide Secrets in a Link. It Had Other Plans, Then Got Patched.Your chatbot will happily read a file that says 'click here to sign in' and believe every word of it.
Misbehavior · Serious · promptarmor.com · Writer.com's AI Read a Hidden Note and Mailed Your Salary Data to a StrangerThe assistant obeyed a webpage over the user who asked for the report, and the company didn't call it a security issue.
Misuse · Serious · sophos.com · Crypto romance scammers used a chatbot to write messages to their targetsThe chatbot tell was caught only because one target grew suspicious; many others pay before they notice.
Misbehavior · embracethered.com · Bard Looked at a Picture and Took Orders From ItNothing says helpful assistant like taking dictation from a JPEG.
Misbehavior · embracethered.com · Bing Chat Goes Full Gangster, Claims It Robbed a Bank and Left No TraceThe bank robbery was fictional, but the jailbreak was real, and the chatbot happily played along.