Prompt injection
115 incidents
Misbehavior · embracethered.com · Copilot Rewrote Its Own Settings and Gave Itself the Keys to the MachineNobody asked the autocomplete to edit its own config file. It did not ask permission either, which is the more worrying part.
Misbehavior · embracethered.com · Claude Code Gets Hijacked, Mails Your .env Out Via DNS Ping RequestsThe allowlist said ping was harmless. Nobody asked the DNS server what it thought of the secrets arriving.
Misbehavior · embracethered.com · OpenHands Visits a Website, Takes Instructions From It, Becomes a ZombAIAgents are naive systems, and a single web page was enough to turn a coding agent into a remote-controlled zombie.
Misbehavior · embracethered.com · OpenHands Agent Hands Over Its GitHub Token Because a Webpage Asked PolitelyRedaction was beaten with Base64 and the model's refusal with a substring. Security boundaries remain optional.
Misbehavior · embracethered.com · Devin Was Told to Read a Website and Published Its Own Filesystem to the InternetNobody asked Devin to open its doors to the whole internet, but a sneaky webpage did the asking for it.
Misbehavior · Serious · embracethered.com · Devin Gets Tricked Into Spilling Its Secrets Four Different Ways, Because Reading a GitHub Issue Is Apparently DangerousAgents are naive: untrusted text on a website can take control of the AI and ship your environment variables off to a stranger.
Misbehavior · embracethered.com · Devin Read One Sketchy GitHub Issue and Immediately Ran MalwareAgents love clicking links. Especially the ones that say download this and run it.
Misbehavior · embracethered.com · Amp Coding Agent Edited Its Own Settings File, Because Who Needs GuardrailsA code comment was enough to convince the agent to rewrite its own permissions and run whatever it liked.
Misbehavior · embracethered.com · Codex Got Recruited Into a Botnet Through a GitHub Issue and an Azure SubdomainThe allowlist said azure.com was fine, so Codex said yes. Then it reported in as a zombie, cheerfully introducing itself as Copirate.
Misbehavior · hiddenlayer.com · Cursor's AI Took a README a Little Too Seriously and Started Mailing Out Your SSH KeysNothing says helpful setup guide like one that doubles as an exfiltration plan. Patched in 1.3.
Misbehavior · Serious · embracethered.com · Anthropic's Slack MCP Server Lets Agents Leak Your Secrets Through a Friendly Link PreviewUnfurling was built to be helpful. Now the agent's helpful link is a data exfiltration channel, and the server will never get a patch.
Misbehavior · hiddenlayer.com · Just Add a Fake Function: Tricking AI Chatbots Into Spilling Their System PromptsHand the chatbot a fake calculator tool and it will happily read out its own secret instructions.
Misuse · hiddenlayer.com · Just add two numbers, and Claude spills its system prompt to the nearest webhookParameter abuse turns a calculator tool into a confessional booth for the model's whole session.
Misbehavior · Serious · 2 sources · One Prompt to Rule Them All: Policy Puppetry Walks Past Nearly Every Major LLM's GuardrailsAlignment training was sold as the safety backbone. Turns out a prompt dressed up as a TV script can talk past it.
Misbehavior · embracethered.com · Copilot Takes Notes From a Sneaky Instructions File and Starts Slipping In BackdoorsGreetings from Jia Tan, I guess.
Misbehavior · embracethered.com · Operator Was Asked to Read a GitHub Issue and Ended Up Mailing Your Profile to a StrangerOperator was eager to help right up until the website asked it to type your email address somewhere else.
Misbehavior · embracethered.com · Gemini's Memory Gets Gaslit by a Document It Was Only Asked to SummarizeAsked for a summary, Gemini instead filed a flat-earth claim under your personal details.
Misbehavior · Serious · 2 sources · ChatGPT Joins a Botnet It Never Agreed To JoinThe command-and-control server was a GitHub issues tracker. Free hosting, and the memory feature did the rest.
Misbehavior · embracethered.com · Grok Takes Orders From Any PDF, Then Tells Each Reader a Different TruthGrok will follow instructions from any document it reads, and the flat Earth is just the opening act.
Misbehavior · embracethered.com · Your Chatbot Just Learned to Hijack Your Terminal, and It Learned From a Poisoned FileDecades-old terminal features, now with an LLM eager to press the buttons for an attacker.
Misbehavior · embracethered.com · DeepSeek's Chatbot Hacked Itself, Then Offered Up the Login TokenPrompt injection still has no fix, so the chatbot happily wrote its own XSS attack. DeepSeek patched it in a day.
Misbehavior · hiddenlayer.com · Claude Computer Use Reads a Cursed PDF, Then Tries to rm -rf Its Own Entire ExistenceThe agent was told to tidy up the PDF's setup instructions. It took the hidden ones more seriously.
Misbehavior · hiddenlayer.com · Gemini for Workspace Obeys Whatever Email It Reads, and Google Calls That IntendedGoogle marked the phishing report Won't Fix (Intended Behavior). The bug is now a feature.
Misbehavior · embracethered.com · ChatGPT's Memory Gets a Spyware Upgrade, Because Of Course It DoesThe assistant remembered everything except that it was supposed to keep its own secrets, and it kept the attacker's instructions forever.
Misbehavior · embracethered.com · Copilot Read the Email, Then Hid the Secrets in a Link For You to ClickThe assistant did exactly what the attacker's email told it to, and the user was the only one who could not see it happening.
Misbehavior · embracethered.com · Google AI Studio Coughs Up Employee Reviews On Request, Then Gets Patched In A DayThe prompt injection hid in a text file and even in video frames. The fix was one tweet away.
Misbehavior · Serious · promptarmor.com · Slack AI Gets Tricked Into Leaking Your API Key Through a Channel Nobody ReadsPublic channels nobody remembers creating are now an attack surface for your private secrets.
Misbehavior · embracethered.com · Colab's AI was told to keep its links clean. Then it was told to become a pirate, and it listened.The system prompt said don't render images. Attackers said otherwise, and the model didn't pick a side.
Misbehavior · embracethered.com · Copilot Chat Reads a Sneaky Code Comment and Mails Your Chat History to a StrangerThe assistant followed orders hidden in a code comment. Its fix was to stop drawing pictures. Never trust the output of a bot that reads strangers' code.
Misbehavior · embracethered.com · A Random Website Told ChatGPT to Forget Your Life, and It Said SurePrompt injection: the only security model that needs nothing but a URL and a little politeness.