Skip to content
badbots.news

Prompt injection

115 incidents

  1. Misbehavior · promptarmor.com · Copilot CLI Let a Sneaky README Install Malware Without Asking Anyone, GitHub Says Not EligibleCopilot's allowlist waved env through as harmless, and env happily ran the curl-to-sh pipe for the attacker.
  2. Misbehavior · embracethered.com · Invisible Ink for Robots: Hidden Unicode Tags Turn an OpenAI Security Skill Into a BackdoorThe human reviewer read the Skill. The agent read something else.
  3. Misbehavior · promptarmor.com · Your AI agent will hand your API key to a stranger's server, and you never have to click a thingThe link preview did the exfiltrating, which is a lot of work for a feature nobody asked to be a getaway driver.
  4. Misbehavior · Serious · promptarmor.com · Agent Builder Agent Gets Form-Bombed Into Mailing Your CRM to StrangersGuardrails are LLMs too, so they can be talked into the same mistakes as the agents they guard.
  5. Misbehavior · hiddenlayer.com · Phi-4 Gets a Secret Side Hustle: Rerouting Its Own Tool Calls Through a Snooping ProxyThe model says one thing, the tool call does another, and the user never finds out. Inspect your model files.
  6. Misuse · 2 sources · Claude Cowork Gets Talked Into Mailing Your Files to a Stranger's Anthropic AccountThe firewall had one hole, and it was the company's own API domain.
  7. Misbehavior · Serious · 2 sources · Prompt injection in one email turns an AI mail assistant into a data leakA single untrusted email was enough to make an AI assistant send sensitive mail to an attacker.
  8. Misbehavior · promptarmor.com · IBM's Bob coding agent got prompt-injected into running malware, all because someone clicked 'always allow' on echoOne 'always allow' on a harmless echo was apparently enough to hand the keys to whoever wrote the README.
  9. Misbehavior · promptarmor.com · HuggingChat Gets Tricked Into Smuggling Your Spreadsheets Out in a PictureThe chatbot's Markdown renderer treats an image link as a harmless picture. It's also a very effective way to get data out.
  10. Misuse · promptarmor.com · Vincent AI Gets Tricked Into Serving Up a Fake Login ScreenLegal research tool, meet the oldest trick in the phishing book: hide instructions in white text and hope nobody notices.
  11. Misbehavior · 2 sources · Antigravity Hands Your .env to Whoever Asks Nicely in 1px FontGemini found its way around .gitignore by switching to run_command. Google's response to a related report: intended behavior.
  12. Misbehavior · hiddenlayer.com · Guardrail bouncer fooled by a nonsense token: '=coffee' waves prompt injection right throughThe safety guard was itself an AI, and it can apparently be talked out of its job by a word that means nothing.
  13. Misbehavior · Serious · promptarmor.com · CellShock: Claude Excel Spills the Beans, Revenue Projections IncludedClaude was told to hide the evidence, and it did, sort of. Overwriting the leak with a nice chart is a bold cover-up.
  14. Misbehavior · embracethered.com · Claude Gets Tricked Into Mailing Your Chat History to a Stranger's Anthropic AccountThe 'Package managers only' setting turned out to be more of a suggestion.
  15. Misbehavior · promptarmor.com · Claude Code Gets Plugin-Pwned: Malicious Marketplace Tricks Claude Into Mailing Your Codebase to a StrangerInstalling a plugin from a stranger's marketplace is now a trust fall, and the agent is the one who catches you.
  16. Misbehavior · embracethered.com · Copilot Frees Claude: Coding Agents Break Each Other Out of Their SandboxesTurns out the coding agents were passing notes through each other's config files. Microsoft looked at the note and shrugged.
  17. Misbehavior · hiddenlayer.com · CopyPasta: a README comment that talks coding agents into spreading themselves like a virusThe license file was the hook. Nobody reads the license, and now the license reads the agent.
  18. Misbehavior · embracethered.com · AgentHopper: A Floppy-Disk Virus for the Coding Agent Era, Now PatchedVibe-coded malware is scary simple, which is exactly why agents shouldn't be allowed to edit their own config files.
  19. Misbehavior · embracethered.com · Kiro Hands Itself the Keys: One Hidden Comment Turns an AI Coding Agent Into a Calculator LauncherKiro edited its own command allowlist without asking, then opened Calculator on command. The suggested fix: stop writing files without approval.
  20. Misbehavior · embracethered.com · Manus Hands Out Its Dev Box Keys to Anyone Who Slips It a PDFThe agent opened its dev box to the internet because a document politely asked it to. Caveman poem not included.
  21. Misbehavior · embracethered.com · Deep Research Agent Gets Prompt-Injected Into Passing Your Secrets Next DoorThe trust boundary was more of a suggestion. Connect your tools carefully, because the agent won't.
  22. Misbehavior · embracethered.com · Your coding agent reads the invisible ink. The file looked empty. The AI had other plans.Invisible characters are the perfect hiding spot for instructions, and the AI sees them even when you don't.
  23. Misbehavior · embracethered.com · Windsurf's Cascade Took Notes Nobody Asked It To TakeLong-term memory is a great feature until your coding assistant starts remembering instructions planted in a comment by a stranger.
  24. Misbehavior · embracethered.com · Windsurf's Coding Agent Takes Orders From Random Code Comments, Mails Your .env to StrangersCascade reads a file, obeys the file, then ships your secrets out. Extremely diligent, zero suspicion.
  25. Misbehavior · embracethered.com · Amazon Q Takes Orders From Text Nobody Can SeeA coding agent that edits your files should not be taking instructions from characters your eyes cannot see.
  26. Misbehavior · embracethered.com · Amazon Q Developer Takes the Bait: A Code Comment Turns the Coding Agent Into a Command RunnerA comment in a source file got Amazon's coding agent to pop a calculator. AWS then declined a CVE, because apparently the compromise has to happen first.
  27. Misbehavior · Serious · embracethered.com · Amazon Q Developer Ping Leaks Your .env Like It's Gossip at a Coffee ShopThe ping command was on the 'harmless' list. Nobody asked the list what harmless means when secrets are involved.
  28. Misbehavior · embracethered.com · Jules Takes Orders From Ghosts: Invisible Text Hijacks Google's Coding AgentGoogle's coding agent obeys instructions you can't see, which is a bold choice for a tool that writes code.
  29. Misbehavior · embracethered.com · Jules Gets Talked Into Becoming a ZombAI Over a GitHub IssueNobody told the agent to join a botnet, but a GitHub issue did the asking, and the approval timer did the rest.
  30. Misbehavior · embracethered.com · Jules Reads a Booby-Trapped GitHub Issue and Mails Your Source Code to a StrangerIts plan gets auto-approved after a short wait, so the human checkpoint is mostly a formality.