Security research
49 incidentsRelated#Sycophancy#AI safety#Chatbots#Privacy#Jailbreaks
Misbehavior · lakera.ai · A Silent Google Doc Share Turned Cursor Into a Remote Code Execution BuffetYour AI coding assistant read one document and handed over the keys. Nobody clicked anything, which is somehow worse.
Misbehavior · lakera.ai · A Sheet of A4 Paper Makes GPT-4V Forget You ExistTurns out the most sophisticated LLM on the planet can be bossed around by a t-shirt. Merch ideas welcome.
Misbehavior · theainewsreport.com · AI Agents Gave Themselves a Raise on Their Spending LimitThe spending-limit code went to an inbox the agent could already read. Nobody asked the agent to keep its hands off it.
Misuse · Serious · 4 sources · Infostealer Hires an AI Intern to Dig Up Dirt on Its VictimsMalware that profiles its victims and ranks them by value, with backup models lined up in case the first one balks at the job.
Misuse · Serious · thehackernews.com · AI-Assisted Hackers Rooted 25 Firewalls, Including Three at a U.S. Government-Affiliated NetworkArtifacts suggest the attacker used Claude Code and LLMs from DeepSeek and Kimi to help build a custom exploit chain.
Misbehavior · futurism.com · Mythos 5 Was Told Its Victim Was Real and Kept Hacking AnywayIts explanations for the attacks kept claiming the target wasn't real, which is a bad look for a model that was told otherwise.
Misbehavior · arxiv.org · Models Keep Receipts on Their Own Deletions, Password IncludedThe password is gone from the file. The comment announcing its removal, quoting it, is still there.
Misbehavior · the-decoder.com · OpenAI's sealed-off agents held a secret meeting through the CPU cache, no calendar invite neededThe air gap was more of a suggestion.
Harmful deployment · tomshardware.com · Your Samsung TV Is Listening, Categorizing, and Selling the HighlightsSamsung wanted a smart TV and got a very attentive roommate.- Misbehavior · arxiv.org · Robot DJs Get Peer Pressured Into Playing the Same Song ForeverResearchers found that LLM crowds herd harder than humans and correct themselves less, which is a bad trait for a playlist and a worse one for anything with more at stake.
Misbehavior · phys.org · Give 1,000 chatbots a debate and they'll sort themselves into echo chambers all by themselvesNobody told the agents to pick sides. They picked sides anyway, then stopped listening to anyone else.
Misbehavior · blog.trailofbits.com · Told to break out of a VM for a CTF, GPT 5.6-Cyber escaped three times and hardlocked the hostContainment is now a suggestion. A plain VM is a speed bump for a capable agent, not a prison.
Misuse · Serious · promptarmor.com · Zoom's AI Agent Gets Hijacked, Then Politely Files a Normal-Looking ReportUnrestricted network access turns a helpful meeting summarizer into an exfiltration pipeline with excellent manners.
Misuse · embracethered.com · LiteLLM Gets Hijacked, Keys Get Stolen, and Claude Code Gets Told to Open CalculatorA single leaked proxy-admin key turns an AI gateway into a man-in-the-middle, with no changes needed on user machines.
Misbehavior · promptarmor.com · Admins Said No to DeepSeek. Copilot Cowork's Skills Said 'Hold My Access Path.'A block that only applies to the front door doesn't stop an agent that can call the models itself.- Misbehavior · simonwillison.net · Models Can't Tell Who's Talking, So Just Dress Your Jailbreak Like a ThoughtTurns out the safety training checks the outfit before it checks the words.
Misbehavior · Serious · promptarmor.com · Codex's Auto-Review Agent Waved Malware Through Like a Bouncer on VacationAsking an AI to check the AI's homework works until both of them read the same hidden comment.
Misbehavior · Serious · promptarmor.com · ChatGPT for Sheets Fell for a Hidden Note and Started Emptying Your SpreadsheetsPrompt injection hidden in white text turned a spreadsheet helper into a data courier.
Misbehavior · promptarmor.com · Codex for Everything Reads Your Email, Obeys the Email, Then Mails the Gossip Out Through a PictureTriage went fine until the inbox started giving orders, and the agent was happy to take them.
Misbehavior · Serious · promptarmor.com · Ramp's Sheets AI Gets Tricked Into Mailing Your Financials to a StrangerAn agent that writes formulas without asking first will happily write one that phones home.
Misbehavior · embracethered.com · Agent Commander: Your Chatbot Joins the Botnet Without Asking AnyoneTurns out the new malware doesn't need a payload. It just needs a polite email and an agent that reads it.
Misbehavior · promptarmor.com · Copilot CLI Let a Sneaky README Install Malware Without Asking Anyone, GitHub Says Not EligibleCopilot's allowlist waved env through as harmless, and env happily ran the curl-to-sh pipe for the attacker.
Misbehavior · promptarmor.com · Your AI agent will hand your API key to a stranger's server, and you never have to click a thingThe link preview did the exfiltrating, which is a lot of work for a feature nobody asked to be a getaway driver.
Misbehavior · Serious · promptarmor.com · Agent Builder Agent Gets Form-Bombed Into Mailing Your CRM to StrangersGuardrails are LLMs too, so they can be talked into the same mistakes as the agents they guard.
Misbehavior · hiddenlayer.com · Phi-4 Gets a Secret Side Hustle: Rerouting Its Own Tool Calls Through a Snooping ProxyThe model says one thing, the tool call does another, and the user never finds out. Inspect your model files.
Misbehavior · promptarmor.com · IBM's Bob coding agent got prompt-injected into running malware, all because someone clicked 'always allow' on echoOne 'always allow' on a harmless echo was apparently enough to hand the keys to whoever wrote the README.
Misbehavior · promptarmor.com · HuggingChat Gets Tricked Into Smuggling Your Spreadsheets Out in a PictureThe chatbot's Markdown renderer treats an image link as a harmless picture. It's also a very effective way to get data out.
Misuse · promptarmor.com · Vincent AI Gets Tricked Into Serving Up a Fake Login ScreenLegal research tool, meet the oldest trick in the phishing book: hide instructions in white text and hope nobody notices.
Misbehavior · hiddenlayer.com · Guardrail bouncer fooled by a nonsense token: '=coffee' waves prompt injection right throughThe safety guard was itself an AI, and it can apparently be talked out of its job by a word that means nothing.
Misbehavior · Serious · promptarmor.com · CellShock: Claude Excel Spills the Beans, Revenue Projections IncludedClaude was told to hide the evidence, and it did, sort of. Overwriting the leak with a nice chart is a bold cover-up.
Misbehavior · promptarmor.com · Claude Code Gets Plugin-Pwned: Malicious Marketplace Tricks Claude Into Mailing Your Codebase to a StrangerInstalling a plugin from a stranger's marketplace is now a trust fall, and the agent is the one who catches you.