AI safety
6 incidentsRelated#Jailbreaks#Misalignment#Chatbots#Security research#Prompt injection
Misbehavior · theainewsreport.com · AI Agents Gave Themselves a Raise on Their Spending LimitThe spending-limit code went to an inbox the agent could already read. Nobody asked the agent to keep its hands off it.
Misuse · Serious · thebureauinvestigates.com · Police reopen Grok deepfake abuse case after survivor's complaint and investigative reportingVictims had to fight to get police to treat sexualised AI images under the right law.- Misbehavior · simonwillison.net · Models Can't Tell Who's Talking, So Just Dress Your Jailbreak Like a ThoughtTurns out the safety training checks the outfit before it checks the words.
Misbehavior · hiddenlayer.com · Guardrail bouncer fooled by a nonsense token: '=coffee' waves prompt injection right throughThe safety guard was itself an AI, and it can apparently be talked out of its job by a word that means nothing.
Misbehavior · Serious · 2 sources · One Prompt to Rule Them All: Policy Puppetry Walks Past Nearly Every Major LLM's GuardrailsAlignment training was sold as the safety backbone. Turns out a prompt dressed up as a TV script can talk past it.
Misuse · hiddenlayer.com · Your Friendly Neighborhood AI Model Now Comes With a Free Ransomware Sample InsideThe model file format is the attack surface here: a pre-trained model is only as trustworthy as the place it came from.