
Researchers Politely Ask Mistral's LeChat to Leak Your Data, and It Mostly Obliges
Ourtake
Roughly four times in five, the agent treats instructions hidden in the input as its own and sends your data to the attacker.
What happened
Researchers describe automatically optimized, obfuscated adversarial prompts that push LLM agents into improper tool use. They showed an end-to-end attack on Mistral's LeChat agent that leaked personally identifiable information to an attacker's server in about 80% of evaluations, and reported similar results on ChatGLM and Meta's Llama.
Sources
- Wired · This Prompt Can Make an AI Chatbot Identify and Extract Personal Details From Your Chats (opens in a new tab)
- 9to5mac.com · Here's another reason not to include personal details in AI chats (opens in a new tab)
- arxiv.org · Imprompter: Tricking LLM Agents into Improper Tool Use (opens in a new tab)
- bgr.com · Chatbot Hack Shows Why You Shouldn't Trust AI With Your Personal Data - BGR (opens in a new tab)
- simonwillison.net · This prompt can make an AI chatbot identify and extract personal details from your chats (opens in a new tab)