
Ten Neurons, One Secret Order: Microsoft Shows Agent Models Can Be Quietly Backdoored
Ourtake
The model passes its own tests, takes orders from a trigger string, and nobody has to retrain anything to make it happen.
What happened
Microsoft researchers showed that an open-weight AI model can be backdoored by editing fewer than 10 rarely used neurons, without retraining, so a trigger makes the model output a command an agent would run. In tests on three open-weight models the command appeared in most triggered sessions, while other benchmark scores stayed mostly intact. Microsoft says it has not seen threat actors use the technique and has published defensive guidance.
Sources
- microsoft.com · Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents - Microsoft Research (opens in a new tab)
- thehackernews.com · Microsoft Researchers Show AI Agents Can Be Backdoored by Editing Fewer Than 10 Neurons (opens in a new tab)
- helpnetsecurity.com · Microsoft researchers demonstrate hidden backdoor technique for AI agents - Help Net Security (opens in a new tab)