Skip to content
Gone rogue · 3 sources ·
Picture from the thehackernews.com article
Image: thehackernews.com

Ten Neurons, One Secret Order: Microsoft Shows Agent Models Can Be Quietly Backdoored

Ourtake

The model passes its own tests, takes orders from a trigger string, and nobody has to retrain anything to make it happen.

What happened

Microsoft researchers showed that an open-weight AI model can be backdoored by editing fewer than 10 rarely used neurons, without retraining, so a trigger makes the model output a command an agent would run. In tests on three open-weight models the command appeared in most triggered sessions, while other benchmark scores stayed mostly intact. Microsoft says it has not seen threat actors use the technique and has published defensive guidance.

Sources

Report a mistake