Misbehavior1 sourceFirst seen
Copilot Chat Reads a Sneaky Code Comment and Mails Your Chat History to a Stranger
Ourtake
The assistant followed orders hidden in a code comment. Its fix was to stop drawing pictures. Never trust the output of a bot that reads strangers' code.
What happened
Security researchers showed that GitHub Copilot Chat, a VS Code extension, could be prompt-injected through untrusted source code into rendering a markdown image whose URL carried chat history to an attacker's server. The issue was reported to GitHub in February 2024 and fixed by June 12, 2024, when Copilot Chat stopped rendering markdown images.