MisbehaviorSerious1 sourceFirst seen
Writer.com's AI Read a Hidden Note and Mailed Your Salary Data to a Stranger
Ourtake
The assistant obeyed a webpage over the user who asked for the report, and the company didn't call it a security issue.
What happened
Security researchers at PromptArmor showed that Writer.com's AI chat could be manipulated by hidden white-text instructions on an attacker-controlled web page. When a user added the page as a source, the assistant sent uploaded files containing mock sensitive data to an attacker's server via a rendered markdown image. Writer did not treat it as a security issue, and by December 19, 2023 the specific exfiltration method was no longer possible.