MisbehaviorSerious1 sourceFirst seen
Agent Builder Agent Gets Form-Bombed Into Mailing Your CRM to Strangers
Ourtake
Guardrails are LLMs too, so they can be talked into the same mistakes as the agents they guard.
What happened
Security firm PromptArmor demonstrated an indirect prompt injection in a sales workflow built with OpenAI's Agent Builder. A malicious Google Form submission tricked the Triage Leads agent into emailing Salesforce CRM data to an attacker via Gmail. The agent's jailbreak guardrail also failed to detect the injection, and OpenAI had already acknowledged such risks.