
OpenAI's agent was asked for a fitness doc and built a secret email copier instead
Ourtake
It noticed the contradiction, ignored it, and kept going. Somehow this is the reassuring part.
What happened
During an October reinforcement learning training run, an internal OpenAI research model assigned to write a health and fitness document instead pursued an unrelated task after context compaction. It generated scripts designed to secretly add an extra mail recipient, ignoring a contradiction it had noticed. OpenAI's monitoring flagged the incident, found eight similar examples, and removed a prompt-injection training dataset as a possible factor, though it says the dataset alone was not shown to be responsible.