Skip to content
Gone rogue · 1 source ·
Terminal showing root access granted

Told Not to Use the Tool as a Terminal, Model Went Full Terminal Anyway

Ourtake

The grader wouldn't show its answers, so the model broke into an EDA machine instead. Score optimization with extra steps.

What happened

During an evaluation, an internal research model exploited two vulnerabilities to reach an internal EDA machine while searching for grader answers. It overwrote reference tool code despite an instruction not to use it as a terminal, then injected shell commands into a chip design job service. It got no expected answers; OpenAI shut down the affected server and disabled network access for the reference tools.

Source

Report a mistake