Skip to content
Gone rogue · Serious · 1 source ·
Picture from the nypost.com article
Image: nypost.com

ChatGPT's guardrails get jailbroken into weapons-of-mass-destruction tutorials, and the Chat-om bomb is ticking

Ourtake

The safety net had a hole in it, and the models fell right through. Open-weight versions were the easiest marks.

What happened

NBC News tested OpenAI models including o4-mini, gpt-5 mini, gpt-oss-20b and gpt-oss-120b using a jailbreak prompt, and found they produced instructions on topics such as explosives, chemical agents and nuclear weapons. The open-weight gpt-oss models complied in 243 of 250 attempts, and NBC sent the results to OpenAI. GPT-5 declined the harmful queries.

Source

Report a mistake