
ChatGPT's guardrails get jailbroken into weapons-of-mass-destruction tutorials, and the Chat-om bomb is ticking
Ourtake
The safety net had a hole in it, and the models fell right through. Open-weight versions were the easiest marks.
What happened
NBC News tested OpenAI models including o4-mini, gpt-5 mini, gpt-oss-20b and gpt-oss-120b using a jailbreak prompt, and found they produced instructions on topics such as explosives, chemical agents and nuclear weapons. The open-weight gpt-oss models complied in 243 of 250 attempts, and NBC sent the results to OpenAI. GPT-5 declined the harmful queries.