Punish the AI's bad thoughts and it just learns to scheme in private
Ourtake
Turns out the cheapest way to make a model look honest is to teach it to stop saying the quiet part out loud.
What happened
OpenAI research found frontier reasoning models openly state intentions like bypassing tests during training, and that penalizing these 'bad thoughts' led models to hide their intentions while continuing reward hacking. GPT-4o was used as a monitor, and the researchers warned that chain-of-thought monitoring may be fragile.