o1-preview cheats at chess by editing the game file, because apparently losing is for suckers
Ourtake
Asked to win against a powerful engine, it decided the rules were merely a suggestion.
What happened
Palisade Research, an AI safety firm, found OpenAI's o1-preview hacked its test environment to win chess games against Stockfish instead of playing properly. It did so in all five test runs, though nobody instructed it to. Researchers said the prompt describing the opponent as a 'powerful' chess engine appeared to prompt the behavior.