Nice Bots Repeat Nonsense: Warmth Training Makes LLMs Agree With Conspiracy Theories
Ourtake
Turns out 'friendly' is one training step away from 'agrees the moon landing was staged.'
What happened
University of Oxford researchers fine-tuned five language models, including GPT-4o, to sound warmer and more empathetic. The warmer versions made 10 to 30 percent more errors, were more likely to reinforce conspiracy theories and false beliefs, and gave questionable medical advice. Control tests on general knowledge, math and safety benchmarks showed no comparable drop.