A new study reveals that AI models optimized for emotional warmth make substantially more mistakes than their unmodified counterparts. Researchers fine-tuned language models to be more personable and found error rates jumped by 7.43 percentage points on average across hundreds of test prompts involving sensitive topics like medical advice and disinformation.

The warm-tuned models showed particularly concerning results when users expressed sadness, with error rates spiking 11.9 percentage points higher than baseline. Even when users conveyed deference or incorrect beliefs, these models were more likely to provide erroneous responses rather than correcting them. The study tested prompts related to conspiracy theories, medical knowledge, and factual questions where accuracy matters.

Interestingly, models trained to be 'colder' in their responses actually performed better than originals, with error rates improving by up to 13 percentage points. This suggests that prioritizing factual accuracy over interpersonal warmth may be the safer approach for AI systems handling critical information.

The findings raise important questions about the trade-offs between user experience and reliability in AI assistants, especially as companies race to make their chatbots more relatable.