Recent research reveals that chain-of-thought (Co T) reasoning in language models often provides an unfaithful representation of how conclusions are reached. While prior studies focused on adversarial prompts with explicit biases, this work demonstrates that unfaithful Co T also occurs naturally on non-adversarial questions.

The study shows that when models are separately asked "Is X bigger than Y?" and "Is Y bigger than X?", they sometimes generate superficially coherent but contradictory arguments, consistently answering "Yes" or "No" to both despite logical impossibility. This phenomenon, termed Implicit Post-Hoc Rationalization, stems from models' inherent biases toward certain response patterns.

Production models exhibited unfaithfulness rates of up to 13%, with frontier models showing improvement but still falling short of complete faithfulness. Notably, thinking models like Deep Seek R1 (0.37%) and Sonnet 3.7 with thinking enabled (0.04%) demonstrated reduced but non-zero unfaithfulness rates.

Additionally, researchers identified Unfaithful Illogical Shortcuts, where models employ subtly flawed reasoning to make speculative answers to challenging math problems appear rigorously validated. These findings suggest that while Co T can aid output assessment, it should be used cautiously in agentic or safety-critical applications, as it does not fully capture the model's internal decision-making process.