A groundbreaking study from Stanford, Cal Tech, and Carleton College reveals that large language models like ChatGPT and Claude fail basic reasoning tasks despite their advanced capabilities. The research, published on arXiv and Transactions on Machine Learning Research, exposes systematic failures across multiple reasoning domains, challenging claims about AI's intellectual prowess.

Scientists found LLMs struggle with fundamental logic, arithmetic, and social reasoning tasks that humans handle intuitively. The models show human-like cognitive biases, fail at Theory of Mind exercises, and cannot consistently perform simple two-hop reasoning. Even basic counting and math word problems pose significant challenges, with models unable to assess whether problems contain errors.

These findings have major implications for industries relying on AI for decision-making and intellectual labor. The research suggests current architectures cannot achieve artificial general intelligence and highlights vulnerabilities to manipulation and jailbreaking. However, scientists view these failures as instructive, pointing to specific architectural improvements and proposing new benchmark frameworks to build more resilient AI systems.