Research from metr reveals LLM programming capabilities may have stagnated for over a year, despite industry claims of continuous improvement. Their analysis shows LLM code performs much worse when judged by maintainer approval rather than just passing tests. The 50% success horizon drops from 50 minutes to just 8 minutes under more realistic criteria.

When examining merge rates—arguably the most valuable metric for code quality—the data shows no improvement since early 2025. Formal analysis using Brier scores confirms constant functions predict these rates better than linear or logistic trends. This contradicts metr's suggested gentle upward slope and suggests genuine progress has stalled.

While some claim recent Anthropic and Google models represent another capability jump, no evidence exists to support these assertions. The gap between industry buzz and actual performance appears larger than believed. Without rigorous measurement of newer models beyond Sonnet 4.5, developers should treat claims of recent breakthroughs with skepticism.