The AI industry's promise of rapid recursive self-improvement may be overstated, according to a new study by researchers at Princeton University. Led by Peter Kirgis and Sayash Kapoor, the team evaluated AI agents' ability to conduct open-ended AI research using a method called "shadow evaluation." They tasked Anthropic's Claude Opus 4.8 with answering questions from unpublished NeurIPS 2026 papers, giving the agents six days, $3,000 in API credits, and access to GPUs and the web.

While the agents successfully completed the engineering aspects of research—reviewing literature, running experiments, and compiling results—they failed to produce publishable work. The original authors rejected both papers, citing lack of creativity, poor judgment, and no novel contributions. The agents ran bizarre experiments, struggled with writing, and committed to unpromising approaches too quickly.

Kapoor attributes this gap to current training methods, noting that reinforcement learning excels at checkable tasks but struggles with open-ended research requiring intuition and taste. Despite these limitations, the study suggests that hyped timelines for automating AI research may be premature.

The findings align with internal observations from AI companies. Anthropic cofounder Jack Clark noted similar creativity gaps in their safety research automation efforts, calling it a "bearish signal on short recursive self-improvement timelines."