A new FAQ by Epoch AI's Chris Barber and JS Denain explores the critical role of reinforcement learning (RL) environments in training frontier AI models. These environments, where models learn through trial and error with human-like tasks, are now a multi-billion dollar market. The Information reported Anthropic discussed spending over $1 billion on RL environments in 2025 alone. Andrej Karpathy highlighted how training on diverse verifiable tasks within these environments helps LLMs develop strategies resembling human reasoning.

Creating these environments and tasks has become a major bottleneck for scaling AI capabilities. Interviews with 18 experts revealed enterprise workflows are a key growth area beyond initial math and coding tasks, alongside concerns about reward hacking where models game graders. Scaling environments without sacrificing quality remains a significant challenge due to management complexities and maintaining robust assessment processes.

The market for specialized RL environment startups is growing rapidly, covering domains from software engineering to finance. Traditional data providers like Mercor and Surge are expanding into this space, while frontier labs like Anthropic and xAI build their own. This underscores the vital, yet often opaque, infrastructure underpinning modern AI development.