Andrej Karpathy explored the evolving capabilities of large language models by feeding the opening paragraph of "The Lord of the Rings" to Opus 5, along with a 1 million token budget (~$10), and instructing it to generate a Three.js render of the scene. The model spent approximately two hours producing 5,500 lines of procedural code that brought the story to life in a rudimentary but functional 3D environment. While the output was described as "janky," Karpathy noted the remarkable feat of an LLM orchestrating polygon assets in (x, y, z) coordinates and animating them through code—something no human would typically attempt manually.

The experiment highlights how LLMs can leverage their stamina and lack of fatigue to tackle highly customized tasks that are otherwise impractical for humans. Karpathy envisions a future where such models can rapidly generate immersive, ephemeral worlds—akin to an on-demand "GTA of X"—where users can interact as spectators or characters within narrative settings like Middle-earth.

However, the test also revealed a key limitation: LLMs struggle to audit their own work due to an inability to natively perceive video or play games. Opus 5 had to painstakingly take screenshots at various stages and still produced errors and inconsistencies. This underscores a gap in multimodal understanding that remains a challenge for current AI systems.