OpenAI has unveiled a groundbreaking approach called Video PreTraining (VPT) that successfully trains neural networks to play the complex sandbox game Minecraft. By leveraging a massive dataset of unlabeled video footage of human gameplay, supplemented by a small amount of labeled contractor data, OpenAI created a model that performs long-horizon tasks. Notably, the model can craft diamond tools—a feat that typically requires proficient human players over 20 minutes and 24,000 individual actions.

Unlike previous reinforcement learning methods, this model utilizes the native human interface of keypresses and mouse movements. This is a significant leap in AI capability, as it demonstrates the potential for AI to generalize computer usage tasks beyond specific game APIs. The success of VPT suggests a future where AI agents can navigate and operate software interfaces just as humans do, representing a major step toward general-purpose computer-using agents.