At Pantograph, we're training fully general robotics models that act autonomously for hours. Learning from internet video could scale with compute rather than limited action datasets. We develop a simple method for learning goal-directed behavior through pretraining on internet-scale video, learning goal-directedness during pretraining rather than post-training, greatly improving complex goal achievement. Minecraft serves as an open-ended testing ground supporting diverse, long-horizon goals. Our largest model, Pan-4B, is a 4B parameter model that fights mobs, explores for objects, completes platforming, and builds structures as commanded, generalizing to unseen environments.

We view internet-scale videos as reinforcement learning trajectories containing only observations. Goal-conditioning side-steps reward specification by using later video frames as goals for earlier parts—hindsight relabeling. Videos lack actions but allow learning state-dependent functions like value functions. After action-agnostic pretraining, a smaller action dataset produces an acting agent.

We pretrained on 500k hours of diverse Minecraft gameplay, then post-trained on 2k hours of contractor trajectories. Evaluated on 104 environments against STEVE-1 and a VLA baseline initialized from Gemma 4, Pan-4B outperforms across categories, especially on semantic and dexterous tasks.