Roger Dickey details the ambitious project of creating "digital twins" for all 64 World Cup games. These twins are essentially 3D reconstructions, allowing users to virtually "be there" and experience the matches from any perspective, akin to a video game.

The process involved capturing raw video feeds and using advanced computer vision techniques, including pose estimation and optical flow, to reconstruct player movements and the ball's trajectory in 3D space. This data was then used to generate a dynamic, interactive model of each game.

The goal was to provide a novel way to re-watch or even "play" through past games, offering a level of immersion far beyond traditional broadcast footage. Dickey highlights the technical challenges and the potential for such digital twins in sports analysis and fan engagement, envisioning a future where fans can interact with historical sporting events in unprecedented ways.

This project showcases the cutting edge of real-time 3D reconstruction applied to live sports, transforming passive viewing into an active, explorable experience. The system aimed to represent the entire 2022 World Cup in this interactive format.