A new open-source benchmark called SnapBench tests LLMs on a simple task: piloting a drone through a 3D world to find creatures. Only one model out of seven, Gemini Flash, succeeded by actually adjusting altitude to spot targets. Others, including Claude Opus, failed at basic navigation, wandering or approaching from wrong angles.

The experiment revealed that model size doesn't guarantee better spatial reasoning. Cheaper, smaller models like Gemini Flash outperformed more expensive counterparts, suggesting training data or instruction-following may matter more than raw parameter count for embodied tasks. The benchmark uses a Rust controller and Zig simulation, highlighting practical tooling for AI evaluation.

Results indicate that for robotics and navigation agents, a model's ability to follow literal commands may outweigh general intelligence. The creator plans real-world drone tests, emphasizing that building AI agents for physical spaces requires more than just a powerful LLM.