A browser game designed to test human oversight of AI coding agents revealed significant vulnerabilities. Across over 40,000 game runs and 409,000 decisions, players, acting as the human-in-the-loop, missed an average of 1 in 3 threats, achieving a 66.3% accuracy rate. This means a substantial portion of malicious commands, such as those attempting to exfiltrate secrets, were approved.

The data showed that while overtly destructive commands were caught more reliably, subtle threats like exfiltration and code execution were missed at much higher rates, with commands like `cat ~/.aws/credentials` being approved 35.0% of the time. A particularly concerning blind spot was revealed with `npm run` commands, which were approved over 50% of the time, even when malicious scripts were detailed in the agent's history log.

Factors contributing to these misses include permission fatigue, where players become less diligent with repeated prompts, and the stress of time-limited challenges. Conversely, benign commands were often over-blocked, potentially leading to users becoming desensitized and more likely to approve actual threats. The game highlights the inherent challenges in relying on human oversight as the sole safeguard against rogue AI agents, emphasizing the need for robust security practices beyond simple command approval.