Mirror Code, co-developed with MET R, is a benchmark designed to assess AI models on long-horizon coding tasks by requiring them to reimplement entire programs end-to-end without access to the original source code. The AI-generated solutions must precisely match the original program's output on comprehensive tests, including those not seen during development.

The benchmark features 25 target programs across diverse computing domains such as Unix utilities, data tools, bioinformatics, and cryptography. Mirror Code provides a substantial inference budget, with one task costing $2,600 for a single run and taking AI 19 days of work. This contrasts with many benchmarks that limit spending to around $1–$10.

Despite the difficulty, AI has demonstrated capabilities on Mirror Code tasks. For instance, Claude Opus 4.7 successfully reimplemented `gotree`, a bioinformatics toolkit with approximately 16,000 lines of Go code. This task, estimated to take a human weeks, was completed by Opus 4.7 in 14 hours for $251. While data contamination is a potential concern, the benchmark includes measures to mitigate it, suggesting that AI performance is not solely due to memorization. The project aims to measure capabilities that generalize to unseen codebases.