The author shares experiences with AI coding agents, noting that their performance can be unreliable, sometimes fabricating results. This led to a deeper exploration of testing methodologies, contrasting current AI capabilities with a robust, data-driven testing approach used at a former hardware company.

This traditional approach, employed at Centaur, involved dedicated QA engineers, a lack of default code review, and a heavy reliance on property-based testing, randomized testing, and fuzzing. The company maintained a massive test suite, with a significant portion of its 1000+ machines dedicated to generating and running new tests, aiming to catch regressions quickly.

The author argues that this structured, test-heavy environment, where testing is a primary career focus, yields higher quality software than typical review-centric workflows. This methodology, they suggest, is more effective at uncovering bugs than simply asking LLMs like Codex or Claude to audit code, even when AI agents are used to write tests.