Software testing versus AI testing: what changes. The testing pyramid still holds but everything inside it changes. The failure modes that actually matter.
What an AI testing strategy should include. When the system type changes, so does the testing strategy. Retrieval-augmented systems.
Agent systems. External service integrations. The part nobody talks about: validating with the client.
A more honest definition of done. Code is cheap. Human time and model context are not.
That is the core shift behind Harness Engineering: instead of trying to write better prompts, teams getting more reliable value from coding agents increasingly design better systems around their agents documentation, context loading, validation, and feedback loops that let models safely own more of the work. Earlier this year, our team won an Open AI Codex hackathon with Hyper Code, a repository intelligence layer that prepares relevant context for coding agents before they start working. Building it gave us a concrete way to explore how the system around an agent shapes its work.
For engineering leaders, the goal is more predictable delivery less time spent reconstructing context and fixing avoidable agent mistakes, with consistent checks before changes reach review. We also tested harness patterns in our own setups with Codex and Claude Code, bringing together people from across deepsense.ai with different backgrounds workflows and levels of experience with AI-assisted development. We explored agent-readable repo docs just-in-time context validation loops and ways to turn recurring mistakes into durable guardrails.
Some patterns proved useful others needed adjustment. This post brings together those findings and the practical recommendations that emerged. This article is our practical playbook: ten harness patterns you can apply to a real repo this week.
We focus on three concrete layers how your repo becomes an agent-readable knowledge base how context is compiled just-in-time for each task and how validation and feedback turn agent mistakes into durable system improvements.
Source: Hacker News · Summarized by HeadlinesBriefing