HeadlinesBriefing HeadlinesBriefing.com

From Demo to Production: Building Reliable AI Agents

Towards Data Science •
×

Anyone can build an AI agent in an afternoon, but making it survive real business environments is another matter. The gap between a working demo and Monday morning operations is called harness engineering. This discipline involves wrapping the model with loops, tools, permissions, files, tests, and human oversight. Thoughtworks summarizes this as Agent = Model + Harness. The concept gained traction after OpenAI published a write-up in February 2026 about shipping an internal product with zero hand-written code. Three engineers using Codex produced approximately one million lines and 1,500 merged pull requests in five months. Their conclusion emphasized designing environments, specifying intent, and building feedback loops rather than relying solely on model capabilities. Safeguards were embedded in the harness, not in human reviewers. The core principle is to constrain what the model can touch, not what it can think about. Let it plan freely but decide in code what it is allowed to read, write, spend, and send. This guide addresses the common question of whether the model should decide, code should enforce, or a human must approve.

Forward-deployed engineers (FDEs) play a key role in production agents. Palantir pioneered this role, and AI labs have adopted it. As of September 2026, 77 of Palantir's 310 open roles were FDE positions, while OpenAI listed 21 of 780. Anthropic had the smallest FDE footprint. Most FDE postings had been open for over six months, but the trend shows labs investing in deployment, not just models. An FDE translates business needs like "clean up supplier data" into systems with clear boundaries. They define how much the agent can do alone—read only, draft but not send, or act under spending limits. They establish which systems, tables, and actions are off-limits and where human sign-off is required. These decisions involve product, risk, and operations teams, not just prompt engineering.

Source: Towards Data Science · Summarized by HeadlinesBriefing