Straight Up AI, a small consultancy, has built an internal control plane for orchestrating coding agents. How much latitude those agents get depends on three factors: the blast radius of a mistake, project context, and project maturity. The team argues that one factor is missing: the cost of building. Running the control plane through a Claude Max account costs £200 a month, so the marginal cost of bad decisions is mainly time. Before developing it further, the team needed to check whether it could genuinely be afforded.
To find out, the team analysed seven weeks of activity, from 15 July to 4 September, covering 44 development cycles across its portfolio. Converting every token to input-token equivalents, with cached reads at 0.1x, cached writes at 1.25x and output tokens at 5x, showed that API rates would cost 22 times more than the current utilisation. As the consultancy grows, that gap quickly becomes untenable and approaches the cost of a mid-level engineer's salary.
Time was the second concern. Adversarial review, which spins up a second agent to challenge every commit, was the suspected cause of slow runs. The data largely supported this, though not for the reason first assumed. Reviewers were dispatched at a ratio of 1.19x the number of implementers, and 26% of them requested changes. Each change triggered a remediation agent and another review cycle, which risked rabbit-holing rather than prompting a step back to consider structural changes beyond the commit's scope.
The control plane uses one controller that lives for the whole job and dispatches one worker per increment. The controller acts as an independent arbiter, validating whether each increment is completed, reviewed or failed, which prevents incorrect progress claims from cascading through the plan.
Source: Towards Data Science · Summarized by HeadlinesBriefing