Autoresearch, an AI framework developed by Andrej Karpathy, automates iterative experimentation to optimize complex tasks. By running thousands of hypothesis tests in a loop, it identifies high-impact solutions without human intervention. Recently, pi-autoresearch, an open-source extension, applied this approach to marketing budget allocation, aiming to maximize revenue within a $30M constraint.

The system starts with a baseline strategy—selecting top-performing campaigns by revenue-per-dollar spent. However, pi-autoresearch iteratively tests alternative allocations, discarding ineffective ideas and refining successful ones. In the experiment, the AI improved upon the baseline’s $107.9M revenue outcome by exploring novel campaign combinations, though specific gains weren’t quantified. The process required defining clear metrics, constraints, and a guardrail to prevent data manipulation.

Technical setup involved Python’s pi framework, which integrates with LLMs to execute code changes autonomously. Developers configured a 30-iteration limit to balance exploration with computational costs. The tool’s ability to self-modify code and evaluate outcomes mirrors biological evolution, offering a blueprint for scaling analytical workflows.

This approach democratizes advanced optimization, enabling teams to tackle problems previously deemed too time-intensive. By automating experimentation, autoresearch shifts human focus from manual iteration to strategic oversight, potentially transforming industries reliant on data-driven decision-making.