Rather than relying on brute-force reinforcement learning, the team iterated on supervised fine-tuning (SFT) and RL recipes with a focus on compute efficiency. They avoided purchasing posttraining data, instead building synthetic data pipelines, quality filters, and coverage metrics. Their mixed RL approach used truncated importance sampling, score centering, and a linear length penalty within high-quality long-horizon synthetic environments.
Tasks were dynamically tuned to model capabilities and progressively hardened as performance improved. This method enabled a steep RL climb from a strong SFT checkpoint. The model achieved a Terminal Bench 2.1 score of 80.5 using only 1.4M rollouts—approximately 70 times fewer than Reflection Beam, the previous state-of-the-art.
Source: Hacker News · Summarized by HeadlinesBriefing