Our latest model advances bring stronger capabilities for building and migrating software in low-resource programming languages such as COBOL, Fortran, and Visual Basic. These languages have less code and documentation available for models to learn from than languages like Python and JavaScript, yet they support critical infrastructure worldwide. COBOL supports more than 40% of online banking systems, 80% of in-person credit card transactions and 95% of ATM transactions, while Fortran is used in weather forecasting and nuclear reactor safety analysis. Improving AI's ability to work in these languages could help teams maintain and modernize the software that essential services depend on.
Maintaining this software is difficult. Documentation may be incomplete, original developers may have left the company, and important business rules may exist only in code. Teams fixing bugs or migrating to another language need to preserve the behavior their systems depend on. To measure progress, the developers created two evaluations. Multi Lang Bench tests whether a model can build a SQLite-compatible database engine from scratch across 20 programming languages. The COBOL migration evaluation tests whether a model can migrate existing programs to another language while preserving their behavior.
The latest models show substantial gains. On Multi Lang Bench, GPT-6 Astra achieves an average test pass rate of 87.4%, up from 26.8% for GPT-5.6 Sol, with improvements in every language tested. GPT-6 Luna also outperforms GPT-5.6 Sol, reaching 40.3% at approximately 78% lower estimated API cost. On the COBOL migration evaluation, GPT-6.1 Sol reaches 86.6% accuracy at low reasoning effort, compared with just 5.4% for GPT-5.6 Sol, while costing 39% less. At higher reasoning effort, accuracy reaches 97.9%.
Together, the results show the models are becoming significantly better at working across a wide range of programming languages while also becoming more cost-efficient.
Source: Hacker News · Summarized by HeadlinesBriefing