Most software is riddled with bugs, and AI can now easily find and exploit them. Standard defenses against cyberattacks are no longer sufficient, so developers need ways to prove that code is free of bugs. Formal software verification could provide that guarantee, but it has historically required years of expert effort, making it impractical for most real-world projects.
In a collaboration with Theorem, researchers show this is changing. LLMs can now produce formal proofs at large scale, enabling new development workflows that include proven security guarantees. Using Claude, the team proved security properties at the assembly code level (RISC-V) across more than 100 real-world open-source software libraries. To their knowledge, this is the first large-scale effort to use AI to apply formal verification to production software at that level. While writing the proofs, Claude also uncovered bugs that had eluded detection for years despite trillions of test runs.
The approach builds on a long-standing method: developers translate code into a mathematical model and try to prove it always follows defined security rules. Failed proofs point to likely bugs. Traditionally, proofs can be many times larger than the original code and demand rare, specialized skills. Building the seL4 kernel took two person-years, but verifying it took 20.
To test the approach, Claude proved that 123 open-source programs will always exit normally without memory-safety errors or failed assertions, for every input under 64 MB, under stated assumptions about the runtime environment. The researchers acknowledge this is far from fixing all possible bugs, but they say it offers a roadmap. A subset of the proofs is being published so others can verify and reuse them.
Source: Hacker News · Summarized by HeadlinesBriefing