AI-generated proofs are often checked using a process called formalisation. OpenAI appears to have made a subtle error when publishing its proofs of the Navier-Stokes problem, a team of mathematicians has claimed. The error doesn't mean that the proofs are incorrect or that OpenAI hasn't correctly solved the problem, but it does call into question whether mathematical results generated by AI models can always be relied on.
"What has to be done with all of these large language model-generated proofs is that they will have to be read by humans, and this creates an enormous extra burden on mathematicians," says Anders Hansen at the University of Cambridge. On 8 September, OpenAI announced that it had found a solution to the Navier-Stokes problem, one of the most famous open problems in mathematics. It published the proof in two versions – one written in "natural language" and another written in the computer code Lean.
The problem is, say Hansen and his team, that the two proofs don't match. "This formalisation process is trying to replace peer review," says team member Fabian Circelli, also at the University of Cambridge. "Peer review would mean that human eyes look at the proofs. But what we've shown in this paper is that using this type of AI auto-formalisation can't serve the same purpose."
The team's specific claim hinges on part of the proofs called Lemma 8.6. In the natural-language proof, an equation requires a value be below m + 4, while in the Lean proof the equivalent value is below m + 5, which is mathematically weaker. OpenAI told New Scientist that it is aware of the mismatch and that this doesn't mean either proof is invalid.
Source: Hacker News · Summarized by HeadlinesBriefing