Roy Rinberg (Anthropic), Alex Allain (Anthropic); Alessandro Chiesa (EPFL, Stark Ware), Miranda Christ (MIT, RESI), Rahul Dalal (Anthropic STEM Fellows Program, Succinct), Giacomo Fenzi (EPFL), Ben Mayer (Anthropic Fellows Program, MIT), Guy Weissenberg (EPFL) introduce Flash Prove, a system generating cryptographic proofs of inference. For a large model like Llama 4 Maverick (400B-parameter mixture of experts), a single H100 GPU generates proofs at around 110 tokens/second, with a proof size of ~19 MB per 1,024-token request using 4-bit quantized weights. This performance is approximately 70-95x faster than running the model itself.
Prior open-source systems are orders of magnitude slower. The commitment to the model and subsequent proofs hide the model's weights. The team released five artifacts under the Apache 2.0 license: the blog post, a technical report, a protocol specification, executable code including a C++ fast verifier and a Lean provably correct verifier, and Lean proofs with an interactive visualizer.
This research post shares intermediate progress to encourage community improvement. When a user sends a request to an AI model, they currently have no way of knowing for certain that the model they selected generated the response. While less pressing for everyday use, this assurance is critical for independent evaluators testing models before and after release.
These evaluators need verification that the model being assessed is the one actually producing outputs, preventing misattribution or tampering. Cryptographic proofs of inference address this by letting a model provider commit to a model and prove each response was produced by that claimed model. Such proofs could enable consumer products and internal security applications, contributing to a toolkit for verifying compliance with AI pacing requirements.
The system primarily uses zk SNARKs, chosen for their strong hiding and efficiency properties, making them uniquely fit for provable inference.
Source: Hacker News · Summarized by HeadlinesBriefing