Formal Verification Breakthrough: AI Proves Code Correctness

Anthropic has taken a major step in software automation by using its model, Claude, to mathematically verify existing code and prove its correctness. This achievement was made possible through a partnership with Theorem, which allowed Claude to generate formal proofs for 123 open-source libraries in just a few weeks. During this process, the model produced roughly 180 million lines of proof code and identified hundreds of bugs that conventional testing had missed.

Proving that software runs error-free under all circumstances has always been prohibitively expensive, as it typically required a team of specialists spending 20 person-years to formally verify a single small operating system kernel. Anthropic's approach, however, has changed this paradigm by allowing AI to take over the proof work. The model translated crash tests into mathematical problems, analyzed the compiled machine code, and produced extensive proofs, which were then automatically checked by an independent verification program called Rocq. According to Anthropic, this process needs almost no human reviewers, and the researchers call it 'fully autonomous' proof generation, though the tools and formal foundations were built by humans at Theorem.

While generating proofs, Claude ran into hundreds of bugs and was allowed to fix them itself. Many of these bugs would have slipped past conventional testing because they only show up under very specific conditions, such as unusually large input files or a particular sequence of accesses. During an effort to verify a widely used PDF reader, for example, the team found as many security bugs in one week as are typically reported for that software in an entire year. In another application, Claude caught a logical error that had gone undetected for over a decade. Even when a complete proof ultimately fails, Anthropic says, the attempt still surfaces real bugs.

Anthropic acknowledges that 'formally verified' doesn't mean 'error-free.' The proofs only hold under strict conditions: a specific processor architecture, single-threaded programs, and inputs under 64 MB. In practice, these libraries run on different hardware and in different configurations. The compute cost is steep, with individual proofs running up hundreds of millions of tokens. Anthropic doesn't report total project costs. But if AI can verify its own code, developers can trust AI-generated software more and spend less time reviewing it. That saves costs not just on verification but on the entire coding process. A smaller test run also showed that proofs are reusable, and newer models are getting more efficient: Opus 5.5 solves half the proofs with about 40 percent fewer tokens than Opus 5.

The project strengthens the case for software automation. If AI can write code and guarantee its correctness under defined conditions, the question shifts from 'Can AI program?' to 'Can AI prove that the code it writes is safe?' If the answer is yes, the consequences could go well beyond today's coding tools. Case in point: Anthropic expects formal verification with AI will soon become as practical as standard testing, and Theorem's roadmap already targets critical infrastructure like glibc, OpenSSL, and cURL.