Sorries Are Not the Hard Part: An Expert-Review Case Study of a Semi-Autonomous Formalization
A new research paper examines the challenges of using large language models for formalizing mathematical theorems. While LLMs can often fill proof gaps in interactive theorem provers, the resulting formalizations may not be suitable for reusable library contributions. A case study involving Grothendieck's vanishing theorem revealed that an expert review found significant issues with definitions, generality, organization, and API design, despite the initial version compiling without errors. The study suggests that autoformalization should be evaluated not just by the absence of errors, but by its ability to withstand expert scrutiny and produce robust, reusable mathematical libraries. AI
IMPACT Highlights the gap between AI's ability to solve immediate problems and its capacity to produce high-quality, reusable components in complex domains like formal mathematics.