My devil's advocate worked. That was the bug.
The author details the iterative development of a "devil's advocate" tool within their sonmat application, which aims to combat AI-generated confident errors. Initially named "imp" and released on April 2nd, the tool underwent five significant revisions over 24 days, culminating in version 0.11.0 on April 26th. Key changes included renaming the tool to "devil" for clarity, integrating it into a larger verification system with "Rhythm Rules," and refining the readability of its balance table by transforming abstract column headers into direct questions. AI
IMPACT Details the process of building and refining AI-assisted reasoning tools, highlighting the importance of clear naming and user interface design.