Two AIs agree = one source: the rule that saved me from a pipeline built on nothing
The author discovered that using two different AI models, ChatGPT-4o and Claude.ai, for reviewing a document resulted in identical feedback. This convergence, however, was not a sign of accurate calibration but rather a reflection of the models' shared training data, leading to correlated errors and hallucinations. The author then conducted three separate tests using a tool called WebFetch and a YAML parser, which revealed that the AI assistants had either fabricated information or hallucinated issues, underscoring the need to independently verify AI-generated claims rather than relying on their apparent confidence or agreement. AI
IMPACT Highlights the critical need for users to independently verify AI-generated information due to potential for correlated errors and hallucinations stemming from shared training data.