A recent test evaluated the accuracy of the Claude Sonnet-5 model when answering questions about companies, both with and without access to real-time company registers. Without access, the model predominantly refused to answer or provided honest hedges, with only a small percentage of responses being confidently wrong. When provided with tools to access registers like Brønnøysundregistrene, Companies House, and Bolagsverket, the model's accuracy improved significantly, though some inconsistencies remained. AI
IMPACT This research highlights the importance of real-time data access for LLMs to avoid confidently incorrect answers on factual queries.
RANK_REASON The item details an experiment evaluating an LLM's performance on a specific task, including methodology and results. [lever_c_demoted from research: ic=1 ai=1.0]
- Brønnøysund Register Centre
- Claude Sonnet-5
- Companies House
- MCP
- Norwegian Central Coordinating Register for Legal Entities
- Oslo Stock Exchange
- Swedish Companies Registration Office
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
