A recent evaluation of xAI's Grok-4.7 model revealed discrepancies in how it handles web search results and citations. When using the web_search API, Grok-4.7 returned 1,656 URL annotations, but none of these were inline citations pointing to specific parts of the answer text. Instead, these URLs represented the model's entire reading list, including pages it deemed irrelevant. The study compared Grok-4.7's responses to those from ChatGPT, Claude, Gemini, and Perplexity across 24 software-buying questions, highlighting potential pitfalls in evaluating LLM answers that incorporate web search. AI
IMPACT Highlights potential issues in evaluating LLM responses that use web search, impacting how users and developers interpret AI-generated information.
RANK_REASON Analysis of an LLM's output format and citation handling. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →