Researchers have developed a new benchmark called Clarify-Then-Search to evaluate the effectiveness of Large Language Models (LLMs) in improving deep search capabilities. This benchmark, built on real-world query data, assesses how LLM-generated clarification questions enhance search utility by restoring missing query constraints like time, location, or scope. The system demonstrated that clarification generally improves search results, with GPT-5.2 performing best at a single clarification step and ERNIE-4.5-Turbo-128K excelling with multiple clarification questions. A common failure mode identified was the over-asking of unanswerable region-specific questions. AI
IMPACT This benchmark could drive improvements in search engine accuracy and user experience by enabling better evaluation of LLM-driven clarification strategies.
RANK_REASON The cluster describes a new academic benchmark for evaluating LLM performance in a specific task (deep search clarification). [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
