Researchers have introduced EntSQL, a new benchmark designed to evaluate Text-to-SQL capabilities in enterprise settings. Unlike previous benchmarks, EntSQL focuses on grounding SQL generation in long-context, proprietary business documents. The benchmark includes 1,066 aligned Chinese-English examples across five business domains, many of which require knowledge beyond the immediate question and schema. Current systems struggle with this task, with the best performing model achieving only 15.9% accuracy on English inputs when provided with long-form documents. AI
IMPACT Highlights the challenge of applying LLMs to enterprise-specific data, potentially driving development of more context-aware Text-to-SQL systems.
RANK_REASON The cluster describes a new academic benchmark for evaluating AI capabilities.
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →