A new benchmark, BizFinBench.v2, has been developed to evaluate Large Language Models (LLMs) in financial applications, using authentic user query-response data from Chinese and U.S. equity markets. The benchmark includes 28,860 questions across eight offline and two online tasks. Initial experiments show that GPT-5 achieved only 61.5% accuracy, falling short of the practical business requirement of 84.8%. DeepSeek-R1 demonstrated superior investment efficacy among commercial models, and error analysis highlighted persistent limitations in current LLMs. AI
IMPACT Highlights the need for more robust LLM evaluation in specialized domains like finance, potentially driving development of more accurate and reliable financial AI tools.
RANK_REASON The cluster is about a new academic paper introducing a benchmark for LLMs. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
