Researchers have introduced MedCalc-Pro, a new benchmark designed to evaluate large language models (LLMs) on complex medical calculations. This benchmark moves beyond simplified settings by incorporating multi-calculator and nested-calculator scenarios, reflecting real-world clinical needs. MedCalc-Pro includes 2,268 cases across 77 medical calculators and 14 departments. To address performance limitations, the team also proposed a novel agent framework that enhances multi-tool selection and nested-tool calling, demonstrating superior results across various LLMs. AI
IMPACT This benchmark and framework could accelerate the development and application of LLMs in specialized domains like medicine.
RANK_REASON The item describes a new academic benchmark and methodology for evaluating LLMs, published on arXiv. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- CatalyzeX
- Connected Papers
- CORE Recommender
- DagsHub
- Gotit.pub
- Hugging Face
- Influence Flower
- large-language models
- Litmaps
- LLM agents
- MedCalc-Pro
- ScienceCast
- scite Smart Citations
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
