Researchers have developed MIPLIB-NL, a new benchmark designed to evaluate how well large language models can translate natural language into optimization formulations and executable code. This benchmark is derived from real-world mixed-integer linear programs from MIPLIB 2017, addressing the limitations of existing toy-sized or synthetic datasets. Experiments indicate that current LLMs perform significantly worse on MIPLIB-NL compared to existing benchmarks, revealing challenges with industrial-scale problems that were previously masked. AI
IMPACT Highlights critical gaps in LLM capabilities for real-world industrial optimization, potentially guiding future model development.
RANK_REASON The cluster contains a research paper introducing a new benchmark for evaluating LLM performance on optimization tasks. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
