New benchmark UTP-Bench tests LLMs for uncertainty in travel planning

Researchers have introduced UTP-Bench, a new benchmark designed to evaluate the robustness of large language models in generating travel itineraries under uncertain conditions. Unlike previous benchmarks that assume deterministic environments, UTP-Bench incorporates real-world data from 504 Indian cities, including empirical delay distributions and crowd patterns. It proposes three new metrics—Buffer Adequacy Score, Crowd-Aware Timing Score, and Transport Delay Absorption Score—to quantify how well generated plans handle transit delays and crowd variability. Experiments using models like GPT-5, Qwen3, Mistral, and Phi-4 revealed significant performance gaps compared to human-authored plans, particularly in temporal buffering and delay-aware scheduling. AI

IMPACT This benchmark could drive improvements in LLM capabilities for real-world applications requiring robust planning under uncertainty.

RANK_REASON The cluster describes a new academic paper introducing a benchmark for evaluating LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Buffer Adequacy Score
• Crowd-Aware Timing Score
• GPT-5
• India
• large-language models
• Mistral AI
• Phi-4
• Qwen3
• Transport Delay Absorption Score
• TravelPlanner
• TRIPCRAFT
• UTP-Bench

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →