New benchmark evaluates LLM negotiation skills, GPT-5 matches human baseline

Researchers have introduced PieArena, a new benchmark designed to evaluate the negotiation capabilities of large language models. This benchmark utilizes realistic scenarios adapted from MBA negotiation courses and assesses models across various pairing regimes, including human-AI interactions. The evaluation goes beyond simple outcome scores to provide a multi-dimensional behavioral profile, examining aspects like instruction compliance, deception, and reputation. Notably, a frontier model, GPT-5, demonstrated performance comparable to or exceeding human baselines in these negotiation tasks. AI

IMPACT Establishes a new standard for evaluating LLM strategic reasoning and negotiation, potentially driving improvements in agentic capabilities for business applications.

RANK_REASON This is a research paper introducing a new benchmark for evaluating LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• GPT-5
• PieArena
• Yu Zhu

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →