A new study published on arXiv investigates how Large Language Model (LLM) agents handle conflicting instructions. Researchers developed a benchmark called WhoWins with over 1,000 scenarios and tested 14 open-weight models. The findings indicate that instructions presented earlier within a single message are generally followed more often than those presented later, though this pattern shifts when instructions are delivered across different messages. The study also observed that the effectiveness of instruction placement varies significantly between different model families, such as Qwen and gpt-oss, and that models perform better when aware of the task's context. AI
IMPACT This research highlights a critical area for improving LLM agent reliability and predictability in complex operational environments.
RANK_REASON The cluster contains a research paper detailing a new benchmark and experimental findings on LLM agent behavior. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
