LLM reranking in recommender systems shows position bias

A new research paper published on arXiv explores the issue of position bias in large language models (LLMs) when used for reranking in recommender systems. The study found that decoder-only LLMs can be sensitive to the order in which candidates are presented, leading to inconsistent preferences and rankings. Researchers developed a framework to measure this instability at pairwise, global, and output levels, demonstrating that reducing simple exposure bias is not enough to ensure reliable LLM-based reranking. AI

IMPACT Highlights a potential flaw in LLM applications for recommendation systems, suggesting further research is needed for reliable deployment.

RANK_REASON Research paper published on arXiv detailing a technical finding about LLM behavior. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.IR (Information Retrieval) →

• arXiv
• Large language models
• recommender systems

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →