Researchers have developed ExecJudges, a novel method that uses LLMs to generate executable programs for relevance labeling in information retrieval. This approach aims to replace expensive and opaque LLM judges with more scalable, interpretable, and reproducible program-based assessments. ExecJudges demonstrated agreement with human assessors comparable to or exceeding that of LLM judges, with a minimal cost per topic. AI
IMPACT Offers a more cost-effective and reproducible alternative to LLM judges for relevance labeling in information retrieval systems.
RANK_REASON The cluster describes a novel method presented in an academic paper for improving information retrieval evaluation. [lever_c_demoted from research: ic=1 ai=1.0]
Read on arXiv cs.IR (Information Retrieval) →
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →