New SESSE framework decomposes LLM-as-Judge evaluations

Researchers have introduced SESSE, a novel framework designed to enhance the evaluation of Large Language Models (LLMs) when used as judges. Unlike traditional methods that rely on single preference choices, SESSE decomposes the judgment process into structured sub-questions derived from the judge's own error cases. This approach requires no oracle responses, task-specific rubrics, or fine-tuning, making it a flexible and training-free solution. In evaluations on RewardBench, SESSE demonstrated performance comparable to chain-of-thought baselines and competitive with specialized models like RISE-Judge-32B, while providing interpretable evidence for diagnosing label ambiguity and judge failures. AI

IMPACT Enhances interpretability and diagnostic capabilities in LLM evaluations, potentially improving model development and reliability.

RANK_REASON The cluster describes a new research paper detailing a novel evaluation framework for LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• Hugging Face
• LLM-as-Judge
• RewardBench
• RISE-Judge-32B

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →