新的大型语言模型评估方法解决对齐和偏见问题

报道来源 [9]

1. Google AI / Research TIER_1 English(EN) · 2026-04-03 08:00

   Evaluating alignment of behavioral dispositions in LLMs

   Generative AI

2. arXiv cs.AI TIER_1 English(EN) · Sangwoo Cho, Kushal Chawla, Pengshan Cai, Zefang Liu, Chenyang Zhu, Shi-Xiong Zhang, Sambit Sahu · 2026-06-26 04:00

   Ask, Don't Judge: Binary Questions for Interpretable LLM Evaluation and Self-Improvement

   arXiv:2606.27226v1 Announce Type: new Abstract: Evaluating LLM outputs remains a major bottleneck in NLP: human evaluation is expensive and slow, lexical metrics correlate poorly with human judgments on open-ended generation, and holistic LLM judges often produce opaque scores th…

3. Hugging Face Daily Papers TIER_1 English(EN) · 2026-06-25 16:14

   Ask, Don't Judge: Binary Questions for Interpretable LLM Evaluation and Self-Improvement

   Evaluating LLM outputs remains a major bottleneck in NLP: human evaluation is expensive and slow, lexical metrics correlate poorly with human judgments on open-ended generation, and holistic LLM judges often produce opaque scores that are hard to debug. We propose BINEVAL, a fram…

4. arXiv cs.AI TIER_1 English(EN) · Sambit Sahu · 2026-06-25 16:14

   Ask, Don't Judge: Binary Questions for Interpretable LLM Evaluation and Self-Improvement

   Evaluating LLM outputs remains a major bottleneck in NLP: human evaluation is expensive and slow, lexical metrics correlate poorly with human judgments on open-ended generation, and holistic LLM judges often produce opaque scores that are hard to debug. We propose BINEVAL, a fram…

5. arXiv cs.CL TIER_1 English(EN) · Yuzheng Xu, Tosho Hirasawa, Tadashi Kozuno, Yoshitaka Ushiku · 2026-06-25 04:00

   Am I More Pointwise or Pairwise? Revealing Position Bias in Rubric-Based LLM-as-a-Judge

   arXiv:2602.02219v2 Announce Type: replace Abstract: Large language models are widely employed as evaluators, a paradigm commonly referred to as LLM-as-a-judge. Prior research has predominantly examined point-wise or pair-wise evaluation protocols; in contrast, our focus is on rub…

6. arXiv cs.LG TIER_1 English(EN) · Kai Qin, Jiaqi Wu, Jianxiang He, Haoyuan Sun, Yifei Zhao, Xu Wang, Bin Liang, Yongzhe Chang, Cheng Li, Tiantian Zhang, Houde Liu · 2026-06-25 04:00

   Distribution Preference Optimization: A Fine-grained Perspective for LLM Unlearning

   arXiv:2510.04773v2 Announce Type: replace Abstract: As Large Language Models (LLMs) demonstrate remarkable capabilities learned from vast corpora, concerns regarding data privacy and safety are receiving increasing attention. LLM unlearning, which aims to remove the influence of …

7. arXiv cs.AI TIER_1 English(EN) · Dani Roytburg, Matthew Bozoukov, Matthew Nguyen, Jou Barzdukas, Mackenzie Puig-Hall, Narmeen Oozeer · 2026-06-24 04:00

   Are LLM Evaluators Really Narcissists? Sanity Checking Self-Preference Evaluations

   arXiv:2601.22548v4 Announce Type: replace-cross Abstract: Recent research has shown that large language models (LLMs) favor their own outputs when acting as judges, undermining the integrity of automated post-training and evaluation workflows. However, it is difficult to disentan…

8. arXiv cs.AI TIER_1 English(EN) · Dani Roytburg, Matthew Bozoukov, Matthew Nguyen, Jou Barzdukas, Simon Fu, Narmeen Oozeer · 2026-06-24 04:00

   Breaking the Mirror: Activation-Based Mitigation of Self-Preference in LLM Evaluators

   arXiv:2509.03647v2 Announce Type: replace-cross Abstract: Large language models (LLMs) increasingly serve as automated evaluators, yet they suffer from "self-preference bias": a tendency to favor their own outputs over those of other models. This bias undermines fairness and reli…

9. arXiv cs.CL TIER_1 English(EN) · Yihuang Kang · 2026-06-20 08:13

   LLM 自我评估的潜在置信度对齐

   Confidence calibration in large language models (LLMs) is commonly evaluated by comparing predicted confidence with observed accuracy. However, such approaches do not model item difficulty, making it difficult to interpret discrepancies and to determine whether model confidence r…