近期研究探讨了大型语言模型(LLM)生成偏好的可靠性和校准性。一项研究发现,虽然LLM在偏好上表现出自洽性,但在不同模型和规模之间存在显著的不一致性,表明提示词措辞的影响不如模型选择大。另一篇论文证明,较小的、冻结的模型可以生成比自身生成更有效的“拒绝”样本用于偏好蒸馏,从而降低计算成本。第三项研究引入了基于评分卡的偏好(Rubric-Calibrated Preferences, RCP),以提高LLM在信息检索中判断的质量和跨查询可比性,显示出与人类评估的一致性有所提高。
AI
arXiv:2610.01000v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly used as probabilistic generators for simulation, synthetic data generation, and decision support in settings where real-world data are unavailable. Yet, the structure and reliability of …
arXiv:2609.38987v1 Announce Type: new Abstract: Preference distillation typically treats a teacher response as preferred and the student's own response as rejected. This assumes that self-generated failures are the most informative negatives and that rejects must come from a mode…
Preference distillation typically treats a teacher response as preferred and the student's own response as rejected. This assumes that self-generated failures are the most informative negatives and that rejects must come from a model at least as large as the student, making gener…
Rerankers decide which documents users and LLMs see, yet their standard metric, nDCG, relies on human relevance labels that are costly, sparse, noisy, and discretely graded. As rerankers approach each other in quality, nDCG on these labels therefore increasingly fails to separate…