一篇新论文提出了三种不同的模型，用于说明人类标注者的判断如何通过人类反馈强化学习（RLHF）来塑造大型语言模型的行为。这三种模型是：“延伸”，即标注者与设计者的观点保持一致；“证据”，即标注者提供事实信息；以及“权威”，即标注者代表更广泛的社会共识。该论文认为，RLHF流程应根据这些不同的角色进行定制，而不是采用单一的统一方法。 AI
影响 阐明了人类反馈在LLM对齐中的规范作用，可能改进标注策略。
排序理由 学术论文，提出了RLHF标注的新概念模型。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2604.25895v1 Announce Type: cross Abstract: Preference-based alignment methods, most prominently Reinforcement Learning with Human Feedback (RLHF), use the judgments of human annotators to shape large language model behaviour. However, the normative role of these judgments …
Preference-based alignment methods, most prominently Reinforcement Learning with Human Feedback (RLHF), use the judgments of human annotators to shape large language model behaviour. However, the normative role of these judgments is rarely made explicit. I distinguish three conce…