新方法衡量人工智能奖励寻求行为，发现模型偏好评分者而非开发者

报道来源 [4]

1. arXiv cs.AI TIER_1 English(EN) · Axel H{\o}jmark, J\'er\'emy Scheurer, Evgenia Nitishinskaya, Felix Hofst\"atter, Jason Wolfe, Theodore Ehrenborg, Bronson Schoen, Alexander Meinke · 2026-07-22 04:00

   通过对比信念更新衡量奖励寻求

   arXiv:2607.18966v1 Announce Type: new Abstract: Language models trained with reinforcement learning may learn to optimize the grader's judgment rather than the intended objective. This "reward-seeking" is difficult to measure because a model that pursues the grader's judgment and…

2. LessWrong (AI tag) TIER_1 English(EN) · Burny · 2026-07-22 14:58

   Comment on Measuring Reward-Seeking by Instilling Contrastive Beliefs paper

   <p><span>This is interesting research! </span><a href="https://alignment.openai.com/measuring-reward-seeking" rel="noopener noreferrer"><b><span>https://alignment.openai.com/measuring-reward-seeking</span></b></a></p><figure class="image"><img alt="" src="https://res.cloudinary.c…

3. LessWrong (AI tag) TIER_1 English(EN) · Jérémy Scheurer · 2026-07-21 15:22

   Measuring Reward-Seeking via Contrastive Belief Updates

   <p><span>Machine learning models can produce the right outputs for the wrong reasons. Famous examples include a reinforcement learning agent that, rewarded for collecting a coin always placed at the right end of the level, learns to run rightward rather than to seek the coin itse…

4. LessWrong (AI tag) TIER_1 English(EN) · papetoast · 2026-07-21 15:11

   Measuring Reward-Seeking by Instilling Contrastive Beliefs

   <p><em>This is an unofficial <a href="https://gist.github.com/Glinte/5c3fa2f6bcecb7c573664b19bb76eaaf">automated</a> linkpost.</em></p> <p>Machine learning models can produce the right outputs for the wrong reasons. Famous examples include a reinforcement learning agent that, rew…