一篇新的研究论文探讨了在强化学习（RL）中预训练Q函数的有效性。研究发现，与随机初始化相比，预训练Q函数在微调策略时通常益处甚微。这归因于预训练期间学习到的Q函数与在线微调收敛到的Q函数之间存在不匹配。为了解决这个问题，研究人员提出了策略集成初始化（IPE）方法，该方法使用多样化的策略来引导Q函数学习，在连续控制基准测试中显示出平均1.26倍的微调性能提升。 AI
影响 挑战了强化学习预训练的传统观念，可能改变策略优化的最佳实践。
排序理由 学术论文发布在arXiv上，详细介绍了新的研究发现。[lever_c_demoted from research: ic=1 ai=1.0]
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2607.27203v1 Announce Type: new Abstract: Pre-training followed by fine-tuning has become the dominant recipe for learning performant policies, and in value-based reinforcement learning (RL) this raises a natural question: given a pretrained policy, should the Q-function be…
Pre-training followed by fine-tuning has become the dominant recipe for learning performant policies, and in value-based reinforcement learning (RL) this raises a natural question: given a pretrained policy, should the Q-function be pretrained on offline data too? Conventional wi…