English(EN) Post-Training Shifts Confidence: A Three-Stage Analysis of How SFT, RL, and OPD Shape CoT Calibration

新研究分析SFT、RL和OPD如何塑造LLM推理置信度

作者 PulseAugur 编辑部 · [1 个来源] · 2026-07-21 04:00

一篇新研究论文提出了一个三阶段框架，用于分析监督微调（SFT）、强化学习（RL）和在线策略蒸馏（OPD）如何影响大型语言模型在推理过程中的置信度校准。该研究在数学推理基准上进行，发现OPD对于预推理置信度最有效，SFT在提前停止方面表现最佳，而RL为答案聚合提供了最可靠的信号。研究人员还开发了PosConf，一种位置感知置信度策略，通过利用来自可靠相对位置区间的置信度信号，改进了RL答案聚合和OPD提前停止。 AI

影响这项研究通过改进生成过程中置信度的测量和利用方式，有望提高LLM推理的可靠性和效率。

排序理由该集群包含一篇学术论文，详细介绍了大型语言模型的新分析框架和方法。[lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.CL 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。我们如何撰写摘要 →

本文如何被排名

Signal score: 0 / 100; Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket: Tool; 该集群包含一篇学术论文，详细介绍了大型语言模型的新分析框架和方法。[lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration: Single-source cluster; Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics: paper, model release; Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance: High; Clearly on-topic for AI-industry coverage.
Story freshness: 48 days old; Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

完整方法见我们的编辑标准。

报道来源 [1]

arXiv cs.CL TIER_1 English(EN) · Shuhao Li, Guodong Du, Anhao Zhao, Wanyu Lin, Tianyu Yuan, Xiaoyu Shen · 2026-07-21 04:00

训练后校准的转变：SFT、RL和OPD如何塑造CoT校准的三阶段分析

arXiv:2607.13753v2 Announce Type: replace Abstract: Large language models have made strong reasoning gains through supervised fine-tuning, reinforcement learning, and on-policy distillation, yet these post-training methods are usually evaluated only by final-answer accuracy. We s…

本文如何被排名

报道来源 [1]

训练后校准的转变：SFT、RL和OPD如何塑造CoT校准的三阶段分析

相关实体

相关话题