PulseAugur
中
实时 13:32:02

新研究改进 LLM 的 On-Policy Distillation;OpenAI 应对模型提取攻击

多篇研究论文探讨了 On-Policy Distillation (OPD) 的高级技术,这是一种通过在训练期间使用教师模型指导学生模型来提高语言模型性能的方法。这些论文引入了函数空间策略优化 (FSPO)、f-OPD、RM-OPD、OPMD、PreSO、Fisher Utility Weighting (FUW) 和 DTDD 等新方法,每种方法都旨在稳定训练、提高效率和增强推理能力。OpenAI 还报告称,他们挫败了一场旨在提取受保护模型推理数据的攻击活动,这凸显了在模型安全方面持续进行的努力。 AI

影响 这些在 on-policy distillation 方面的进展可能带来更高效、更强大的语言模型,而 OpenAI 的努力则凸显了人工智能发展中持续存在的安全挑战。

排序理由 多篇 arXiv 论文详细介绍了 on-policy distillation 的新方法,以及 OpenAI 关于模型安全的公告。

在 arXiv cs.LG 阅读 →

AI 生成摘要 · Google Gemini · 来自 107 个来源。 我们如何撰写摘要 →

新研究改进 LLM 的 On-Policy Distillation;OpenAI 应对模型提取攻击

本文如何被排名

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
多篇 arXiv 论文详细介绍了 on-policy distillation 的新方法,以及 OpenAI 关于模型安全的公告。
Source corroboration
107 independent sources
Strong cross-source corroboration — multiple independent publishers covered this within the clustering window.
Topics
paper, model release
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
57 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.
Coverage growth since scoring
+13 source(s) since last score
New sources have picked up this story since our last re-score. Score will update on the next scoring pass.

完整方法见我们的编辑标准。

报道来源 [107]

  1. OpenAI News TIER_1 Italiano(IT) ·

    颠覆一项协调的模型蒸馏活动

    Learn how OpenAI disrupted a campaign to extract protected model reasoning and is strengthening defenses against adversarial distillation.

  2. arXiv cs.CL TIER_1 English(EN) · Gaotang Li, Bhavana Dalvi Mishra, Zifeng Wang, Jun Yan, Yanfei Chen, Chun-Liang Li, Long T. Le, Rujun Han, George Lee, Hanghang Tong, Chen-Yu Lee, Tomas Pfister ·

    RubricEM:超越可验证奖励的、由规则指导的策略分解的元强化学习

    arXiv:2605.10899v2 Announce Type: replace Abstract: Deep research agents plan, search, evaluate evidence, and synthesize long-form reports. Training these agents with reinforcement learning (RL) is challenging because they operate over long horizons and their outputs lack exact a…

  3. arXiv cs.LG TIER_1 English(EN) · Yushu Wang, Renkai Yang, Jingcao Xu, Zili Geng, Haihong Tang, Yaohui Jin, Hao He ·

    GR-OPD:用于在线策略蒸馏的组相对序列信用

    arXiv:2610.23042v1 Announce Type: new Abstract: On-policy distillation (OPD) provides token-level teacher feedback, yet common sampled-token updates assign credit locally and omit the effects of earlier actions on subsequent feedback. We introduce \method{} (Group-Relative On-Pol…

  4. arXiv cs.LG TIER_1 English(EN) · Kaijian Zou, Emiliano Penaloza, MohammadHossein Rezaei, Anisha Gunjal, Mohit Raghavendra, Lu Wang, Yunzhong He, Anas Mahmoud ·

    预测最优策略下自我蒸馏的特权信息有效性

    arXiv:2610.22512v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) enables large language models (LLMs) to learn from privileged information (PI) without a separate teacher model. However, prior work largely treats PI as a fixed design choice, leaving the effects …

  5. arXiv cs.AI TIER_1 English(EN) · Haoyu Chen, Jun Bai, Yang Liu, Jianfei Zhang, Wenge Rong ·

    更强的内部教师:通过 on-policy 双向蒸馏加强 MoE 模型

    arXiv:2610.23185v1 Announce Type: cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has substantially improved Mixture-of-Experts (MoE) LLMs, but its outcome-level rewards provide only sparse supervision. On-Policy Distillation offers dense token-level feedbac…

  6. arXiv cs.AI TIER_1 English(EN) · Kai Tang, Xing Zheng, Haobo Wang, Guangcheng Zhu, Shenzhi Yang, Zhengqing Zang, Yingfan Ma, Bowen Song, Weiqiang Wang ·

    SimOPD:用于 on-policy 蒸馏的关系知识迁移

    arXiv:2610.23306v1 Announce Type: new Abstract: A key challenge in on-policy distillation (OPD) is extracting effective supervision from teacher knowledge. Supervision from token probabilities may underutilize internal knowledge, while direct hidden-state matching forces the stud…

  7. arXiv cs.CL TIER_1 English(EN) · Bingxi Hou, Guochao Jiang, Guofeng Quan, Weiqing Li, Wenfeng Feng, Guohua Liu, Yuewei Zhang ·

    Waypoint-OPD:通过门控 on-policy 蒸馏转移 RL 引起的策略转移

    arXiv:2610.21601v1 Announce Type: cross Abstract: Policy-shift distillation reuses the difference between a teacher's log probabilities before and after reinforcement learning (RL) as supervision on student-generated trajectories. However, this endpoint policy shift cannot reveal…

  8. arXiv cs.CL TIER_1 English(EN) · Yuechen Luo, Zhiyi Lai, Miao Ziqi, Zihang Xu, Jingxuan Xu, Songwei Yu, Wenqiang Zhu, Weihao Li, Junhao Fan, Haoyang Huang, Yanan Wu, Ken Deng ·

    OPRD:面向长时域编码代理的策略内提醒蒸馏

    arXiv:2610.21496v1 Announce Type: new Abstract: Reliable repository-level coding requires agents not only to generate correct code but also to preserve constraints and respond to tool feedback across long interactions. However, our diagnostic analysis shows that agents often fail…

  9. arXiv cs.AI TIER_1 English(EN) · Bangji Yang, Jiajun Fan, Zhaoyu Zhu, Hongbo Ma, Ge Liu ·

    无训练的策略内蒸馏

    arXiv:2610.22236v1 Announce Type: cross Abstract: On-policy distillation consolidates specialist models into one student at the price of rollouts, teacher passes and gradient steps for every teacher. We show that the consolidation does not need training: with forward passes only,…

  10. arXiv cs.LG TIER_1 English(EN) · Qijia He, Zequn Li, Spencer Luo, Tianle Chen, Pradyumna Narayana, Zixian Ma, Ranjay Krishna, Lan Nie, Sugato Basu ·

    DTDD:用于可靠的 On-Policy 监督的分歧触发动态蒸馏

    arXiv:2610.19302v1 Announce Type: new Abstract: On-policy distillation (OPD) provides dense token-level supervision on student rollouts, but this supervision can become unreliable when the student and teacher diverge, either because of a large initial policy gap or compounding er…

  11. arXiv cs.CL TIER_1 English(EN) · Qinglin Su, Licheng Pan, Zhen Fang, Jiakang Wang, Qingnan Ren, Yunsheng Lu, Yan Gao, Zhixuan Chu ·

    RM-OPD:生成奖励模型的在线策略蒸馏

    arXiv:2610.20174v1 Announce Type: cross Abstract: Reinforcement Learning on Mixed-domain preference data (Mix-RL) has become the mainstream approach for training general-purpose Generative Reward Models (GRMs). However, Mix-RL suffers from two critical bottlenecks: the reward spa…

  12. arXiv cs.CL TIER_1 English(EN) · Aryan Das, Arnab Bhattacharya, Vinay Kumar Verma ·

    Function-Space Temporal Self-Distillation for Policy Optimization

    arXiv:2610.19687v1 Announce Type: cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) improves reasoning in language models, but scalar outcomes provide coarse supervision over response trajectories. Self-distillation offers denser token-level guidance by compar…

  13. arXiv cs.AI TIER_1 English(EN) · Yuhang Yang, Mingxuan Xia, Chao Ye, Shuai Zhu, Shouqing Yang, Haobo Wang, Shenzhi Yang, Zhengqing Zang, Chenglong Wang ·

    PreSO:用于策略内自蒸馏的特权表示隔离

    arXiv:2610.14001v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) uses a self-teacher conditioned on privileged information to turn the sparse outcome signal of reinforcement learning with verifiable rewards (RLVR) into dense token-level supervision. Yet the te…

  14. arXiv cs.AI TIER_1 English(EN) · Yingdong Shi, Zhihong Li, Changming Li, Jingyi Yu, Quan Li, Kan Ren ·

    OPMD:超越输出监督的在线策略机制蒸馏

    arXiv:2610.14022v1 Announce Type: cross Abstract: On-policy distillation (OPD) improves language-model by supervising a student on prefixes sampled from its own policy, but its supervision remains confined to the output space. Consequently, OPD transfers what a teacher predicts w…

  15. arXiv cs.AI TIER_1 English(EN) · Shuo Yang, Zheyu Zhang, Mahito Sugiyama, Gjergji Kasneci ·

    纠正还是继续?On-Policy Distillation 从学生错误中学习什么

    arXiv:2610.18011v1 Announce Type: cross Abstract: On-policy distillation (OPD) trains a student on its own rollouts under a frozen teacher, mitigating the distribution mismatch of off-policy supervision. While prior work has primarily trained on problems a student can largely sol…

  16. arXiv cs.AI TIER_1 English(EN) · Wei Fan, Qingyue Yang, Chao-Hong Tan, Qian Chen, Wen Wang, Xiangang Li, Kejiang Chen, Weiming Zhang, Nenghai Yu ·

    LoOPSD:具有循环教师的在线策略自蒸馏

    arXiv:2610.18162v1 Announce Type: cross Abstract: On-policy self-distillation improves a model using guidance from the model itself, but it still requires a teacher that provides more useful predictions than the student. Existing approaches obtain such guidance by giving the teac…

  17. arXiv cs.AI TIER_1 English(EN) · Xianwei Chen, Shimin Zhang, Jibin Wu ·

    $\boldsymbol{f}$-OPD:利用新鲜度感知控制稳定长视域策略内蒸馏

    arXiv:2605.17862v2 Announce Type: replace-cross Abstract: Scaling on-policy distillation (OPD) for large language models (LLMs) confronts a fundamental tension: asynchronous execution is necessary for system efficiency, but structurally deviates from the ideal on-policy objective…

  18. arXiv cs.LG TIER_1 English(EN) · Narayanan Sadagopan, Yilan Chen, Vijay Lingam ·

    通过局部信任区域优化实现 on-policy 蒸馏的 Token 效用

    arXiv:2610.18883v1 Announce Type: new Abstract: Token-selection methods for on-policy distillation (OPD) prioritize student-visited positions using diverse signals, including uncertainty, teacher-student disagreement, and mismatch direction. We instead derive token importance fro…

  19. arXiv cs.LG TIER_1 English(EN) · Kaixiang Zhao, Tianrun Yu, Shangzhe Li, Yuxiao Yang, Amanda Hughes, Weitong Zhang, Taylor W. Killian ·

    通过批评指导的策略内蒸馏恢复可靠的监督

    arXiv:2610.20572v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a student model on its own rollouts with token-level supervision from a teacher, assuming that the teacher can reach a correct answer from any prefix the student writes. We call this property the …

  20. arXiv cs.LG TIER_1 English(EN) · Junze Zhu, Weihao Chen, Zhiyang Zhou, Zhen Wu, Xinyu Dai ·

    Harness Policy Distillation: Aligning Agent Policies with Runtime Semantics

    arXiv:2610.20261v1 Announce Type: new Abstract: Deploying a general-purpose model as an agent means running it inside a product-specific runtime harness that decides which actions are executable, how state advances, and when the loop may terminate. Ordinary trajectory supervision…

  21. Hugging Face Daily Papers TIER_1 English(EN) ·

    Waypoint-OPD:通过门控 on-policy 蒸馏转移 RL 引起的策略转移

    Waypoint-OPD improves policy-shift distillation by gating endpoint shifts using directional agreement between early and late teacher policy changes, boosting reasoning performance with minimal overhead.

  22. arXiv cs.IR (Information Retrieval) TIER_1 English(EN) · Litian Zhang ·

    面向深度搜索代理的信息增益校准策略内蒸馏

    On-policy distillation (OPD) has recently emerged as a promising approach for transferring the capabilities of large agents to compact students. In deep search, where agents iteratively retrieve external evidence, a teacher's preference among plausible queries before retrieval ma…

  23. Hugging Face Daily Papers TIER_1 English(EN) ·

    Function-Space Temporal Self-Distillation for Policy Optimization

    Reinforcement Learning with Verifiable Rewards (RLVR) improves reasoning in language models, but scalar outcomes provide coarse supervision over response trajectories. Self-distillation offers denser token-level guidance by comparing an unassisted policy with a feedback-informed …

  24. Hugging Face Daily Papers TIER_1 English(EN) ·

    通过批评指导的策略内蒸馏恢复可靠的监督

    On-policy distillation (OPD) trains a student model on its own rollouts with token-level supervision from a teacher, assuming that the teacher can reach a correct answer from any prefix the student writes. We call this property the recoverability of the teacher and show it fails …

  25. Hugging Face Daily Papers TIER_1 English(EN) ·

    简化和改进少样本扩散的策略内蒸馏

    Few-step distillation reduces the sampling cost of diffusion and flow models, but existing approaches often require auxiliary score models or specialized architectures. We show that simple on-policy velocity regression already provides a strong foundation for 4-NFE generation. To…

  26. arXiv cs.AI TIER_1 English(EN) · Dongyang Chen, Shudong Liu, Shubo Li, Duoxun Tang, Jinwei Liang, Zehuan Wu, Lewei Lu ·

    Branching On-Policy Distillation

    arXiv:2610.13481v1 Announce Type: new Abstract: On-policy distillation (OPD) provides teacher supervision on student-generated trajectories, but a single trajectory realizes only one continuation at each decision, limiting supervision coverage under a finite sampling budget. We p…

  27. arXiv cs.LG TIER_1 English(EN) · Zhentao Zhang, Yanzhao Zheng, Yanlin Liu, Yuanqiang Yu, Tianze Xu, Chao Ma, Jihuai Zhu, Lelun Wu, Baohua Dong, Hangcheng Zhu, Ruohui Huang, Gang Yu ·

    State-OPD:用于多轮代理的基于状态的策略蒸馏

    arXiv:2610.13238v1 Announce Type: new Abstract: On-Policy Distillation (OPD) has emerged as a powerful paradigm for transferring reasoning capabilities from frontier models to smaller models, while extending its success to multi-turn agentic environments remains challenging. We s…

  28. arXiv cs.LG TIER_1 English(EN) · Rong Wu, Daocheng Fu, Yu Yang, Xuemeng Yang, Siqi Li, Shu Zou, Yao Zhu, Botian Shi, Bo Zhang, Cong Zhang, Pinlong Cai, Jianbiao Mei ·

    重新思考同策略蒸馏的相对策略差异迁移

    arXiv:2610.13205v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a target policy on its own rollouts using token-level supervision from a stronger teacher, but typically relies on the teacher being substantially more capable than the target. Recent work relaxes…

  29. arXiv cs.CL TIER_1 English(EN) · Jingru Yang, Jinhang Wu, Minjing Shi, Shengfeng He, Xuyan Li, Man Tan, Biye Li ·

    CAST:跨分词器策略内蒸馏的信用对齐

    arXiv:2610.13257v1 Announce Type: cross Abstract: Cross-tokenizer on-policy distillation (OPD) requires aligning teacher and student sequences before their scores can be compared. Decoded-text alignment provides such correspondence at the chunk level, but leaves a distinct proble…

  30. arXiv cs.AI TIER_1 English(EN) · Anhao Zhao, Haoran Xin, Junlong Tong, Yingqi Fan, Xuan Lu, Ping Nie, Wenjie Li, Xiaoyu Shen ·

    DIAL-OPD:在同策略蒸馏中从更少的 token 中学习更多

    arXiv:2610.11659v1 Announce Type: cross Abstract: On-policy distillation (OPD) supervises student-generated trajectories with token-level teacher signals. Its sampled-token variant avoids the cost of full-vocabulary probabilities. Yet we find that training on fewer tokens can out…

  31. arXiv cs.AI TIER_1 English(EN) · Zipeng Wang, Xinpeng Dong, Yuefan Wang, Pingchen Lu, Xian Wei, Kun Kuang, Fei Wu, Zhongxiang Dai, Min Zhang ·

    MetaOPD:用于策略内蒸馏的元学习令牌加权

    arXiv:2610.11989v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a student on its own generated responses using token-level teacher supervision. However, uniform weighting overlooks differences in token learning value, while existing weighting methods rely on p…

  32. arXiv cs.LG TIER_1 English(EN) · Mohammad Khoshnazar, Mohammad Dehghani Tezerjani, Deyuan Qu, Zhiyuan Gao, Yanxiang Zhan, Jeroen Schafer, Andrew Melnik, Qing Yang, Michael Beetz ·

    CAPABLE:通过行为潜在编码实现能力感知策略适应

    arXiv:2610.11971v1 Announce Type: cross Abstract: Vision-language-action (VLA) policies assume the embodiment on which they were trained and can fail when a joint fault changes how commanded actions are physically executed. Existing fault-recovery methods often require task-speci…

  33. arXiv cs.LG TIER_1 English(EN) · Yilong Yang, Wenzhuo Shang, Yule Liu, Jiale Teng, Zhuo Ma ·

    策略对齐:策略内蒸馏中成员审计的新信号

    arXiv:2610.11423v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a student model by aligning its policy with a teacher model on trajectories generated by the student model itself. Through this process, the student policy moves toward the teacher on the prompts …

  34. arXiv cs.CL TIER_1 English(EN) · Lei Zhao, Qichao Zhao, Bowen Zuo, Qishi Zhan ·

    为何On-Policy蒸馏有时会失败:学习信号的消失

    arXiv:2610.11247v1 Announce Type: cross Abstract: On-policy distillation (OPD) enables effective capability transfer between language models, yet the mechanisms underlying its failures are not fully understood. Across code generation and mathematical reasoning, OPD with larger-sc…

  35. arXiv cs.CL TIER_1 English(EN) · Yi Wang, Rui Qian, Yu Li, Haoyang Yao, Wenjie Wang ·

    循环自我改进:用于循环语言模型的动态跨循环策略内蒸馏

    arXiv:2610.10623v1 Announce Type: cross Abstract: Looped Language Models (LoopLMs) offer a parameter efficient approach to scaling reasoning by reusing shared parameters across recurrent computation steps. Despite their promise, effective post-training of LoopLMs remains challeng…

  36. arXiv cs.AI TIER_1 English(EN) · Megha Srivastava, Cedric Colas, Dorsa Sadigh, Jacob Andreas ·

    Policy Learning with a Language Bottleneck

    arXiv:2405.04118v4 Announce Type: replace-cross Abstract: Modern AI systems such as self-driving cars and game-playing agents can achieve superhuman performance, but often lack human-like generalization, interpretability, and inter-operability with human users. Inspired by the ri…

  37. Hugging Face Daily Papers TIER_1 English(EN) ·

    Branching On-Policy Distillation

    On-policy distillation (OPD) provides teacher supervision on student-generated trajectories, but a single trajectory realizes only one continuation at each decision, limiting supervision coverage under a finite sampling budget. We propose BranchOPD (Branching On-Policy Distillati…

  38. Hugging Face Daily Papers TIER_1 English(EN) ·

    重新思考同策略蒸馏的相对策略差异迁移

    On-policy distillation (OPD) trains a target policy on its own rollouts using token-level supervision from a stronger teacher, but typically relies on the teacher being substantially more capable than the target. Recent work relaxes this requirement by transferring the relative p…

  39. Hugging Face Daily Papers TIER_1 English(EN) ·

    策略对齐:策略内蒸馏中成员审计的新信号

    On-policy distillation (OPD) trains a student model by aligning its policy with a teacher model on trajectories generated by the student model itself. Through this process, the student policy moves toward the teacher on the prompts used for distillation. However, these prompts ar…

  40. arXiv cs.LG TIER_1 English(EN) · Bohan Lin, Liyi Chen, Zhuoning Guo, Muyang Li, Qimeng Wang, Yan Gao, Yao Hu, Yudong Zhang ·

    GraphOPD:用于LLM代理的图增强On-Policy蒸馏

    arXiv:2610.08959v1 Announce Type: new Abstract: On-policy distillation post-trains large language model agents by supplying dense, step-level guidance from a teacher policy when the reinforcement-learning reward is sparse and arrives only once per trajectory. Existing instantiati…

  41. arXiv cs.LG TIER_1 English(EN) · Jonas Schweisthal, Yuxin Wang, Athiya Deviyani, Stefan Feuerriegel, Dennis Frauen ·

    高效的N模型选择策略用于推理时对齐

    arXiv:2610.09250v1 Announce Type: new Abstract: Best-of-N (BoN) is a common inference-time alignment method that selects the highest-scoring response among N samples from a reference model. Evaluating BoN policies from logged data is challenging under sample-only access because s…

  42. arXiv cs.LG TIER_1 English(EN) · Laura Fuentes-Vicente, Mathieu Even, Ga\"elle Dormion, Antoine Chambaz, Uri Shalit, Julie Josse ·

    集合值策略学习

    arXiv:2605.19830v2 Announce Type: replace Abstract: Conventional treatment policies map patient covariates to a single recommended intervention in order to maximize expected clinical outcomes. However, when multiple treatments yield statistically indistinguishable outcomes or whe…

  43. Hugging Face Daily Papers TIER_1 English(EN) ·

    循环自我改进:用于循环语言模型的动态跨循环策略内蒸馏

    Looped Language Models (LoopLMs) offer a parameter efficient approach to scaling reasoning by reusing shared parameters across recurrent computation steps. Despite their promise, effective post-training of LoopLMs remains challenging. Existing approaches either provide reward bas…

  44. arXiv cs.LG TIER_1 English(EN) · Ravenor Davion, Nick Rui ·

    特权上下文作为策略内自蒸馏中的漂移

    arXiv:2610.07842v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) trains a language model to match a copy of itself conditioned on privileged context. Existing work varies what privileged context contains and how it is produced while also changing models, data, a…

  45. arXiv cs.AI TIER_1 English(EN) · Hoang Phan, Minh Pham, Chau Pham, Chinmay Hegde, Trung Le, Qi Lei ·

    基于推理的策略优化:通过自适应推理脚手架进行学习推理

    arXiv:2610.07342v1 Announce Type: new Abstract: On-policy reinforcement learning has become a central paradigm for improving the reasoning abilities of large language models. However, its effectiveness is often limited by reward sparsity: when a model fails to discover correct tr…

  46. arXiv cs.AI TIER_1 English(EN) · Bingxi Hou, Guochao Jiang, Guofeng Quan, Weiqing Li, Wenfeng Feng, Guohua Liu, Yuewei Zhang ·

    重新思考跨分词器策略内蒸馏:从对齐覆盖到监督可靠性

    arXiv:2610.08448v1 Announce Type: cross Abstract: On-Policy Distillation (OPD) trains a student on its own generations using teacher feedback. With different tokenizers, comparing teacher and student predictions requires alignment at both sequence and vocabulary levels. In this p…

  47. arXiv cs.AI TIER_1 English(EN) · Yifei Liu, Minghao Fang, Xinyu Gu, Chengkai Yao, Mengdi Liu, Tengfei Ma, Jiangbin Zheng, Chang Yu, Zhangyang Gao ·

    E$^2$-OPSD:在策略内自蒸馏中控制熵超调

    arXiv:2610.05048v2 Announce Type: replace-cross Abstract: On-policy self-distillation (OPSD) provides dense token-level supervision without a second model: one network acts as teacher with the reference solution and as student with only the problem. We identify a specific failure…

  48. arXiv cs.LG TIER_1 English(EN) · Jaehui Hwang, Dongyoon Han, Sangdoo Yun, Byeongho Heo ·

    使用负策略采样进行策略蒸馏

    arXiv:2610.07874v1 Announce Type: new Abstract: On-policy distillation (OPD) has been widely studied as a post-training method in which a student model obtains token-level supervision from a stronger teacher on its own rollouts. Recent studies have improved OPD through alternativ…

  49. Hugging Face Daily Papers TIER_1 English(EN) ·

    使用负策略采样进行策略蒸馏

    On-policy distillation (OPD) has been widely studied as a post-training method in which a student model obtains token-level supervision from a stronger teacher on its own rollouts. Recent studies have improved OPD through alternative distillation reward formulations and teacher c…

  50. Hugging Face Daily Papers TIER_1 English(EN) ·

    重新思考跨分词器策略内蒸馏:从对齐覆盖到监督可靠性

    On-Policy Distillation (OPD) trains a student on its own generations using teacher feedback. With different tokenizers, comparing teacher and student predictions requires alignment at both sequence and vocabulary levels. In this paper, we examine whether expanding this alignment …

  51. Hugging Face Daily Papers TIER_1 English(EN) ·

    Off-Policy Merging 优于 On-Policy Self-Distillation 用于持续学习

    A long-standing goal of AI is a model that can continually learn and improve itself. On post-trained models, supervised finetuning (SFT) on new data often causes poor generalization and catastrophic forgetting. As such, the conventional wisdom is that on-policy training is a prer…

  52. arXiv cs.CL TIER_1 English(EN) · Chenglei Shen, Haoyang Yao, Weijie Yu, Song Jin, Xiao Zhang, Jun Xu ·

    从修复的推理中学习:根因引导的策略内蒸馏

    arXiv:2610.03515v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) uses reference solutions as privileged hindsight to supervise student-generated reasoning trajectories. However, reference-based guidance may explain a correct solution without addressing why the s…

  53. arXiv cs.CL TIER_1 English(EN) · Rui Li, Liyang He, Zheng Zhang, Zhenya Huang, Linbo Zhu, Qi Liu ·

    从不断演变的错误中学习:用于在线策略蒸馏的自适应迭代修复

    arXiv:2610.02700v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) supplies dense token-level feedback on trajectories sampled from the student's own policy, a richer training signal than the outcome-level rewards of reinforcement learning. This feedback comes f…

  54. arXiv cs.AI TIER_1 English(EN) · Yuxiang Zhang, Ding Cao, Shuting Cui, Lei Wang, Weijieying Ren, Tianxiang Zhao ·

    通过分段策略蒸馏学习修正推理

    arXiv:2610.02703v1 Announce Type: new Abstract: On-policy distillation (OPD) improves large language model reasoning by training students on their own rollouts with dense token-wise supervision from the teacher. However, token-wise OPD does not explicitly provide a coherent alter…

  55. arXiv cs.AI TIER_1 English(EN) · Han Cui, Jianhao Yan, Yun Luo, Hongbo Zhang, Zhizhang Fu, Yue Zhang ·

    On-Policy Distillation 的收益与崩溃:强化学习视角

    arXiv:2610.03185v1 Announce Type: new Abstract: On-policy distillation (OPD) has become an important approach to language model post-training. However, despite its performance gains, OPD can also collapse into excessively long and repetitive generation, and the mechanism underlyi…

  56. arXiv cs.AI TIER_1 English(EN) · Doseok Jang, Jon Ander Campos, Youran Qi ·

    Lexicographic Multi-Objective On-Policy Distillation

    arXiv:2610.02359v1 Announce Type: cross Abstract: Reinforcement learning from verifiable rewards (RLVR) usually optimizes answer correctness, yet useful language-model behavior also requires high-quality reasoning and concise responses. Existing multi-reward post-training methods…

  57. Hugging Face Daily Papers TIER_1 English(EN) ·

    基于推理的策略优化:通过自适应推理脚手架进行学习推理

    On-policy reinforcement learning has become a central paradigm for improving the reasoning abilities of large language models. However, its effectiveness is often limited by reward sparsity: when a model fails to discover correct trajectories for difficult problems, the optimizat…

  58. arXiv cs.AI TIER_1 English(EN) · Sophia Sirko-Galouchenko, Monika Wysoczanska, Andrei Bursuc, Nicolas Thome, Spyros Gidaris ·

    Where-OPD:具有合成场景的MLLM的空间引导策略内蒸馏

    arXiv:2610.02117v1 Announce Type: cross Abstract: On-policy self-distillation has recently emerged as an effective approach for improving language-model reasoning by supervising students with a frozen or EMA version of themselves that receives privileged information. Its applicat…

  59. arXiv cs.AI TIER_1 English(EN) · Feiyu Gavin Zhu, Qi Xu, Zhifei Deng, Zhigang Hua, Luke Simon, Jean Oh, Reid Simmons ·

    通过语义推广分析进行迭代策略优化

    arXiv:2610.01652v1 Announce Type: cross Abstract: Structured policies improve efficiency, robustness, and interpretability in imitation learning by introducing task-specific inductive bias, but existing structure generation methods rely either on extensive human input or on stati…

  60. arXiv cs.LG TIER_1 English(EN) · Zixiang Ni, Zhuo Hu, Renjie Cao, Weijie Ren, Binqin Shi, Weijia Zhang, Shuheng Cao, Zhicheng Shi, Zhenhao Zhang, Haomin Wen, Zhiyuan Hu ·

    ReTaCo:用于在线策略蒸馏的残差目标控制

    arXiv:2609.39275v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a student on its own generated prefixes with token-level teacher feedback, but transmitting or storing the teacher's full-vocabulary distribution at every token is costly. Entropy-aware OPD (EOPD)…

  61. arXiv cs.LG TIER_1 English(EN) · Jingang Zhou, Feiyu Han, Han Zhu, Yuyi Zhou, Ruiyang Zhang, Jian Xu, Sirui Gao, Qingpei Guo, Xu-Yao Zhang ·

    任务向量非孤岛:一项关于在线策略蒸馏任务向量可组合性的综合研究

    arXiv:2609.39405v1 Announce Type: new Abstract: Task vectors provide a simple mechanism for composing learned capabilities through model merging. However, the composability of task vectors produced by on-policy distillation (OPD) remains largely unexplored. OPD trains a student u…

  62. arXiv cs.AI TIER_1 English(EN) · Youngjun Jun, Kyumin Choi, Youngmin Kim, Seonghyun Jin, Sunwoo Park, Jangho Park, Jong Chul Ye ·

    DriftOPD:序列级反向KL蒸馏用于单步VLA策略

    arXiv:2610.00317v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models increasingly rely on action experts that generate short action chunks under receding-horizon control. While chunk-level training is convenient across robot embodiments, it optimizes local action…

  63. Hugging Face Daily Papers TIER_1 English(EN) ·

    On-Policy Distillation 中的收益与崩溃:强化学习视角

    On-policy distillation (OPD) has become an important approach to language model post-training. However, despite its performance gains, OPD can also collapse into excessively long and repetitive generation, and the mechanism underlying these divergent outcomes remains poorly under…

  64. arXiv cs.AI TIER_1 English(EN) · Qiwei Di, Xuheng Li, Kaixuan Ji, Chenggong Zhang, Heyang Zhao, Quanquan Gu ·

    理解离线 vs 在线蒸馏:两种不同训练目标的叙事

    arXiv:2609.38666v1 Announce Type: cross Abstract: On-policy distillation (OPD) learns from teacher feedback on student-generated responses and has shown promise in reducing forgetting relative to supervised fine-tuning (SFT). However, its benefits and fragility remain incompletel…

  65. arXiv cs.AI TIER_1 English(EN) · Fang Wu, Da Xing, Yanjie Huang, Junxi Wang, Ji Wang, Hejia Geng, Guancheng Wan, Bowen Zuo, Xiaomin Li, Shixiang Tang, Xinyu Xiang, Zehong Wang, Shiyi Du, Peng Xia, Shuangjia Zheng, Yining Hong, Li Erran Li, Jure Leskovec, Yejin Choi ·

    UniEvo-VL:一种用于多模态模型自我改进的在线策略自蒸馏训练方法

    arXiv:2609.38721v1 Announce Type: new Abstract: Modern multimodal models bring generation and understanding into a single unified system, which enables them to provide and learn from their own feedback. Motivated by this unified capacity, we introduce UniEvo-VL, a self-evolving f…

  66. arXiv cs.AI TIER_1 English(EN) · Weijie Ren, Yanwen Zhang, Hao Li, Zhuolin Qi, Hengyi Zhang, Naibo Wang ·

    OPSRD:基于策略的自角色蒸馏

    arXiv:2609.39884v1 Announce Type: cross Abstract: Role prompting elicits specialized behavior from large language models through an expert identity, offering a lightweight way to guide reasoning on demanding tasks. However, evaluating or distilling complete role-prompted answers …

  67. arXiv cs.CL TIER_1 English(EN) · Xincheng Wei, Yifan Ding, Yoshua Li, Yuquan Lu, Ziheng Li, Yi Lu, Dongsheng Ma, Rongxiang Weng, Xunliang Cai ·

    更好的监督就在附近:邻域在线策略自蒸馏

    arXiv:2609.39687v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) trains mathematical reasoning models using a privileged teacher that sees a reference solution and supervises student-sampled prefixes. Standard OPSD uses one fixed parameter setting at every state…

  68. arXiv cs.CL TIER_1 English(EN) · Yang Li, Gongle Xue, Yuheng Yuan, Yijia Guo, Shizhe Zhang, Liwen Hu, Lei Ma ·

    诊断用于推理语言模型的 On-Policy 自蒸馏

    arXiv:2609.39118v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) has attracted growing interest as a promising approach to improve the reasoning ability of language models. Without external rewards nor a separate stronger teacher, the self-teacher with privilege…

  69. arXiv cs.CL TIER_1 English(EN) · Di Huang, Hao Li, Yixin Chen, Fuhai Li ·

    当裁剪逆转修正:逐点前向KL策略内自蒸馏的失效动力学

    arXiv:2609.38995v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) trains a student on its own generated responses using feedback from the same model conditioned on privileged information. On mathematical reasoning, the original OPSD study finds that stylistic tok…

  70. arXiv cs.AI TIER_1 English(EN) · Hantao Yu, Xiaoxue Han, Udaya Ghai, Ferhat Erata, Joseph Lilien, Aman Goel, Ali Torkamani ·

    指令而非回答:在策略内指令特权用于上下文蒸馏

    arXiv:2609.32201v2 Announce Type: replace Abstract: On-Policy Context Distillation (OPCD) has recently emerged as a powerful technique for transferring context to student models and for self-improvement. In OPCD, the teacher is conditioned on privileged information, and the goal …

  71. arXiv cs.AI TIER_1 English(EN) · Jiaxuan Wang, Xuan Ouyang, Zhiyu Chen, Yulan Hu, Lan-Zhe Guo ·

    STEPS:用于推理的选择性策略内蒸馏

    arXiv:2605.10194v2 Announce Type: replace Abstract: On-policy self-distillation (OPSD) uses a model as its own teacher under privileged context, providing token-level supervision on the model's own reasoning trajectories. However, persistent all-token guidance can degrade trainin…

  72. arXiv cs.AI TIER_1 English(EN) · Zhenxing Zhang, Jiayan Teng, Wenxu Wu, Zhuoyi Yang, Jiazheng Xu, Wendi Zheng, Jie Tang, Dan Guo, Meng Wang ·

    GFD-OPD: 跨尺度扩散模型的引导折叠策略蒸馏

    arXiv:2609.39692v1 Announce Type: cross Abstract: On-policy distillation (OPD) has demonstrated two important capabilities in language models: compressing large teachers into smaller students and merging expert models into a single model. Existing diffusion OPD, however, mostly f…

  73. Hugging Face Daily Papers TIER_1 English(EN) ·

    Where-OPD:具有合成场景的MLLM的空间引导策略内蒸馏

    On-policy self-distillation has recently emerged as an effective approach for improving language-model reasoning by supervising students with a frozen or EMA version of themselves that receives privileged information. Its application to multimodal large language models (MLLMs), h…

  74. arXiv cs.LG TIER_1 English(EN) · Yuxiao Yang, Shangzhe Li, Tianrun Yu, Kaixiang Zhao, Taylor W. Killian, Weitong Zhang ·

    REVO:通过方差引导复用来实现滚动高效的离策略蒸馏

    arXiv:2609.37500v1 Announce Type: new Abstract: On-policy distillation (OPD) trains language models using dense token-level teacher supervision on student-generated trajectories. However, its reliance on frequently refreshed student rollouts often incurs substantial generation co…

  75. arXiv cs.AI TIER_1 English(EN) · Hongyang Li, Yiming Zhu, Xiao Li, Caesar Wu, Said Mammar, Pascal Bouvry ·

    无停歇求解:小规模上的在线策略蒸馏

    arXiv:2609.37326v1 Announce Type: new Abstract: On-policy distillation, where a student learns from a stronger teacher's feedback on its own outputs, is a common way to pass reasoning to smaller models. We analyze what it transfers at small scale, distilling Qwen3-8B into Qwen3 4…

  76. arXiv cs.AI TIER_1 English(EN) · Jiaxuan Wang, Jiafei Lyu, Yuchen Cai, Siye Wu, Pengyuan Wang, Jiashun Liu, Xiang Cheng, Kai Yang, Yangkun Chen, Saiyong Yang, Lan-Zhe Guo ·

    超越提示词数量:数据如何塑造同策略蒸馏中的迁移

    arXiv:2609.37377v1 Announce Type: new Abstract: On-policy distillation (OPD) trains students using teacher feedback on their own sampled responses, yet how prompt choice shapes transfer across teacher-student pairs remains poorly understood. We systematically study prompt quantit…

  77. arXiv cs.AI TIER_1 English(EN) · Shutong Wu, Xiwen Chen, Brendan Rappazzo, Daiheng Zhang, Anderson Schneider, Yuriy Nevmyvaka, Jiawei Zhang ·

    具有双向提议-验证的交互式策略蒸馏

    arXiv:2609.36546v1 Announce Type: cross Abstract: On-policy distillation (OPD) trains a student model on its self-generated trajectories with dense token-level teacher feedback. However, naive OPD may suffer from teacher unanchoring, where the student's reasoning trajectory drift…

  78. arXiv cs.AI TIER_1 English(EN) · Youxu Shi, Yifan Sun, Dacheng Yin, Haomiao Tang, Guangting Wang, Fengyun Rao, Jing Lyu, Dong Liu ·

    插值策略蒸馏:离策略与在策略蒸馏之间的可控连续体

    arXiv:2609.37170v1 Announce Type: cross Abstract: Off-policy and on-policy distillation have traditionally been formulated as separate paradigms, each favoring a different property of distillation trajectories. Teacher-generated (off-policy) traces are typically high-quality but …

  79. arXiv cs.AI TIER_1 English(EN) · Zizhuo Lin, Quanling Liu, Yi Yang, Yawei Luo ·

    共享前缀隐藏了什么:在线策略蒸馏的轨迹 Dropout

    arXiv:2609.33455v2 Announce Type: replace Abstract: On-policy distillation (OPD) trains a student model on its own trajectories using dense token-level feedback from a stronger teacher model. Since each update is conditioned on the reasoning prefix already generated by the studen…

  80. arXiv cs.AI TIER_1 English(EN) · Taesung Kwon, Jangho Park, Sunwoo Park, Youngmin Kim, Seonghyun Jin, Youngjun Jun, Kyumin Choi, Jong Chul Ye ·

    别扔掉尾巴:加速政策的行动升级再造

    arXiv:2609.34911v2 Announce Type: replace-cross Abstract: Modern robot policies predict a chunk of future actions from a single observation, execute only a prefix, and discard the rest before replanning. Choosing the length of this prefix, the execution horizon, poses a trade-off…

  81. arXiv cs.CL TIER_1 English(EN) · Wanqi Ren, Jianxiang Wang, Danxuan Liu, Linyi Ding, Yuan Zhang ·

    通过在线策略蒸馏学习思考模式优势

    arXiv:2609.37044v1 Announce Type: new Abstract: Explicit intermediate reasoning gives large language models (LLMs) a stronger problem-solving mode. We study learning from this think-mode advantage via on-policy distillation (OPD). OPD preserves student-generated trajectories and …

  82. arXiv cs.CL TIER_1 English(EN) · Zubin Zheng, Jiahao Wu, Shaofeng Zhang, Zhirui Zhang, Yew-Soon Ong, Shengcai Liu ·

    先行动,后推理:通过参考条件逆动力学加速多轮智能体的策略内蒸馏

    arXiv:2609.36608v1 Announce Type: cross Abstract: On-policy distillation (OPD) trains multi-turn language agents with dense teacher supervision on student-generated responses. However, standard think-then-act rollouts require lengthy reasoning before each short action, delaying e…

  83. arXiv cs.CL TIER_1 English(EN) · Safaeid Hossain Arib, Rabeya Akter, Ismam Nur Swapnil, Md. Faiyaz Abdullah Sayeedi, Tasnim Mohiuddin, Md Mofijul Islam ·

    教会自己看向何处:基于策略的注意力自蒸馏用于推理

    arXiv:2609.33200v2 Announce Type: replace-cross Abstract: On-policy self-distillation trains reasoning models on their own trajectories using dense token distribution guidance from a privileged teacher with access to a verified solution. This supervision transfers what the teache…

  84. arXiv cs.LG TIER_1 English(EN) · Hao Li, MeiJia Chen, Weijie Ren, Donghan Li, Zijun Tian, Jingchun Huang, Naibo Wang ·

    教师是方向而非终点:外推在线策略蒸馏中的RL诱导表示残差

    arXiv:2609.36484v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a student to match the teacher's next-token distributions on the student's own trajectories and has yielded substantial empirical gains. Generalized variants allow the student to surpass the teach…

  85. arXiv cs.LG TIER_1 English(EN) · Mingfeng Lin, Chengfei Cai, Lin Xu, Chengqian Ma, Yuxiang Wei, Liang Han ·

    PE-OPSD:通过 On-Policy Self-Distillation 将 Prompt 增强内化到 Flow-matching 模型中

    arXiv:2609.36638v1 Announce Type: new Abstract: Text-to-image users often provide concise and underspecified prompts, whereas generative models benefit from detailed textual conditions for reliable instruction following. Existing systems bridge this gap with Prompt Enhancers (PEs…

  86. arXiv cs.LG TIER_1 English(EN) · Muyang Li, Jie Yang, Zhengyu Fang, Junchao Zhu, Zhengkun Xiao, Ruining Deng, Zhe Jiang, Shigang Chen ·

    PR-OPD:用于代理强化学习的特权表示策略内自蒸馏

    arXiv:2609.36642v1 Announce Type: new Abstract: Language-model agents are usually trained by reinforcement learning from one reward per episode, and privileged self-distillation enriches it by letting the same policy, given a skill, teach its skill-free self through token probabi…

  87. arXiv cs.LG TIER_1 English(EN) · Xiaohan Yi, Wen Luo, Yani Huang, Junfeng Zhan, Asher Qin, Peilin Zhao, Xi Xiao ·

    Graph-Conditioned On-Policy Agent Distillation from Off-the-Shelf Teachers

    arXiv:2609.37522v1 Announce Type: new Abstract: On-policy distillation (OPD) trains compact language agents with teacher feedback on student-generated trajectories. In multi-turn tasks, compounding errors can move students beyond the teacher's effective supervision. We introduce …

  88. arXiv cs.LG TIER_1 English(EN) · Zhiqi Li, Bo Zhu ·

    FlowMap-OPD:推出用于少步流图生成器按策略蒸馏的核分离

    arXiv:2609.37851v1 Announce Type: cross Abstract: Few-step flow-map generators, including MeanFlow and consistency models, enable efficient sampling through long-range transport, yet their on-policy distillation remains underexplored. We introduce FlowMap-OPD, an on-policy distil…

  89. Hugging Face Daily Papers TIER_1 English(EN) ·

    UniEvo-VL:一种用于多模态模型自我改进的在线策略自蒸馏训练方法

    Modern multimodal models bring generation and understanding into a single unified system, which enables them to provide and learn from their own feedback. Motivated by this unified capacity, we introduce UniEvo-VL, a self-evolving framework for multimodal models to learn from thi…

  90. Hugging Face Daily Papers TIER_1 English(EN) ·

    更好的监督就在附近:邻域策略内自蒸馏

    On-policy self-distillation (OPSD) trains mathematical reasoning models using a privileged teacher that sees a reference solution and supervises student-sampled prefixes. Standard OPSD uses one fixed parameter setting at every state, but nearby settings may offer additional super…

  91. Hugging Face Daily Papers TIER_1 English(EN) ·

    UniEvo-VL:一种用于多模态模型自我改进的在线策略自蒸馏训练方法

    Modern multimodal models bring generation and understanding into a single unified system, which enables them to provide and learn from their own feedback. Motivated by this unified capacity, we introduce UniEvo-VL, a self-evolving framework for multimodal models to learn from thi…

  92. Hugging Face Daily Papers TIER_1 English(EN) ·

    FlowMap-OPD:用于少样本流图生成器按策略蒸馏的核分离

    Few-step flow-map generators, including MeanFlow and consistency models, enable efficient sampling through long-range transport, yet their on-policy distillation remains underexplored. We introduce FlowMap-OPD, an on-policy distillation framework that separates student-state acqu…

  93. arXiv cs.AI TIER_1 English(EN) · Taeckyung Lee, Rinat Amankos, Jeonghye Kim, Hyungjun Yoon, Woogyeol Jin, Sung-Ju Lee ·

    TISD:具有轨迹干预的策略内自蒸馏

    arXiv:2609.30878v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) provides dense teacher targets, but evaluates them only along student-sampled rollouts. When the privileged teacher favors an alternative action at a visited prefix, OPSD can provide a target for t…

  94. Hugging Face Daily Papers TIER_1 English(EN) ·

    OPMD:超越输出监督的策略内机制蒸馏

    On-Policy Mechanism Distillation improves language model distillation by transferring teacher intermediate computations compatible with the student's architecture, yielding stronger reasoning and internalization.

  95. Hugging Face Daily Papers TIER_1 English(EN) ·

    PR-OPD:用于 Agentic 强化学习的特权表示 On-policy 自蒸馏

    Language-model agents are usually trained by reinforcement learning from one reward per episode, and privileged self-distillation enriches it by letting the same policy, given a skill, teach its skill-free self through token probabilities. However, we identify two phenomena that …

  96. Hugging Face Daily Papers TIER_1 English(EN) ·

    教师是方向而非终点:外推在线策略蒸馏中RL诱导的表征残差

    On-policy distillation (OPD) trains a student to match the teacher's next-token distributions on the student's own trajectories and has yielded substantial empirical gains. Generalized variants allow the student to surpass the teacher by extrapolating an implicit reward in output…

  97. Hugging Face Daily Papers TIER_1 English(EN) ·

    先行动,后推理:通过参考条件逆动力学加速多轮智能体的策略内蒸馏

    On-policy distillation (OPD) trains multi-turn language agents with dense teacher supervision on student-generated responses. However, standard think-then-act rollouts require lengthy reasoning before each short action, delaying environment transitions and experience collection. …

  98. Hugging Face Daily Papers TIER_1 English(EN) ·

    超越提示词数量:数据如何塑造同策略蒸馏中的迁移

    On-policy distillation (OPD) trains students using teacher feedback on their own sampled responses, yet how prompt choice shapes transfer across teacher-student pairs remains poorly understood. We systematically study prompt quantity, source, and selection across RL- and SFT-cont…

  99. Hugging Face Daily Papers TIER_1 English(EN) ·

    On-Policy 或 Off-Policy 学习?蒸馏动力学的系统研究

    On-policy learning has been argued to reduce catastrophic forgetting, produce sparser parameter updates, and improve generalisation. However, existing comparisons between supervised fine-tuning and reinforcement learning vary many factors simultaneously, making the contribution o…

  100. Hugging Face Daily Papers TIER_1 English(EN) ·

    从稀疏交叉编码器机制可解释性视角理解在线策略蒸馏

    On-policy distillation (OPD) is a widely adopted post-training technique for LLM reasoning. It is commonly believed to transfer knowledge from a stronger teacher, yet what OPD actually distills into the student's internal representations remains unclear. We study this question wi…

  101. Hugging Face Daily Papers TIER_1 English(EN) ·

    同族同策略蒸馏的规模化性质

    *Reinforcement learning (RL)* can induce substantial reasoning capabilities in large language models (LLMs), but how much of this capability transfers across model scales, and how quickly, remains unclear. We study the scaling properties of *on-policy distillation (OPD)* across *…

  102. arXiv stat.ML TIER_1 English(EN) · Linjian Meng, Siyuan Gan, YuHan Li, Xiran Wang, Ziyang Ding, Ditang Gou, Yiming Wu, Zhen Zhao ·

    无偏的 On-Policy Distillation 的 Top-$k$ 估计

    arXiv:2609.34447v2 Announce Type: replace-cross Abstract: On-policy distillation (OPD) is becoming an important component of large language model (LLM) post-training for transferring the reasoning capability of a strong teacher LLM to a weaker student LLM. OPD trains the student …

  103. arXiv cs.CV TIER_1 English(EN) · Junxian Li, Ruixuan Yang, Tianao Zhang, Tiange Xu, Weisheng Dong, Yulun Zhang ·

    更少的Token,更多的自学:用于极端视觉Token缩减的On-Policy自蒸馏

    arXiv:2609.32353v2 Announce Type: replace Abstract: Visual token reduction is an effective way to accelerate multimodal large language models (MLLMs), but performance deteriorates rapidly under extremely low token budgets. Existing work has explored both visual-token selection an…

  104. arXiv cs.CV TIER_1 English(EN) · Amirhossein Tighkhorshid, Zahra Dehghanian, Hamid R. Rabiee ·

    ReDiF:通过强化学习实现资源高效的少样本扩散蒸馏

    arXiv:2512.22802v2 Announce Type: replace-cross Abstract: Step distillation accelerates diffusion sampling by training a few-step student to imitate a many-step teacher, but distillation itself remains expensive. Typically, this requires thousands of GPU-hours and a large pre-gen…

  105. arXiv cs.CV TIER_1 English(EN) · Pengyang Ling, Jiazi Bu, Yujie Zhou, Yibin Wang, Zeqiang Lai, Xiaoxiao Ma, Yi Jin, Huaian Chen, Yuhang Zang ·

    CapField-OPD:通过联合锚定多教师策略内蒸馏学习流模型的连续能力场

    arXiv:2609.34658v2 Announce Type: replace Abstract: Reward-specialized post-training produces strong experts for flow-based generative models, while multi-teacher on-policy distillation (OPD) consolidates their capabilities into a single student. Existing methods, however, route …

  106. Mastodon — mastodon.social TIER_1 English(EN) · opensourceaitech ·

    📄 Waypoint-OPD:通过门控 on-policy 蒸馏转移 RL 引起的策略转移在 Hugging Face 上获得 663 票——这是有效策略转移的信号

    📄 Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation hit 663 upvotes on Hugging Face – a signal that efficient policy transfer via distillation is a hot topic right now. Open the paper page to see why. https:// huggingface.co/papers/2610.216 01 #…

  107. Mastodon — mastodon.social TIER_1 English(EN) · opensourceaitech ·

    📄 《重新思考跨分词器策略内蒸馏:从对齐覆盖到监督可靠性》认为真正的瓶颈并非覆盖率而是监督

    📄 'Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliability' argues the real bottleneck isn't coverage but supervision reliability. 153 upvotes on Hugging Face. https:// huggingface.co/papers/2610.084 48 # AI # MachineLearning # Resear…