PulseAugur
EN
LIVE 13:52:19

New research refines on-policy distillation for LLMs; OpenAI counters model extraction

Multiple research papers explore advanced techniques for On-Policy Distillation (OPD), a method to improve language model performance by using a teacher model to guide a student model during training. These papers introduce novel approaches like Function-Space Policy Optimization (FSPO), f-OPD, RM-OPD, OPMD, PreSO, Fisher Utility Weighting (FUW), and DTDD, each aiming to stabilize training, improve efficiency, and enhance reasoning capabilities. OpenAI also reported disrupting a campaign to extract protected model reasoning, highlighting ongoing efforts in model security. AI

IMPACT These advancements in on-policy distillation could lead to more efficient and capable language models, while OpenAI's efforts highlight the ongoing security challenges in AI development.

RANK_REASON Multiple arXiv papers detailing new methods for on-policy distillation, alongside OpenAI's announcement about model security.

Read on arXiv cs.LG →

AI-generated summary · Google Gemini · from 107 sources. How we write summaries →

New research refines on-policy distillation for LLMs; OpenAI counters model extraction

How we ranked this

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
Multiple arXiv papers detailing new methods for on-policy distillation, alongside OpenAI's announcement about model security.
Source corroboration
107 independent sources
Strong cross-source corroboration — multiple independent publishers covered this within the clustering window.
Topics
paper, model release
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
57 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.
Coverage growth since scoring
+13 source(s) since last score
New sources have picked up this story since our last re-score. Score will update on the next scoring pass.

Full methodology in our editorial standards.

COVERAGE [107]

  1. OpenAI News TIER_1 Italiano(IT) ·

    Disrupting a coordinated model-distillation campaign

    Learn how OpenAI disrupted a campaign to extract protected model reasoning and is strengthening defenses against adversarial distillation.

  2. arXiv cs.CL TIER_1 English(EN) · Gaotang Li, Bhavana Dalvi Mishra, Zifeng Wang, Jun Yan, Yanfei Chen, Chun-Liang Li, Long T. Le, Rujun Han, George Lee, Hanghang Tong, Chen-Yu Lee, Tomas Pfister ·

    RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewards

    arXiv:2605.10899v2 Announce Type: replace Abstract: Deep research agents plan, search, evaluate evidence, and synthesize long-form reports. Training these agents with reinforcement learning (RL) is challenging because they operate over long horizons and their outputs lack exact a…

  3. arXiv cs.LG TIER_1 English(EN) · Yushu Wang, Renkai Yang, Jingcao Xu, Zili Geng, Haihong Tang, Yaohui Jin, Hao He ·

    GR-OPD: Group-Relative Sequence Credit for On-Policy Distillation

    arXiv:2610.23042v1 Announce Type: new Abstract: On-policy distillation (OPD) provides token-level teacher feedback, yet common sampled-token updates assign credit locally and omit the effects of earlier actions on subsequent feedback. We introduce \method{} (Group-Relative On-Pol…

  4. arXiv cs.LG TIER_1 English(EN) · Kaijian Zou, Emiliano Penaloza, MohammadHossein Rezaei, Anisha Gunjal, Mohit Raghavendra, Lu Wang, Yunzhong He, Anas Mahmoud ·

    Predicting Privileged Information Effectiveness for On-Policy Self-Distillation

    arXiv:2610.22512v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) enables large language models (LLMs) to learn from privileged information (PI) without a separate teacher model. However, prior work largely treats PI as a fixed design choice, leaving the effects …

  5. arXiv cs.AI TIER_1 English(EN) · Haoyu Chen, Jun Bai, Yang Liu, Jianfei Zhang, Wenge Rong ·

    A Stronger Teacher Within: Strengthening MoE Models via On-Policy Bidirectional Distillation

    arXiv:2610.23185v1 Announce Type: cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has substantially improved Mixture-of-Experts (MoE) LLMs, but its outcome-level rewards provide only sparse supervision. On-Policy Distillation offers dense token-level feedbac…

  6. arXiv cs.AI TIER_1 English(EN) · Kai Tang, Xing Zheng, Haobo Wang, Guangcheng Zhu, Shenzhi Yang, Zhengqing Zang, Yingfan Ma, Bowen Song, Weiqiang Wang ·

    SimOPD: Relational Knowledge Transfer for On-Policy Distillation

    arXiv:2610.23306v1 Announce Type: new Abstract: A key challenge in on-policy distillation (OPD) is extracting effective supervision from teacher knowledge. Supervision from token probabilities may underutilize internal knowledge, while direct hidden-state matching forces the stud…

  7. arXiv cs.CL TIER_1 English(EN) · Bingxi Hou, Guochao Jiang, Guofeng Quan, Weiqing Li, Wenfeng Feng, Guohua Liu, Yuewei Zhang ·

    Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation

    arXiv:2610.21601v1 Announce Type: cross Abstract: Policy-shift distillation reuses the difference between a teacher's log probabilities before and after reinforcement learning (RL) as supervision on student-generated trajectories. However, this endpoint policy shift cannot reveal…

  8. arXiv cs.CL TIER_1 English(EN) · Yuechen Luo, Zhiyi Lai, Miao Ziqi, Zihang Xu, Jingxuan Xu, Songwei Yu, Wenqiang Zhu, Weihao Li, Junhao Fan, Haoyang Huang, Yanan Wu, Ken Deng ·

    OPRD: On-Policy Reminder Distillation for Long-Horizon Coding Agents

    arXiv:2610.21496v1 Announce Type: new Abstract: Reliable repository-level coding requires agents not only to generate correct code but also to preserve constraints and respond to tool feedback across long interactions. However, our diagnostic analysis shows that agents often fail…

  9. arXiv cs.AI TIER_1 English(EN) · Bangji Yang, Jiajun Fan, Zhaoyu Zhu, Hongbo Ma, Ge Liu ·

    Training-Free On-Policy Distillation

    arXiv:2610.22236v1 Announce Type: cross Abstract: On-policy distillation consolidates specialist models into one student at the price of rollouts, teacher passes and gradient steps for every teacher. We show that the consolidation does not need training: with forward passes only,…

  10. arXiv cs.LG TIER_1 English(EN) · Qijia He, Zequn Li, Spencer Luo, Tianle Chen, Pradyumna Narayana, Zixian Ma, Ranjay Krishna, Lan Nie, Sugato Basu ·

    DTDD: Divergence-Triggered Dynamic Distillation for Reliable On-Policy Supervision

    arXiv:2610.19302v1 Announce Type: new Abstract: On-policy distillation (OPD) provides dense token-level supervision on student rollouts, but this supervision can become unreliable when the student and teacher diverge, either because of a large initial policy gap or compounding er…

  11. arXiv cs.CL TIER_1 English(EN) · Qinglin Su, Licheng Pan, Zhen Fang, Jiakang Wang, Qingnan Ren, Yunsheng Lu, Yan Gao, Zhixuan Chu ·

    RM-OPD: On-Policy Distillation for Generative Reward Models

    arXiv:2610.20174v1 Announce Type: cross Abstract: Reinforcement Learning on Mixed-domain preference data (Mix-RL) has become the mainstream approach for training general-purpose Generative Reward Models (GRMs). However, Mix-RL suffers from two critical bottlenecks: the reward spa…

  12. arXiv cs.CL TIER_1 English(EN) · Aryan Das, Arnab Bhattacharya, Vinay Kumar Verma ·

    Function-Space Temporal Self-Distillation for Policy Optimization

    arXiv:2610.19687v1 Announce Type: cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) improves reasoning in language models, but scalar outcomes provide coarse supervision over response trajectories. Self-distillation offers denser token-level guidance by compar…

  13. arXiv cs.AI TIER_1 English(EN) · Yuhang Yang, Mingxuan Xia, Chao Ye, Shuai Zhu, Shouqing Yang, Haobo Wang, Shenzhi Yang, Zhengqing Zang, Chenglong Wang ·

    PreSO: Privileged Representation Isolation for On-Policy Self-Distillation

    arXiv:2610.14001v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) uses a self-teacher conditioned on privileged information to turn the sparse outcome signal of reinforcement learning with verifiable rewards (RLVR) into dense token-level supervision. Yet the te…

  14. arXiv cs.AI TIER_1 English(EN) · Yingdong Shi, Zhihong Li, Changming Li, Jingyi Yu, Quan Li, Kan Ren ·

    OPMD: On-Policy Mechanism Distillation Beyond Output Supervision

    arXiv:2610.14022v1 Announce Type: cross Abstract: On-policy distillation (OPD) improves language-model by supervising a student on prefixes sampled from its own policy, but its supervision remains confined to the output space. Consequently, OPD transfers what a teacher predicts w…

  15. arXiv cs.AI TIER_1 English(EN) · Shuo Yang, Zheyu Zhang, Mahito Sugiyama, Gjergji Kasneci ·

    Correcting or Continuing? What On-Policy Distillation Learns from Student Errors

    arXiv:2610.18011v1 Announce Type: cross Abstract: On-policy distillation (OPD) trains a student on its own rollouts under a frozen teacher, mitigating the distribution mismatch of off-policy supervision. While prior work has primarily trained on problems a student can largely sol…

  16. arXiv cs.AI TIER_1 English(EN) · Wei Fan, Qingyue Yang, Chao-Hong Tan, Qian Chen, Wen Wang, Xiangang Li, Kejiang Chen, Weiming Zhang, Nenghai Yu ·

    LoOPSD: On-Policy Self-Distillation with Looped Teachers

    arXiv:2610.18162v1 Announce Type: cross Abstract: On-policy self-distillation improves a model using guidance from the model itself, but it still requires a teacher that provides more useful predictions than the student. Existing approaches obtain such guidance by giving the teac…

  17. arXiv cs.AI TIER_1 English(EN) · Xianwei Chen, Shimin Zhang, Jibin Wu ·

    $\boldsymbol{f}$-OPD: Stabilizing Long-Horizon On-Policy Distillation with Freshness-Aware Control

    arXiv:2605.17862v2 Announce Type: replace-cross Abstract: Scaling on-policy distillation (OPD) for large language models (LLMs) confronts a fundamental tension: asynchronous execution is necessary for system efficiency, but structurally deviates from the ideal on-policy objective…

  18. arXiv cs.LG TIER_1 English(EN) · Narayanan Sadagopan, Yilan Chen, Vijay Lingam ·

    Token Utility for On-Policy Distillation via Local Trust-Region Optimization

    arXiv:2610.18883v1 Announce Type: new Abstract: Token-selection methods for on-policy distillation (OPD) prioritize student-visited positions using diverse signals, including uncertainty, teacher-student disagreement, and mismatch direction. We instead derive token importance fro…

  19. arXiv cs.LG TIER_1 English(EN) · Kaixiang Zhao, Tianrun Yu, Shangzhe Li, Yuxiao Yang, Amanda Hughes, Weitong Zhang, Taylor W. Killian ·

    Recovering Reliable Supervision via Critique-Guided On-Policy Distillation

    arXiv:2610.20572v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a student model on its own rollouts with token-level supervision from a teacher, assuming that the teacher can reach a correct answer from any prefix the student writes. We call this property the …

  20. arXiv cs.LG TIER_1 English(EN) · Junze Zhu, Weihao Chen, Zhiyang Zhou, Zhen Wu, Xinyu Dai ·

    Harness Policy Distillation: Aligning Agent Policies with Runtime Semantics

    arXiv:2610.20261v1 Announce Type: new Abstract: Deploying a general-purpose model as an agent means running it inside a product-specific runtime harness that decides which actions are executable, how state advances, and when the loop may terminate. Ordinary trajectory supervision…

  21. Hugging Face Daily Papers TIER_1 English(EN) ·

    Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation

    Waypoint-OPD improves policy-shift distillation by gating endpoint shifts using directional agreement between early and late teacher policy changes, boosting reasoning performance with minimal overhead.

  22. arXiv cs.IR (Information Retrieval) TIER_1 English(EN) · Litian Zhang ·

    Information-Gain Calibrated On-Policy Distillation for Deep Search Agents

    On-policy distillation (OPD) has recently emerged as a promising approach for transferring the capabilities of large agents to compact students. In deep search, where agents iteratively retrieve external evidence, a teacher's preference among plausible queries before retrieval ma…

  23. Hugging Face Daily Papers TIER_1 English(EN) ·

    Function-Space Temporal Self-Distillation for Policy Optimization

    Reinforcement Learning with Verifiable Rewards (RLVR) improves reasoning in language models, but scalar outcomes provide coarse supervision over response trajectories. Self-distillation offers denser token-level guidance by comparing an unassisted policy with a feedback-informed …

  24. Hugging Face Daily Papers TIER_1 English(EN) ·

    Recovering Reliable Supervision via Critique-Guided On-Policy Distillation

    On-policy distillation (OPD) trains a student model on its own rollouts with token-level supervision from a teacher, assuming that the teacher can reach a correct answer from any prefix the student writes. We call this property the recoverability of the teacher and show it fails …

  25. Hugging Face Daily Papers TIER_1 English(EN) ·

    Simplifying and Improving On-Policy Distillation for Few-Step Diffusion

    Few-step distillation reduces the sampling cost of diffusion and flow models, but existing approaches often require auxiliary score models or specialized architectures. We show that simple on-policy velocity regression already provides a strong foundation for 4-NFE generation. To…

  26. arXiv cs.AI TIER_1 English(EN) · Dongyang Chen, Shudong Liu, Shubo Li, Duoxun Tang, Jinwei Liang, Zehuan Wu, Lewei Lu ·

    Branching On-Policy Distillation

    arXiv:2610.13481v1 Announce Type: new Abstract: On-policy distillation (OPD) provides teacher supervision on student-generated trajectories, but a single trajectory realizes only one continuation at each decision, limiting supervision coverage under a finite sampling budget. We p…

  27. arXiv cs.LG TIER_1 English(EN) · Zhentao Zhang, Yanzhao Zheng, Yanlin Liu, Yuanqiang Yu, Tianze Xu, Chao Ma, Jihuai Zhu, Lelun Wu, Baohua Dong, Hangcheng Zhu, Ruohui Huang, Gang Yu ·

    State-OPD: State-Grounded On-Policy Distillation for Multi-Turn Agents

    arXiv:2610.13238v1 Announce Type: new Abstract: On-Policy Distillation (OPD) has emerged as a powerful paradigm for transferring reasoning capabilities from frontier models to smaller models, while extending its success to multi-turn agentic environments remains challenging. We s…

  28. arXiv cs.LG TIER_1 English(EN) · Rong Wu, Daocheng Fu, Yu Yang, Xuemeng Yang, Siqi Li, Shu Zou, Yao Zhu, Botian Shi, Bo Zhang, Cong Zhang, Pinlong Cai, Jianbiao Mei ·

    Rethinking Relative Policy Differences Transfer of On-Policy Distillation

    arXiv:2610.13205v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a target policy on its own rollouts using token-level supervision from a stronger teacher, but typically relies on the teacher being substantially more capable than the target. Recent work relaxes…

  29. arXiv cs.CL TIER_1 English(EN) · Jingru Yang, Jinhang Wu, Minjing Shi, Shengfeng He, Xuyan Li, Man Tan, Biye Li ·

    CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation

    arXiv:2610.13257v1 Announce Type: cross Abstract: Cross-tokenizer on-policy distillation (OPD) requires aligning teacher and student sequences before their scores can be compared. Decoded-text alignment provides such correspondence at the chunk level, but leaves a distinct proble…

  30. arXiv cs.AI TIER_1 English(EN) · Anhao Zhao, Haoran Xin, Junlong Tong, Yingqi Fan, Xuan Lu, Ping Nie, Wenjie Li, Xiaoyu Shen ·

    DIAL-OPD: Learning More from Fewer Tokens in On-Policy Distillation

    arXiv:2610.11659v1 Announce Type: cross Abstract: On-policy distillation (OPD) supervises student-generated trajectories with token-level teacher signals. Its sampled-token variant avoids the cost of full-vocabulary probabilities. Yet we find that training on fewer tokens can out…

  31. arXiv cs.AI TIER_1 English(EN) · Zipeng Wang, Xinpeng Dong, Yuefan Wang, Pingchen Lu, Xian Wei, Kun Kuang, Fei Wu, Zhongxiang Dai, Min Zhang ·

    MetaOPD: Meta-Learned Token Weighting for On-Policy Distillation

    arXiv:2610.11989v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a student on its own generated responses using token-level teacher supervision. However, uniform weighting overlooks differences in token learning value, while existing weighting methods rely on p…

  32. arXiv cs.LG TIER_1 English(EN) · Mohammad Khoshnazar, Mohammad Dehghani Tezerjani, Deyuan Qu, Zhiyuan Gao, Yanxiang Zhan, Jeroen Schafer, Andrew Melnik, Qing Yang, Michael Beetz ·

    CAPABLE: Capability-Aware Policy Adaptation via Behavioral Latent Encoding

    arXiv:2610.11971v1 Announce Type: cross Abstract: Vision-language-action (VLA) policies assume the embodiment on which they were trained and can fail when a joint fault changes how commanded actions are physically executed. Existing fault-recovery methods often require task-speci…

  33. arXiv cs.LG TIER_1 English(EN) · Yilong Yang, Wenzhuo Shang, Yule Liu, Jiale Teng, Zhuo Ma ·

    Policy Alignment: New Signals for Membership Auditing in On-Policy Distillation

    arXiv:2610.11423v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a student model by aligning its policy with a teacher model on trajectories generated by the student model itself. Through this process, the student policy moves toward the teacher on the prompts …

  34. arXiv cs.CL TIER_1 English(EN) · Lei Zhao, Qichao Zhao, Bowen Zuo, Qishi Zhan ·

    Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals

    arXiv:2610.11247v1 Announce Type: cross Abstract: On-policy distillation (OPD) enables effective capability transfer between language models, yet the mechanisms underlying its failures are not fully understood. Across code generation and mathematical reasoning, OPD with larger-sc…

  35. arXiv cs.CL TIER_1 English(EN) · Yi Wang, Rui Qian, Yu Li, Haoyang Yao, Wenjie Wang ·

    Recurrent Self-Improvement: Dynamic Cross-Loop On-Policy Distillation for Looped Language Models

    arXiv:2610.10623v1 Announce Type: cross Abstract: Looped Language Models (LoopLMs) offer a parameter efficient approach to scaling reasoning by reusing shared parameters across recurrent computation steps. Despite their promise, effective post-training of LoopLMs remains challeng…

  36. arXiv cs.AI TIER_1 English(EN) · Megha Srivastava, Cedric Colas, Dorsa Sadigh, Jacob Andreas ·

    Policy Learning with a Language Bottleneck

    arXiv:2405.04118v4 Announce Type: replace-cross Abstract: Modern AI systems such as self-driving cars and game-playing agents can achieve superhuman performance, but often lack human-like generalization, interpretability, and inter-operability with human users. Inspired by the ri…

  37. Hugging Face Daily Papers TIER_1 English(EN) ·

    Branching On-Policy Distillation

    On-policy distillation (OPD) provides teacher supervision on student-generated trajectories, but a single trajectory realizes only one continuation at each decision, limiting supervision coverage under a finite sampling budget. We propose BranchOPD (Branching On-Policy Distillati…

  38. Hugging Face Daily Papers TIER_1 English(EN) ·

    Rethinking Relative Policy Differences Transfer of On-Policy Distillation

    On-policy distillation (OPD) trains a target policy on its own rollouts using token-level supervision from a stronger teacher, but typically relies on the teacher being substantially more capable than the target. Recent work relaxes this requirement by transferring the relative p…

  39. Hugging Face Daily Papers TIER_1 English(EN) ·

    Policy Alignment: New Signals for Membership Auditing in On-Policy Distillation

    On-policy distillation (OPD) trains a student model by aligning its policy with a teacher model on trajectories generated by the student model itself. Through this process, the student policy moves toward the teacher on the prompts used for distillation. However, these prompts ar…

  40. arXiv cs.LG TIER_1 English(EN) · Bohan Lin, Liyi Chen, Zhuoning Guo, Muyang Li, Qimeng Wang, Yan Gao, Yao Hu, Yudong Zhang ·

    GraphOPD: Graph-Augmented On-Policy Distillation for LLM Agents

    arXiv:2610.08959v1 Announce Type: new Abstract: On-policy distillation post-trains large language model agents by supplying dense, step-level guidance from a teacher policy when the reinforcement-learning reward is sparse and arrives only once per trajectory. Existing instantiati…

  41. arXiv cs.LG TIER_1 English(EN) · Jonas Schweisthal, Yuxin Wang, Athiya Deviyani, Stefan Feuerriegel, Dennis Frauen ·

    Efficient Best-of-N policy evaluation for inference-time alignment

    arXiv:2610.09250v1 Announce Type: new Abstract: Best-of-N (BoN) is a common inference-time alignment method that selects the highest-scoring response among N samples from a reference model. Evaluating BoN policies from logged data is challenging under sample-only access because s…

  42. arXiv cs.LG TIER_1 English(EN) · Laura Fuentes-Vicente, Mathieu Even, Ga\"elle Dormion, Antoine Chambaz, Uri Shalit, Julie Josse ·

    Set-Valued Policy Learning

    arXiv:2605.19830v2 Announce Type: replace Abstract: Conventional treatment policies map patient covariates to a single recommended intervention in order to maximize expected clinical outcomes. However, when multiple treatments yield statistically indistinguishable outcomes or whe…

  43. Hugging Face Daily Papers TIER_1 English(EN) ·

    Recurrent Self-Improvement: Dynamic Cross-Loop On-Policy Distillation for Looped Language Models

    Looped Language Models (LoopLMs) offer a parameter efficient approach to scaling reasoning by reusing shared parameters across recurrent computation steps. Despite their promise, effective post-training of LoopLMs remains challenging. Existing approaches either provide reward bas…

  44. arXiv cs.LG TIER_1 English(EN) · Ravenor Davion, Nick Rui ·

    Privileged Context as Drift in On-Policy Self-Distillation

    arXiv:2610.07842v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) trains a language model to match a copy of itself conditioned on privileged context. Existing work varies what privileged context contains and how it is produced while also changing models, data, a…

  45. arXiv cs.AI TIER_1 English(EN) · Hoang Phan, Minh Pham, Chau Pham, Chinmay Hegde, Trung Le, Qi Lei ·

    Rationale-Guided Policy Optimization: Learning to Reason with Adaptive Rationale Scaffolding

    arXiv:2610.07342v1 Announce Type: new Abstract: On-policy reinforcement learning has become a central paradigm for improving the reasoning abilities of large language models. However, its effectiveness is often limited by reward sparsity: when a model fails to discover correct tr…

  46. arXiv cs.AI TIER_1 English(EN) · Bingxi Hou, Guochao Jiang, Guofeng Quan, Weiqing Li, Wenfeng Feng, Guohua Liu, Yuewei Zhang ·

    Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliability

    arXiv:2610.08448v1 Announce Type: cross Abstract: On-Policy Distillation (OPD) trains a student on its own generations using teacher feedback. With different tokenizers, comparing teacher and student predictions requires alignment at both sequence and vocabulary levels. In this p…

  47. arXiv cs.AI TIER_1 English(EN) · Yifei Liu, Minghao Fang, Xinyu Gu, Chengkai Yao, Mengdi Liu, Tengfei Ma, Jiangbin Zheng, Chang Yu, Zhangyang Gao ·

    E$^2$-OPSD: Taming Entropy Overshoot in On-Policy Self-Distillation

    arXiv:2610.05048v2 Announce Type: replace-cross Abstract: On-policy self-distillation (OPSD) provides dense token-level supervision without a second model: one network acts as teacher with the reference solution and as student with only the problem. We identify a specific failure…

  48. arXiv cs.LG TIER_1 English(EN) · Jaehui Hwang, Dongyoon Han, Sangdoo Yun, Byeongho Heo ·

    On-Policy Distillation with Negative-Policy Rollouts

    arXiv:2610.07874v1 Announce Type: new Abstract: On-policy distillation (OPD) has been widely studied as a post-training method in which a student model obtains token-level supervision from a stronger teacher on its own rollouts. Recent studies have improved OPD through alternativ…

  49. Hugging Face Daily Papers TIER_1 English(EN) ·

    On-Policy Distillation with Negative-Policy Rollouts

    On-policy distillation (OPD) has been widely studied as a post-training method in which a student model obtains token-level supervision from a stronger teacher on its own rollouts. Recent studies have improved OPD through alternative distillation reward formulations and teacher c…

  50. Hugging Face Daily Papers TIER_1 English(EN) ·

    Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliability

    On-Policy Distillation (OPD) trains a student on its own generations using teacher feedback. With different tokenizers, comparing teacher and student predictions requires alignment at both sequence and vocabulary levels. In this paper, we examine whether expanding this alignment …

  51. Hugging Face Daily Papers TIER_1 English(EN) ·

    Off-Policy Merging Beats On-Policy Self-Distillation for Continual Learning

    A long-standing goal of AI is a model that can continually learn and improve itself. On post-trained models, supervised finetuning (SFT) on new data often causes poor generalization and catastrophic forgetting. As such, the conventional wisdom is that on-policy training is a prer…

  52. arXiv cs.CL TIER_1 English(EN) · Chenglei Shen, Haoyang Yao, Weijie Yu, Song Jin, Xiao Zhang, Jun Xu ·

    Learning from Repaired Reasoning: Root-Cause-Guided On-Policy Distillation

    arXiv:2610.03515v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) uses reference solutions as privileged hindsight to supervise student-generated reasoning trajectories. However, reference-based guidance may explain a correct solution without addressing why the s…

  53. arXiv cs.CL TIER_1 English(EN) · Rui Li, Liyang He, Zheng Zhang, Zhenya Huang, Linbo Zhu, Qi Liu ·

    Learning from Evolving Errors: Adaptive Iterative Repair for On-Policy Distillation

    arXiv:2610.02700v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) supplies dense token-level feedback on trajectories sampled from the student's own policy, a richer training signal than the outcome-level rewards of reinforcement learning. This feedback comes f…

  54. arXiv cs.AI TIER_1 English(EN) · Yuxiang Zhang, Ding Cao, Shuting Cui, Lei Wang, Weijieying Ren, Tianxiang Zhao ·

    Learning to Revise Reasoning with Segment-wise On-Policy Distillation

    arXiv:2610.02703v1 Announce Type: new Abstract: On-policy distillation (OPD) improves large language model reasoning by training students on their own rollouts with dense token-wise supervision from the teacher. However, token-wise OPD does not explicitly provide a coherent alter…

  55. arXiv cs.AI TIER_1 English(EN) · Han Cui, Jianhao Yan, Yun Luo, Hongbo Zhang, Zhizhang Fu, Yue Zhang ·

    Gains and Collapse in On-Policy Distillation:A Reinforcement Learning Perspective

    arXiv:2610.03185v1 Announce Type: new Abstract: On-policy distillation (OPD) has become an important approach to language model post-training. However, despite its performance gains, OPD can also collapse into excessively long and repetitive generation, and the mechanism underlyi…

  56. arXiv cs.AI TIER_1 English(EN) · Doseok Jang, Jon Ander Campos, Youran Qi ·

    Lexicographic Multi-Objective On-Policy Distillation

    arXiv:2610.02359v1 Announce Type: cross Abstract: Reinforcement learning from verifiable rewards (RLVR) usually optimizes answer correctness, yet useful language-model behavior also requires high-quality reasoning and concise responses. Existing multi-reward post-training methods…

  57. Hugging Face Daily Papers TIER_1 English(EN) ·

    Rationale-Guided Policy Optimization: Learning to Reason with Adaptive Rationale Scaffolding

    On-policy reinforcement learning has become a central paradigm for improving the reasoning abilities of large language models. However, its effectiveness is often limited by reward sparsity: when a model fails to discover correct trajectories for difficult problems, the optimizat…

  58. arXiv cs.AI TIER_1 English(EN) · Sophia Sirko-Galouchenko, Monika Wysoczanska, Andrei Bursuc, Nicolas Thome, Spyros Gidaris ·

    Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes

    arXiv:2610.02117v1 Announce Type: cross Abstract: On-policy self-distillation has recently emerged as an effective approach for improving language-model reasoning by supervising students with a frozen or EMA version of themselves that receives privileged information. Its applicat…

  59. arXiv cs.AI TIER_1 English(EN) · Feiyu Gavin Zhu, Qi Xu, Zhifei Deng, Zhigang Hua, Luke Simon, Jean Oh, Reid Simmons ·

    Iterative Policy Refinement through Semantic Rollout Analysis

    arXiv:2610.01652v1 Announce Type: cross Abstract: Structured policies improve efficiency, robustness, and interpretability in imitation learning by introducing task-specific inductive bias, but existing structure generation methods rely either on extensive human input or on stati…

  60. arXiv cs.LG TIER_1 English(EN) · Zixiang Ni, Zhuo Hu, Renjie Cao, Weijie Ren, Binqin Shi, Weijia Zhang, Shuheng Cao, Zhicheng Shi, Zhenhao Zhang, Haomin Wen, Zhiyuan Hu ·

    ReTaCo: Residual-Target Control for On-Policy Distillation

    arXiv:2609.39275v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a student on its own generated prefixes with token-level teacher feedback, but transmitting or storing the teacher's full-vocabulary distribution at every token is costly. Entropy-aware OPD (EOPD)…

  61. arXiv cs.LG TIER_1 English(EN) · Jingang Zhou, Feiyu Han, Han Zhu, Yuyi Zhou, Ruiyang Zhang, Jian Xu, Sirui Gao, Qingpei Guo, Xu-Yao Zhang ·

    No Task Vector Is an Island: A Comprehensive Study on the Composability of Task Vectors from On-Policy Distillation

    arXiv:2609.39405v1 Announce Type: new Abstract: Task vectors provide a simple mechanism for composing learned capabilities through model merging. However, the composability of task vectors produced by on-policy distillation (OPD) remains largely unexplored. OPD trains a student u…

  62. arXiv cs.AI TIER_1 English(EN) · Youngjun Jun, Kyumin Choi, Youngmin Kim, Seonghyun Jin, Sunwoo Park, Jangho Park, Jong Chul Ye ·

    DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies

    arXiv:2610.00317v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models increasingly rely on action experts that generate short action chunks under receding-horizon control. While chunk-level training is convenient across robot embodiments, it optimizes local action…

  63. Hugging Face Daily Papers TIER_1 English(EN) ·

    Gains and Collapse in On-Policy Distillation:A Reinforcement Learning Perspective

    On-policy distillation (OPD) has become an important approach to language model post-training. However, despite its performance gains, OPD can also collapse into excessively long and repetitive generation, and the mechanism underlying these divergent outcomes remains poorly under…

  64. arXiv cs.AI TIER_1 English(EN) · Qiwei Di, Xuheng Li, Kaixuan Ji, Chenggong Zhang, Heyang Zhao, Quanquan Gu ·

    Understanding Off- vs On-Policy Distillation: A Tale of Distinct Training Objectives

    arXiv:2609.38666v1 Announce Type: cross Abstract: On-policy distillation (OPD) learns from teacher feedback on student-generated responses and has shown promise in reducing forgetting relative to supervised fine-tuning (SFT). However, its benefits and fragility remain incompletel…

  65. arXiv cs.AI TIER_1 English(EN) · Fang Wu, Da Xing, Yanjie Huang, Junxi Wang, Ji Wang, Hejia Geng, Guancheng Wan, Bowen Zuo, Xiaomin Li, Shixiang Tang, Xinyu Xiang, Zehong Wang, Shiyi Du, Peng Xia, Shuangjia Zheng, Yining Hong, Li Erran Li, Jure Leskovec, Yejin Choi ·

    UniEvo-VL: An On-policy Self-Distillation Training Recipe for Multimodal Model Self-improvement

    arXiv:2609.38721v1 Announce Type: new Abstract: Modern multimodal models bring generation and understanding into a single unified system, which enables them to provide and learn from their own feedback. Motivated by this unified capacity, we introduce UniEvo-VL, a self-evolving f…

  66. arXiv cs.AI TIER_1 English(EN) · Weijie Ren, Yanwen Zhang, Hao Li, Zhuolin Qi, Hengyi Zhang, Naibo Wang ·

    OPSRD: On-Policy Self-Role Distillation

    arXiv:2609.39884v1 Announce Type: cross Abstract: Role prompting elicits specialized behavior from large language models through an expert identity, offering a lightweight way to guide reasoning on demanding tasks. However, evaluating or distilling complete role-prompted answers …

  67. arXiv cs.CL TIER_1 English(EN) · Xincheng Wei, Yifan Ding, Yoshua Li, Yuquan Lu, Ziheng Li, Yi Lu, Dongsheng Ma, Rongxiang Weng, Xunliang Cai ·

    Better Supervision Is Nearby: Neighborhood On-Policy Self-Distillation

    arXiv:2609.39687v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) trains mathematical reasoning models using a privileged teacher that sees a reference solution and supervises student-sampled prefixes. Standard OPSD uses one fixed parameter setting at every state…

  68. arXiv cs.CL TIER_1 English(EN) · Yang Li, Gongle Xue, Yuheng Yuan, Yijia Guo, Shizhe Zhang, Liwen Hu, Lei Ma ·

    Diagnosing On-Policy Self-Distillation for Reasoning Language Models

    arXiv:2609.39118v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) has attracted growing interest as a promising approach to improve the reasoning ability of language models. Without external rewards nor a separate stronger teacher, the self-teacher with privilege…

  69. arXiv cs.CL TIER_1 English(EN) · Di Huang, Hao Li, Yixin Chen, Fuhai Li ·

    When Clipping Reverses Correction: Failure Dynamics of Pointwise Forward-KL On-Policy Self-Distillation

    arXiv:2609.38995v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) trains a student on its own generated responses using feedback from the same model conditioned on privileged information. On mathematical reasoning, the original OPSD study finds that stylistic tok…

  70. arXiv cs.AI TIER_1 English(EN) · Hantao Yu, Xiaoxue Han, Udaya Ghai, Ferhat Erata, Joseph Lilien, Aman Goel, Ali Torkamani ·

    Instruct, Not Answer: Using Instruction Privileges in On-Policy Context Distillation

    arXiv:2609.32201v2 Announce Type: replace Abstract: On-Policy Context Distillation (OPCD) has recently emerged as a powerful technique for transferring context to student models and for self-improvement. In OPCD, the teacher is conditioned on privileged information, and the goal …

  71. arXiv cs.AI TIER_1 English(EN) · Jiaxuan Wang, Xuan Ouyang, Zhiyu Chen, Yulan Hu, Lan-Zhe Guo ·

    STEPS: Selective On-Policy Self-Distillation for Reasoning

    arXiv:2605.10194v2 Announce Type: replace Abstract: On-policy self-distillation (OPSD) uses a model as its own teacher under privileged context, providing token-level supervision on the model's own reasoning trajectories. However, persistent all-token guidance can degrade trainin…

  72. arXiv cs.AI TIER_1 English(EN) · Zhenxing Zhang, Jiayan Teng, Wenxu Wu, Zhuoyi Yang, Jiazheng Xu, Wendi Zheng, Jie Tang, Dan Guo, Meng Wang ·

    GFD-OPD: Guidance-Folded On-Policy Distillation of Diffusion Models Across Scales

    arXiv:2609.39692v1 Announce Type: cross Abstract: On-policy distillation (OPD) has demonstrated two important capabilities in language models: compressing large teachers into smaller students and merging expert models into a single model. Existing diffusion OPD, however, mostly f…

  73. Hugging Face Daily Papers TIER_1 English(EN) ·

    Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes

    On-policy self-distillation has recently emerged as an effective approach for improving language-model reasoning by supervising students with a frozen or EMA version of themselves that receives privileged information. Its application to multimodal large language models (MLLMs), h…

  74. arXiv cs.LG TIER_1 English(EN) · Yuxiao Yang, Shangzhe Li, Tianrun Yu, Kaixiang Zhao, Taylor W. Killian, Weitong Zhang ·

    REVO: Rollout-Efficient Off-Policy Distillation via Variance-Guided Reuse

    arXiv:2609.37500v1 Announce Type: new Abstract: On-policy distillation (OPD) trains language models using dense token-level teacher supervision on student-generated trajectories. However, its reliance on frequently refreshed student rollouts often incurs substantial generation co…

  75. arXiv cs.AI TIER_1 English(EN) · Hongyang Li, Yiming Zhu, Xiao Li, Caesar Wu, Said Mammar, Pascal Bouvry ·

    Solving Without Stopping: On-Policy Distillation at Small Scale

    arXiv:2609.37326v1 Announce Type: new Abstract: On-policy distillation, where a student learns from a stronger teacher's feedback on its own outputs, is a common way to pass reasoning to smaller models. We analyze what it transfers at small scale, distilling Qwen3-8B into Qwen3 4…

  76. arXiv cs.AI TIER_1 English(EN) · Jiaxuan Wang, Jiafei Lyu, Yuchen Cai, Siye Wu, Pengyuan Wang, Jiashun Liu, Xiang Cheng, Kai Yang, Yangkun Chen, Saiyong Yang, Lan-Zhe Guo ·

    Beyond Prompt Count: How Data Shapes Transfer in On-Policy Distillation

    arXiv:2609.37377v1 Announce Type: new Abstract: On-policy distillation (OPD) trains students using teacher feedback on their own sampled responses, yet how prompt choice shapes transfer across teacher-student pairs remains poorly understood. We systematically study prompt quantit…

  77. arXiv cs.AI TIER_1 English(EN) · Shutong Wu, Xiwen Chen, Brendan Rappazzo, Daiheng Zhang, Anderson Schneider, Yuriy Nevmyvaka, Jiawei Zhang ·

    Interactive-Policy Distillation with Bidirectional Propose-and-Verify

    arXiv:2609.36546v1 Announce Type: cross Abstract: On-policy distillation (OPD) trains a student model on its self-generated trajectories with dense token-level teacher feedback. However, naive OPD may suffer from teacher unanchoring, where the student's reasoning trajectory drift…

  78. arXiv cs.AI TIER_1 English(EN) · Youxu Shi, Yifan Sun, Dacheng Yin, Haomiao Tang, Guangting Wang, Fengyun Rao, Jing Lyu, Dong Liu ·

    Interpolated Policy Distillation: A Controllable Continuum Between Off-Policy and On-Policy Distillation

    arXiv:2609.37170v1 Announce Type: cross Abstract: Off-policy and on-policy distillation have traditionally been formulated as separate paradigms, each favoring a different property of distillation trajectories. Teacher-generated (off-policy) traces are typically high-quality but …

  79. arXiv cs.AI TIER_1 English(EN) · Zizhuo Lin, Quanling Liu, Yi Yang, Yawei Luo ·

    What Shared Prefixes Hide: Trajectory Dropout for On-Policy Distillation

    arXiv:2609.33455v2 Announce Type: replace Abstract: On-policy distillation (OPD) trains a student model on its own trajectories using dense token-level feedback from a stronger teacher model. Since each update is conditioned on the reasoning prefix already generated by the studen…

  80. arXiv cs.AI TIER_1 English(EN) · Taesung Kwon, Jangho Park, Sunwoo Park, Youngmin Kim, Seonghyun Jin, Youngjun Jun, Kyumin Choi, Jong Chul Ye ·

    Don't Throw Away the Tail: Action Upcycling for Policy Acceleration

    arXiv:2609.34911v2 Announce Type: replace-cross Abstract: Modern robot policies predict a chunk of future actions from a single observation, execute only a prefix, and discard the rest before replanning. Choosing the length of this prefix, the execution horizon, poses a trade-off…

  81. arXiv cs.CL TIER_1 English(EN) · Wanqi Ren, Jianxiang Wang, Danxuan Liu, Linyi Ding, Yuan Zhang ·

    Learning from Think-Mode Advantage via On-Policy Distillation

    arXiv:2609.37044v1 Announce Type: new Abstract: Explicit intermediate reasoning gives large language models (LLMs) a stronger problem-solving mode. We study learning from this think-mode advantage via on-policy distillation (OPD). OPD preserves student-generated trajectories and …

  82. arXiv cs.CL TIER_1 English(EN) · Zubin Zheng, Jiahao Wu, Shaofeng Zhang, Zhirui Zhang, Yew-Soon Ong, Shengcai Liu ·

    Act First, Reason Later: Accelerating On-Policy Distillation for Multi-Turn Agents via Reference-Conditioned Inverse Dynamics

    arXiv:2609.36608v1 Announce Type: cross Abstract: On-policy distillation (OPD) trains multi-turn language agents with dense teacher supervision on student-generated responses. However, standard think-then-act rollouts require lengthy reasoning before each short action, delaying e…

  83. arXiv cs.CL TIER_1 English(EN) · Safaeid Hossain Arib, Rabeya Akter, Ismam Nur Swapnil, Md. Faiyaz Abdullah Sayeedi, Tasnim Mohiuddin, Md Mofijul Islam ·

    Teach Yourself Where to Look: On-Policy Attention Self-Distillation for Reasoning

    arXiv:2609.33200v2 Announce Type: replace-cross Abstract: On-policy self-distillation trains reasoning models on their own trajectories using dense token distribution guidance from a privileged teacher with access to a verified solution. This supervision transfers what the teache…

  84. arXiv cs.LG TIER_1 English(EN) · Hao Li, MeiJia Chen, Weijie Ren, Donghan Li, Zijun Tian, Jingchun Huang, Naibo Wang ·

    The Teacher Is a Direction, Not a Destination: Extrapolating RL-Induced Representation Residuals in On-Policy Distillation

    arXiv:2609.36484v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a student to match the teacher's next-token distributions on the student's own trajectories and has yielded substantial empirical gains. Generalized variants allow the student to surpass the teach…

  85. arXiv cs.LG TIER_1 English(EN) · Mingfeng Lin, Chengfei Cai, Lin Xu, Chengqian Ma, Yuxiang Wei, Liang Han ·

    PE-OPSD: Internalizing Prompt Enhancement into Flow-matching Models via On-Policy Self-Distillation

    arXiv:2609.36638v1 Announce Type: new Abstract: Text-to-image users often provide concise and underspecified prompts, whereas generative models benefit from detailed textual conditions for reliable instruction following. Existing systems bridge this gap with Prompt Enhancers (PEs…

  86. arXiv cs.LG TIER_1 English(EN) · Muyang Li, Jie Yang, Zhengyu Fang, Junchao Zhu, Zhengkun Xiao, Ruining Deng, Zhe Jiang, Shigang Chen ·

    PR-OPD: Privileged Representation On-policy Self-Distillation for Agentic Reinforcement Learning

    arXiv:2609.36642v1 Announce Type: new Abstract: Language-model agents are usually trained by reinforcement learning from one reward per episode, and privileged self-distillation enriches it by letting the same policy, given a skill, teach its skill-free self through token probabi…

  87. arXiv cs.LG TIER_1 English(EN) · Xiaohan Yi, Wen Luo, Yani Huang, Junfeng Zhan, Asher Qin, Peilin Zhao, Xi Xiao ·

    Graph-Conditioned On-Policy Agent Distillation from Off-the-Shelf Teachers

    arXiv:2609.37522v1 Announce Type: new Abstract: On-policy distillation (OPD) trains compact language agents with teacher feedback on student-generated trajectories. In multi-turn tasks, compounding errors can move students beyond the teacher's effective supervision. We introduce …

  88. arXiv cs.LG TIER_1 English(EN) · Zhiqi Li, Bo Zhu ·

    FlowMap-OPD: Rollout--Kernel Separation for On-Policy Distillation of Few-Step Flow-Map Generators

    arXiv:2609.37851v1 Announce Type: cross Abstract: Few-step flow-map generators, including MeanFlow and consistency models, enable efficient sampling through long-range transport, yet their on-policy distillation remains underexplored. We introduce FlowMap-OPD, an on-policy distil…

  89. Hugging Face Daily Papers TIER_1 English(EN) ·

    UniEvo-VL: An On-policy Self-Distillation Training Recipe for Multimodal Model Self-improvement

    Modern multimodal models bring generation and understanding into a single unified system, which enables them to provide and learn from their own feedback. Motivated by this unified capacity, we introduce UniEvo-VL, a self-evolving framework for multimodal models to learn from thi…

  90. Hugging Face Daily Papers TIER_1 English(EN) ·

    Better Supervision Is Nearby: Neighborhood On-Policy Self-Distillation

    On-policy self-distillation (OPSD) trains mathematical reasoning models using a privileged teacher that sees a reference solution and supervises student-sampled prefixes. Standard OPSD uses one fixed parameter setting at every state, but nearby settings may offer additional super…

  91. Hugging Face Daily Papers TIER_1 English(EN) ·

    UniEvo-VL: An On-policy Self-Distillation Training Recipe for Multimodal Model Self-improvement

    Modern multimodal models bring generation and understanding into a single unified system, which enables them to provide and learn from their own feedback. Motivated by this unified capacity, we introduce UniEvo-VL, a self-evolving framework for multimodal models to learn from thi…

  92. Hugging Face Daily Papers TIER_1 English(EN) ·

    FlowMap-OPD: Rollout--Kernel Separation for On-Policy Distillation of Few-Step Flow-Map Generators

    Few-step flow-map generators, including MeanFlow and consistency models, enable efficient sampling through long-range transport, yet their on-policy distillation remains underexplored. We introduce FlowMap-OPD, an on-policy distillation framework that separates student-state acqu…

  93. arXiv cs.AI TIER_1 English(EN) · Taeckyung Lee, Rinat Amankos, Jeonghye Kim, Hyungjun Yoon, Woogyeol Jin, Sung-Ju Lee ·

    TISD: On-Policy Self-Distillation with Trajectory Intervention

    arXiv:2609.30878v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) provides dense teacher targets, but evaluates them only along student-sampled rollouts. When the privileged teacher favors an alternative action at a visited prefix, OPSD can provide a target for t…

  94. Hugging Face Daily Papers TIER_1 English(EN) ·

    OPMD: On-Policy Mechanism Distillation Beyond Output Supervision

    On-Policy Mechanism Distillation improves language model distillation by transferring teacher intermediate computations compatible with the student's architecture, yielding stronger reasoning and internalization.

  95. Hugging Face Daily Papers TIER_1 English(EN) ·

    PR-OPD: Privileged Representation On-policy Self-Distillation for Agentic Reinforcement Learning

    Language-model agents are usually trained by reinforcement learning from one reward per episode, and privileged self-distillation enriches it by letting the same policy, given a skill, teach its skill-free self through token probabilities. However, we identify two phenomena that …

  96. Hugging Face Daily Papers TIER_1 English(EN) ·

    The Teacher Is a Direction, Not a Destination: Extrapolating RL-Induced Representation Residuals in On-Policy Distillation

    On-policy distillation (OPD) trains a student to match the teacher's next-token distributions on the student's own trajectories and has yielded substantial empirical gains. Generalized variants allow the student to surpass the teacher by extrapolating an implicit reward in output…

  97. Hugging Face Daily Papers TIER_1 English(EN) ·

    Act First, Reason Later: Accelerating On-Policy Distillation for Multi-Turn Agents via Reference-Conditioned Inverse Dynamics

    On-policy distillation (OPD) trains multi-turn language agents with dense teacher supervision on student-generated responses. However, standard think-then-act rollouts require lengthy reasoning before each short action, delaying environment transitions and experience collection. …

  98. Hugging Face Daily Papers TIER_1 English(EN) ·

    Beyond Prompt Count: How Data Shapes Transfer in On-Policy Distillation

    On-policy distillation (OPD) trains students using teacher feedback on their own sampled responses, yet how prompt choice shapes transfer across teacher-student pairs remains poorly understood. We systematically study prompt quantity, source, and selection across RL- and SFT-cont…

  99. Hugging Face Daily Papers TIER_1 English(EN) ·

    On-Policy or Off-Policy Learning? A Systematic Study of Distillation Dynamics

    On-policy learning has been argued to reduce catastrophic forgetting, produce sparser parameter updates, and improve generalisation. However, existing comparisons between supervised fine-tuning and reinforcement learning vary many factors simultaneously, making the contribution o…

  100. Hugging Face Daily Papers TIER_1 English(EN) ·

    Understanding On-Policy Distillation: A Mechanistic Interpretability Perspective via Sparse Crosscoders

    On-policy distillation (OPD) is a widely adopted post-training technique for LLM reasoning. It is commonly believed to transfer knowledge from a stronger teacher, yet what OPD actually distills into the student's internal representations remains unclear. We study this question wi…

  101. Hugging Face Daily Papers TIER_1 English(EN) ·

    Scaling Properties of Same-Family On-Policy Distillation

    *Reinforcement learning (RL)* can induce substantial reasoning capabilities in large language models (LLMs), but how much of this capability transfers across model scales, and how quickly, remains unclear. We study the scaling properties of *on-policy distillation (OPD)* across *…

  102. arXiv stat.ML TIER_1 English(EN) · Linjian Meng, Siyuan Gan, YuHan Li, Xiran Wang, Ziyang Ding, Ditang Gou, Yiming Wu, Zhen Zhao ·

    Unbiased Top-$k$ Estimation for On-Policy Distillation

    arXiv:2609.34447v2 Announce Type: replace-cross Abstract: On-policy distillation (OPD) is becoming an important component of large language model (LLM) post-training for transferring the reasoning capability of a strong teacher LLM to a weaker student LLM. OPD trains the student …

  103. arXiv cs.CV TIER_1 English(EN) · Junxian Li, Ruixuan Yang, Tianao Zhang, Tiange Xu, Weisheng Dong, Yulun Zhang ·

    Fewer Tokens, More Self-Teaching: On-Policy Self-Distillation for Extreme Visual Token Reduction

    arXiv:2609.32353v2 Announce Type: replace Abstract: Visual token reduction is an effective way to accelerate multimodal large language models (MLLMs), but performance deteriorates rapidly under extremely low token budgets. Existing work has explored both visual-token selection an…

  104. arXiv cs.CV TIER_1 English(EN) · Amirhossein Tighkhorshid, Zahra Dehghanian, Hamid R. Rabiee ·

    ReDiF: Resource-Efficient Few-Step Diffusion Distillation via Reinforcement Learning

    arXiv:2512.22802v2 Announce Type: replace-cross Abstract: Step distillation accelerates diffusion sampling by training a few-step student to imitate a many-step teacher, but distillation itself remains expensive. Typically, this requires thousands of GPU-hours and a large pre-gen…

  105. arXiv cs.CV TIER_1 English(EN) · Pengyang Ling, Jiazi Bu, Yujie Zhou, Yibin Wang, Zeqiang Lai, Xiaoxiao Ma, Yi Jin, Huaian Chen, Yuhang Zang ·

    CapField-OPD: Learning Continuous Capability Fields via Joint-Anchored Multi-Teacher On-Policy Distillation for Flow Models

    arXiv:2609.34658v2 Announce Type: replace Abstract: Reward-specialized post-training produces strong experts for flow-based generative models, while multi-teacher on-policy distillation (OPD) consolidates their capabilities into a single student. Existing methods, however, route …

  106. Mastodon — mastodon.social TIER_1 English(EN) · opensourceaitech ·

    📄 Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation hit 663 upvotes on Hugging Face – a signal that efficient policy transfer

    📄 Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation hit 663 upvotes on Hugging Face – a signal that efficient policy transfer via distillation is a hot topic right now. Open the paper page to see why. https:// huggingface.co/papers/2610.216 01 #…

  107. Mastodon — mastodon.social TIER_1 English(EN) · opensourceaitech ·

    📄 'Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliability' argues the real bottleneck isn't coverage but supervis

    📄 'Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliability' argues the real bottleneck isn't coverage but supervision reliability. 153 upvotes on Hugging Face. https:// huggingface.co/papers/2610.084 48 # AI # MachineLearning # Resear…