Two new research papers explore advancements in black-box on-policy distillation, a technique used to train smaller student language models from larger, API-accessible teacher models. The first paper introduces "persistent-negative adversarial distillation," which addresses the challenge of a moving target in adversarial training by using a pool of historical comparisons. The second paper proposes "Groupwise Reward Geometry Conditioning" (GRGC) to improve the construction of advantages by calibrating the reward geometry during both critic training and policy optimization. Both methods aim to enhance the effectiveness and stability of distillation processes, showing consistent performance improvements across various benchmarks and model families. AI
IMPACT These advancements could lead to more efficient and stable training of smaller language models from larger ones, improving accessibility and performance.
RANK_REASON Two academic papers published on arXiv detailing new methods for on-policy distillation.
- Adversarial Distillation
- arXiv
- Grpo
- Hugging Face
- On-Policy Distillation
- persistent-negative adversarial distillation
- Persistent Negatives for Adversarial Black-Box On-Policy Distillation
- GAD
- GRGC
- Groupwise Reward Geometry Conditioning
- SeqKD
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →