LLM post-training recipes evolve with new distillation techniques

A review of post-training recipes for large language models highlights significant evolution in the past year. Historically, models followed a pipeline of Supervised Fine-Tuning (SFT), reward modeling, and Reinforcement Learning (RL). However, recent advancements in 2024 and projections for 2025-2026 indicate a shift towards more complex, multi-stage processes. These include Direct Preference Optimization (DPO) and Reinforcement Learning from AI Feedback (RLAIF), with a notable emergence of Multi-teacher On-Policy Distillation (MOPD) for frontier models. AI

IMPACT Understanding evolving LLM training methodologies is crucial for optimizing model performance and efficiency.

RANK_REASON This cluster is a review and discussion of existing and projected LLM training recipes, rather than a new release or research paper.

Read on Interconnects (Nathan Lambert) →

• Ai2
• DeepSeek R1
• DeepSeek V4
• Finbarr Timbers
• GLM 5
• InstructGPT
• Interconnects
• Kimi K2.6
• Llama 3
• MiMo Flash
• Nemotron 3 Ultra
• Olmo
• OLMo 3
• Tülu 3

• model release
• paper

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →