masked distillation

Masked distillation trains LLMs to internalize reasoning steps

Researchers have developed a new method called masked distillation to train language models to internalize the computational steps of reasoning, thereby reducing latency and cost. This technique trains a student model t…