RAFT: Data Refinement and Adaptive Distillation for Domain Fine-Tuning with Alleviated Forgetting
Researchers have introduced RAFT, a novel two-stage framework designed to improve domain-specific fine-tuning of language models while mitigating performance degradation on general tasks. RAFT addresses issues like supervision-compatibility and trajectory-preservation by first refining domain-specific data through self-conditioned rewriting and semantic filtering. It then employs an adaptive distillation process that uses the original model's behavior on generated trajectories as soft targets, conditioned on the refined answers. AI
IMPACT This research offers a method to improve domain-specific AI models without sacrificing general capabilities, potentially leading to more robust and versatile AI applications.