New DRIFT method refines LLM training data for improved performance

Researchers have developed DRIFT, a novel method for refining instruction data to improve the performance ceiling of large language models. Unlike existing data curation techniques that focus on subset selection, DRIFT aims to enhance the data distribution itself. It utilizes on-policy influence functions, leveraging the model's own rollouts as validation targets to address limitations like proximity gaps and gradient norm bias found in standard influence function formulations. Experiments with 7B-parameter models demonstrate that DRIFT effectively raises performance on instruction and reasoning tasks, outperforming current data curation baselines. AI

IMPACT This research could lead to more capable LLMs by improving the efficiency and effectiveness of training data curation.

RANK_REASON The cluster contains a research paper detailing a new method for improving LLM training data. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• 7B-parameter
• arXiv
• DRIFT
• Hugging Face
• Influence Functions
• large-language models
• Supervised Fine-Tuning

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →