New Data-DPO method optimizes LLM post-training data selection

Researchers have introduced Data-DPO, a novel method for selecting effective data samples during LLM post-training. This approach focuses on the compatibility between candidate data and the target model's capabilities by probing local training feedback. Data-DPO transforms activation differences into pairwise preferences, trains a lightweight reward model, and combines these preferences with external quality scores and diversity metrics for final subset construction. Experiments on Vision-Flan and LLaVA-CoT demonstrated that Data-DPO consistently outperforms existing data selection baselines and even surpasses full data training performance across various budgets. AI

IMPACT Optimizes LLM training efficiency and performance by improving data selection.

RANK_REASON The cluster contains a research paper detailing a new method for LLM post-training. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Data-DPO
• LLaVA-CoT
• Vision-FLAN

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →