New QMAC method optimizes LLM training data selection

Researchers have introduced QMAC, a novel method for selecting high-quality data for training large language and vision-language models. QMAC addresses the computational and storage costs associated with massive datasets by formulating data selection as a joint optimization of distribution matching and quality filtering. This approach ensures the selected subset's distribution aligns with the full dataset in the embedding space, while also filtering out noisy or low-quality samples. Experiments across various data types and models demonstrate that QMAC outperforms existing methods and even surpasses full data training in certain scenarios. AI

IMPACT Optimizes training data selection, potentially reducing costs and improving model performance.

RANK_REASON This is a research paper detailing a new method for data selection in LLM training. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• alphaXiv
• arXiv
• CatalyzeX
• Connected Papers
• CORE Recommender
• DagsHub
• Gotit.pub
• Hugging Face
• IArxiv Recommender
• Litmaps
• QMAC
• ScienceCast
• scite Smart Citations

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →