New LESSER method speeds up LLM post-training data selection

Researchers have developed a new method called LESSER for selecting post-training data for large language models. This technique utilizes output-layer gradients, which are significantly cheaper to compute than full-parameter gradients, to approximate the data selection process. LESSER reduces the computational cost by up to 9.7x for supervised fine-tuning and 3x for reinforcement learning benchmarks while maintaining performance on downstream tasks. The method has been shown to select batches with aligned gradients, even when individual sample rankings differ between output-layer and full gradients. AI

IMPACT Reduces computational costs for LLM fine-tuning, potentially accelerating research and development.

RANK_REASON The cluster describes a new method presented in an academic paper on arXiv. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• full-parameter gradients
• Hugging Face
• large language models
• LESSER
• output-layer gradients
• reinforcement learning
• supervised fine-tuning

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →