SLAP: Stratified Loss-based Pruning for On-Policy Data-Efficient Instruction Tuning
Researchers have introduced SLAP, a new framework designed to make instruction tuning of large language models more efficient. SLAP focuses on selecting batches of data that are most learnable and diverse, rather than individual data points. This approach allows models to achieve comparable or even better performance using 20-40% less training data, significantly reducing computational costs. AI
IMPACT Reduces training data and computational costs for LLM fine-tuning, potentially accelerating model development.