Beyond Log Likelihood: Probability-Based Objectives for Supervised Fine-Tuning across the Model Capability Continuum
Researchers have explored alternative objectives for supervised fine-tuning (SFT) of large language models, moving beyond the standard negative log likelihood (NLL). Their study, involving extensive experiments across various models and benchmarks, reveals that different objectives perform better depending on the model's capability. Objectives that downweight low-probability tokens are more effective for highly capable models, while NLL excels with less capable models. AI
IMPACT New fine-tuning objectives could improve LLM generalization and performance on specific tasks.