New FINCH method cuts LLM forgetting by 93%

Researchers have developed a new method called FINCH to address catastrophic forgetting during the fine-tuning of large language models. FINCH employs a loss-adaptive learning rate schedule that decreases the learning rate for high-loss batches and increases it as the model converges. This approach effectively reduces forgetting by an average of 93% across various benchmarks while maintaining task performance. FINCH also shows improvements in preserving model calibration and confidence. AI

影响 FINCH significantly reduces catastrophic forgetting in LLMs, potentially enabling more effective and stable fine-tuning for specialized tasks.

排序理由 The cluster contains an academic paper detailing a new method for fine-tuning language models. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

• FINCH
• large language models
• Qwen3-4B

• 论文
• 模型发布

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →