New CIG-SFT method enhances LLM generalization in post-training

Researchers have introduced CIG-SFT, a novel framework for supervised fine-tuning (SFT) of large language models. This method uses an information-theoretic approach to estimate the conditional information carried by each demonstrated token, quantifying how much the task itself reduces uncertainty about a given decision. By converting this information into a KL-regularized projection of the current policy, CIG-SFT derives a weighted SFT objective that improves generalization on mathematical reasoning and code-generation tasks without requiring additional data or online rollouts. AI

IMPACT This method could improve the generalization capabilities of LLMs on complex tasks like reasoning and code generation.

RANK_REASON The cluster contains an academic paper detailing a new method for fine-tuning large language models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• CIG-SFT
• KL-regularized projection
• large-language models
• supervised fine-tuning

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →