Autolearn framework enables language models to learn from documents without supervision

Researchers have introduced Autolearn, a novel framework designed to enable language models to learn from documents without external supervision. The system identifies passages that generate unusually high per-token loss, verifies them through self-generated question-and-answer chains, and then updates the model's parameters. A key metric, the perturbation gap, demonstrates that this Q&A format training significantly reduces memorization compared to standard fine-tuning, leading to a substantial increase in the acquisition of novel factual knowledge. AI

影响 Introduces a method for unsupervised learning in LLMs, potentially reducing the need for labeled data and improving knowledge acquisition.

排序理由 This is a research paper detailing a new framework for unsupervised learning in language models. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

• arXiv
• Autolearn
• Phi-4
• Qwen

• 论文
• 模型发布

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →