Edit Knowledge, Not Just Facts via Multi-Step Reasoning over Background Stories
Researchers have developed a new training strategy for large language models that focuses on updating knowledge through multi-step reasoning rather than just memorizing facts. This approach introduces new information as a coherent background story, encouraging models to use self-generated, multi-hop questions that require combining new and existing knowledge. The strategy also employs knowledge distillation to help a student model learn the teacher's reasoning process without direct access to the new information, leading to improved performance on complex reasoning tasks. AI
IMPACT This research could lead to AI systems that are more adaptable and capable of complex reasoning by effectively integrating new information.