Pyrecall

New method restores long-context recall lost during LLM fine-tuning

Researchers have identified that Chain-of-Thought (CoT) fine-tuning, while improving reasoning, significantly degrades long-context recall in hybrid linear-attention models. This issue, termed "attention amnesia," cause…