Natural Ungrokking

AI models forget learned rules mid-training, new research finds

Researchers have identified a phenomenon called "natural ungrokking" in language models, where learned rules can disappear mid-training without any change in the loss curve. This forgetting is directly correlated with t…