PulseAugur
EN
LIVE 12:06:22

AI models forget learned rules mid-training, research finds

By PulseAugur Editorial · [3 sources] ·

A new research paper introduces the concept of "natural ungrokking," describing how language models can learn a rule during pretraining, only to forget it later without any change in the loss curve. The study found that the survival of learned rules is determined by how frequently they appear in the training data, rather than the data-to-parameter ratio. Interestingly, the research also demonstrated that while it's possible to intentionally destroy a learned rule, restoring it proved to be an asymmetric process, with no recovery observed even with significantly increased supportive data. AI

IMPACT This research highlights a potential vulnerability in LLM training, suggesting that learned behaviors can be lost without clear indicators, impacting model reliability and interpretability.

RANK_REASON The cluster consists of a research paper detailing a phenomenon in language model training.

Read on arXiv cs.AI →

AI-generated summary · Google Gemini · from 3 sources. How we write summaries →

AI models forget learned rules mid-training, research finds

COVERAGE [3]

  1. arXiv cs.CL TIER_1 English(EN) · Juliana Li, Diya Sreedhar ·

    Natural Ungrokking: Asymmetric Control of Which Rules Survive Pretraining

    arXiv:2606.26050v1 Announce Type: cross Abstract: Midway through an ordinary pretraining run, a small language model learns the pronoun-gender rule: cued with a girl's name ("Sue cried because"), it resolves the next pronoun to she, generalizing to held-out probes (0.94 by step 9…

  2. arXiv cs.AI TIER_1 English(EN) · Diya Sreedhar ·

    Natural Ungrokking: Asymmetric Control of Which Rules Survive Pretraining

    Midway through an ordinary pretraining run, a small language model learns the pronoun-gender rule: cued with a girl's name ("Sue cried because"), it resolves the next pronoun to she, generalizing to held-out probes (0.94 by step 925). By step 3,500 the same model scores near zero…

  3. Hugging Face Daily Papers TIER_1 English(EN) ·

    Natural Ungrokking: Asymmetric Control of Which Rules Survive Pretraining

    Midway through an ordinary pretraining run, a small language model learns the pronoun-gender rule: cued with a girl's name ("Sue cried because"), it resolves the next pronoun to she, generalizing to held-out probes (0.94 by step 925). By step 3,500 the same model scores near zero…

RELATED ENTITIES

RELATED TOPICS