prose

New TFGN architecture prevents LLM knowledge loss during continual training

Researchers have developed TFGN, a novel architectural overlay for transformer language models designed to enable continual pre-training without catastrophic forgetting. This method allows models to learn from new data …