Lola

New LoLA method boosts transformer memory and recall

Researchers have developed LoLA, a novel augmentation for linear attention mechanisms that significantly enhances associative recall and memory capacity in transformer models. LoLA distributes past key-value pairs acros…