Linear RNNs
PulseAugur coverage of Linear RNNs — every cluster mentioning Linear RNNs across labs, papers, and developer communities, ranked by signal.
-
Sparse Delta Memory boosts linear RNNs for better long-context recall
Researchers have introduced Sparse Delta Memory (SDM), a novel architecture designed to enhance the long-context recall capabilities of linear RNNs. By employing a sparse addressing scheme, SDM significantly increases t…
-
Linear RNNs show promise in state-tracking tasks by converting them to code
Researchers have developed a method to convert permutation composition tasks into code, enabling linear RNNs to excel where Transformers have previously struggled. This approach addresses the incompatibility of state-tr…
-
Paper links linear RNNs to circuits, explaining parallelization
Researchers have explored linear RNNs (LRNNs) as language models, noting their expressivity and parallelizability. A new paper connects LRNNs to arithmetic circuits, explaining their parallel nature by showing they are …