Selective-Update RNNs match Transformer accuracy with greater efficiency

Researchers have developed a new type of Recurrent Neural Network (RNN) called Selective-Update RNNs (suRNNs) that can efficiently handle long-range sequence modeling. Unlike traditional RNNs that update at every time step, suRNNs use a binary switch at the neuron level to learn when to preserve memory, decoupling updates from sequence length. This allows them to maintain exact past information during redundant intervals, enabling better gradient flow and Transformer-level accuracy with greater efficiency on benchmarks like the Long Range Arena. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Offers a more efficient alternative to Transformers for long-sequence data, potentially improving performance in areas like audio and video processing.

RANK_REASON This is a research paper detailing a novel model architecture.

Read on arXiv cs.LG →

• RNN
• suRNNs
• Long Range Arena
• Transformers
• Bojian Yin

• paper
• model release