baseVultra: New compact nonlinear recurrence cell outperforms LSTM/GRU in specific settings

Researchers have introduced baseVultra, a novel compact nonlinear recurrent neural network cell designed for streaming sequence modeling. This new architecture reallocates computational resources from gate transformations to candidate generation, featuring a concentrated dense recurrent budget on a single matrix and replacing recurrent gate matrices with simpler diagonal interactions. In experiments on TinyShakespeare and text8 datasets, baseVultra demonstrated competitive or superior performance compared to LSTM and GRU models, particularly in compact settings with constrained hidden-state capacity. AI

IMPACT Introduces a novel recurrent cell architecture that may offer efficiency gains in specific sequence modeling tasks.

RANK_REASON Academic paper detailing a new model architecture. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• baseVultra
• gated recurrent unit
• Hugging Face
• long short-term memory
• text8
• TinyShakespeare

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →