New Von-Neumann State-Space Transformer boosts neural decoding efficiency

Researchers have introduced the Von-Neumann State-Space Transformer (VN-SST), a novel architecture designed for more data-efficient neural decoding. Inspired by von Neumann's computing principles, this model uses a low-rank instruction bank within its feed-forward block, allowing a shared base operator to synthesize token-specific weight matrices. This approach mirrors how low-dimensional dynamics might guide cortical computation. In tests on motor-cortex neural-decoding benchmarks, VN-SST significantly outperformed standard Transformers in data efficiency, particularly on sparse datasets, and demonstrated improved parameter efficiency on text-based language modeling tasks. AI

IMPACT This novel architecture could lead to more efficient AI models, particularly in applications requiring decoding from limited data.

RANK_REASON The cluster contains an academic paper detailing a new model architecture. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Morteza Sarafyazd
• Transformer
• Von-Neumann State-Space Transformer

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →