New Mamba model variant enhances memory retention and bilinear computation

Researchers have introduced Bilinear Input Modulation (BIM) to enhance Selective State Space Models (SSMs), specifically Mamba, by incorporating state-input products. This augmentation allows for improved memory retention and multiplicative computation, addressing limitations in Mamba's diagonal state transitions. The proposed methods, including Coupled Bilinear Input Modulation (seq-BIM) and Parallel Bilinear Input Modulation (p-BIM), demonstrate significant performance gains on tasks requiring memory and bilinear processing, outperforming simpler gating mechanisms. AI

影响 Introduces a new method to improve memory retention and computational capacity in state-space models.

排序理由 Academic paper introducing a novel computational technique for existing models.

在 arXiv cs.LG 阅读 →

• arXiv
• Hiroki Fujii
• Koopman
• Mamba

• 论文
• 其他

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →