Phasor Memory Networks tackle gradient instability in explicit memory models

Researchers have introduced Phasor Memory Networks (PMNet), a novel architecture designed to overcome the gradient instability issues that have historically plagued explicit memory models. By employing Unitary Phasor Dynamics and Hierarchical Learnable Anchors, PMNet maintains stable gradients, enabling more effective training through Backpropagation Through Time. In a byte-level demonstration, PMNet successfully utilized an 85-slot memory tree to achieve near-perfect retrieval over long temporal distances, outperforming larger Mamba models in zero-shot long-context robustness. AI

影响 Introduces a novel architecture that could enable more scalable and robust long-context sequence modeling.

排序理由 The cluster contains a new academic paper detailing a novel model architecture. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.CL 阅读 →

• Mamba
• Neural Turing Machine
• Phasor Memory Network
• PMNet

• 论文
• 模型发布

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →