New DSSMs enhance long-sequence modeling with explicit memory

Researchers have introduced Delay State Space Models (DSSMs), an extension of diagonal State Space Models designed to improve long-sequence modeling by incorporating explicit delayed-state feedback. This approach addresses the limitation of traditional SSMs in compressing unbounded history into a fixed state, which hinders precise retrieval over long contexts. DSSMs achieve this through new stability parameterizations, history management, and FFT-training tools, enabling them to outperform existing models on targeted delayed-retrieval tasks and maintain strong performance on standard sequence metrics. AI

IMPACT DSSMs offer improved context retention and retrieval for long sequences, potentially benefiting applications requiring deep historical understanding.

RANK_REASON The cluster contains a research paper detailing a new model architecture. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Delay differential equations for mode-locked semiconductor lasers
• DSSMs
• Hugging Face
• State Space Models

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →