Diagonal SSMs have limited expressivity for state-tracking, research finds

A new research paper explores the theoretical limitations of Diagonal State-Space Models (SSMs) when applied to state-tracking tasks. The study demonstrates that single-layer Diagonal SSMs are incapable of tracking non-Abelian groups with finite precision. Furthermore, the research establishes that k-layer Diagonal SSMs can only track groups with a specific subnormal series structure, effectively defining their expressivity within solvable groups. Empirically, the paper notes that multi-layer models often struggle to learn state-tracking for non-Abelian groups, indicating a discrepancy between theoretical expressivity and practical learnability. AI

IMPACT This research clarifies the theoretical boundaries of Diagonal SSMs, potentially guiding future model development and application in sequence modeling tasks.

RANK_REASON The cluster contains a research paper detailing theoretical findings about the expressive limits of a specific type of AI model. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• alphaXiv
• arXiv
• CatalyzeX
• DagsHub
• Diagonal SSMs
• Gotit.pub
• Hugging Face
• IArxiv
• Mehran Shakerinava
• ScienceCast
• State-Space Models

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →