State Space Models
PulseAugur coverage of State Space Models — every cluster mentioning State Space Models across labs, papers, and developer communities, ranked by signal.
11 day(s) with sentiment data
-
ChronoSSM model jointly learns events and timestamps
Researchers have developed ChronoSSM, a novel autoregressive State Space Model designed to jointly learn event and timestamp representations. This approach contrasts with traditional methods that treat timing as a secon…
-
MixFormer: New Linear Transformer Enhances Long-Sequence Modeling
Researchers have introduced MixFormer, a novel linear Transformer designed to enhance efficiency in modeling ultra-long sequences. This model addresses limitations in existing State Space Models (SSMs) by incorporating …
-
New research explores advanced training methods for Spiking Neural Networks
Two recent arXiv papers explore advanced techniques for training spiking neural networks (SNNs). The first paper introduces a general framework for incorporating delays into SNNs using additional state variables, enhanc…
-
New frameworks enhance mask transformers and adapt State Space Models for missing data
Researchers have developed iFAN, a training framework designed to enhance mask transformers by aligning query ranking with mask quality and improving intermediate prediction distillation. This method addresses mismatche…
-
New research explores transformers for modeling dynamical systems · 2 sources tracked
Two new arXiv papers explore the application of transformer models to understanding and predicting dynamical systems. The first paper analyzes the mechanistic properties of single-layer transformers, interpreting causal…
-
New Python package simplifies differentiable particle filters for state-space models
Researchers have developed PyDPF, a new Python package built on PyTorch that implements several differentiable particle filters (DPFs). This package aims to make advanced Monte Carlo methods for state-space models more …
-
New metric uses eigenvalues to analyze memory dynamics in sequence models
Researchers have introduced a novel metric using eigenvalues to analyze and compare the memory dynamics of different sequence modeling architectures, specifically softmax attention and State Space Models (SSMs). This ap…
-
DART architecture enhances long-context sequence modeling by merging Transformers and SSMs
Researchers have introduced DART (Decoded Attention over Recurrent States), a novel architecture that combines the strengths of Transformers and State Space Models (SSMs) for efficient long-context sequence modeling. DA…
-
New research tackles catastrophic forgetting in AI models · 7 sources tracked
Researchers are developing novel methods to address catastrophic forgetting in continual learning, a challenge where AI models lose previously acquired knowledge when learning new tasks. Several recent arXiv papers prop…
-
DM3D: Dynamic Mamba architecture enhances point cloud understanding
Researchers have developed DM3D, a novel dynamic Mamba architecture designed to enhance point cloud understanding. This approach addresses the limitations of existing State Space Models (SSMs) by adapting local feature …
-
AI in Eye Care: Data, Preprocessing, and Models Evolve Together
A recent review paper details the co-evolution of data, preprocessing, and modeling techniques in the field of AI for color fundus photography (CFP) analysis. The paper highlights the progression of CFP datasets from sm…
-
Mamba's state-space models offer linear time and constant memory for LLMs
State-space models (SSMs), particularly the Mamba architecture, offer a more efficient alternative to attention mechanisms in large language models. Unlike attention, which has a quadratic time and memory cost with inpu…
-
New hybrid model combines CoLES and State Space Models for user transaction analysis
Researchers have developed a novel hybrid approach for user-centric modeling of transactional event sequences, combining contrastive representation learning (CoLES) with State Space Models (SSMs). This method addresses …
-
New DSSMs enhance long-sequence modeling with explicit memory
Researchers have introduced Delay State Space Models (DSSMs), an extension of diagonal State Space Models designed to improve long-sequence modeling by incorporating explicit delayed-state feedback. This approach addres…
-
New Dirichlet-Process Cache Stores Distinct Information, Outperforming Attention
Researchers have developed a novel memory system for sequence models that stores distinct information rather than individual tokens, addressing the limitations of fixed-state models and the computational cost of attenti…
-
New Vision SSM Eliminates Directional Scanning for Improved Image Recognition
Researchers have introduced the Vision Non-Causal Trapezoidal Mamba (VNCT), a novel second-order non-causal State Space Model (SSM) designed for visual recognition tasks. Unlike previous vision SSMs that rely on directi…
-
Backpropagation and Matrix Calculus Explained Through Code and Analogy
This article explains the mathematical concepts of backpropagation and matrix calculus as they apply to deep learning. It uses an analogy of a factory assembly line to illustrate how errors are identified and corrected …
-
MuonSSM framework enhances State Space Models for sequence modeling · 2 sources tracked
Researchers have introduced MuonSSM, a novel framework designed to enhance the stability and performance of State Space Models (SSMs) in sequence modeling tasks. By focusing on conditioning the geometry of memory update…
-
Higher-Order FNO advances neural operators for nonlinear PDEs · 2 sources tracked
Researchers have introduced the Higher-Order Fourier Neural Operator (HO-FNO), an advancement on the Fourier Neural Operator (FNO) designed to better handle nonlinear partial differential equations (PDEs). HO-FNO incorp…
-
New MASS framework improves video frame interpolation with motion-aligned scanning
Researchers have introduced MASS (Motion-Aligned Selective Scan), a novel framework for video frame interpolation that addresses challenges posed by large, non-linear motions and occlusions. Unlike previous methods that…