xLSTM: Extended Long Short-Term Memory
PulseAugur coverage of xLSTM: Extended Long Short-Term Memory — every cluster mentioning xLSTM: Extended Long Short-Term Memory across labs, papers, and developer communities, ranked by signal.
3 day(s) with sentiment data
-
Researchers explore using initial state for conditioning in recurrent TSFMs
Researchers have explored utilizing the initial state in recurrent time series foundation models (TSFMs) as a conditioning interface, rather than leaving it unused by convention. A case study using the xLSTM-based TiRex…
-
Near-zero embeddings stall gated model training; new fixes proposed
A new paper identifies a critical training instability in gated sequence models, particularly those with untied embeddings. This issue, caused by near-zero input embeddings, can lead to extreme gradient amplification, e…
-
Deep learning models struggle with financial forecasting benchmarks, study finds
A new study published on arXiv evaluates 16 deep learning architectures for financial time series forecasting, finding that many popular models like generic Transformers and State Space Models underperform simpler bench…
-
KronLSTM introduces Kronecker Delta Rule for stable long-context AI memory
Researchers have introduced KronLSTM, a novel linear attention architecture that utilizes a Kronecker Delta Rule for stable associative memory updates. This approach enhances long-context modeling by allowing for precis…
-
RNN Homunculus improves long-term memory in recurrent neural networks
Researchers have developed a new approach called the "RNN Homunculus" to improve the ability of Recurrent Neural Networks (RNNs) to learn long-range dependencies. This method focuses on how the gates within RNNs, such a…
-
Post-Transformer AI Architectures Mapped: Mamba, RWKV, and More
A Reddit user has compiled a comprehensive comparison of post-Transformer AI architectures, detailing advancements beyond the standard Transformer model. The analysis highlights various approaches like Mamba, RWKV, xLST…
-
New research tackles autonomous vehicle perception under communication loss · 2 sources tracked
Two new research papers propose novel frameworks for improving collaborative perception in autonomous vehicles, particularly under challenging communication conditions. LR-V2X focuses on resilience to packet loss and lo…
-
Review paper details long horizon forecasting challenges and deep learning solutions
A new review paper published on arXiv details the long horizon forecasting (LHF) problem in time series analysis, a challenge that has persisted for over 35 years. The paper explores how deep learning techniques, includ…
-
xLSTM models achieve near-lossless distillation from larger LLMs
Researchers have developed an effective distillation pipeline to transfer knowledge from large language models (LLMs) with quadratic attention to sub-quadratic architectures based on xLSTM. This method aims for lossless…
-
TiRex-2 model advances multivariate time series forecasting with recurrent xLSTM design
Researchers have introduced TiRex-2, a novel recurrent foundation model based on xLSTM architecture designed for multivariate time series forecasting. This model addresses limitations of existing Transformer-based appro…
-
xLSTM outperforms Mamba-2 and DeltaNet in sequence modeling tasks
A new research paper compares three subquadratic architectures—xLSTM, Mamba-2, and Gated DeltaNet—for sequence modeling tasks. The study found that xLSTM outperformed the others in code-model pre-training, distillation,…
-
New X-TRACK model uses xLSTM and physics for realistic vehicle trajectory prediction
Researchers have developed X-TRACK, a novel trajectory prediction model for autonomous driving that leverages the extended Long Short-Term Memory (xLSTM) architecture. This new model explicitly incorporates vehicle moti…
-
CogScale benchmark accelerates AI sequence processing evaluation
Researchers have introduced CogScale, a new benchmark designed to efficiently evaluate the sequential processing capabilities of AI architectures. This benchmark comprises 14 scalable synthetic tasks that allow for rapi…
-
Short window attention boosts long-term memory in AI models
Researchers have developed a hybrid architecture combining sliding window attention and xLSTM layers to improve long-term memorization in AI models. Their findings indicate that surprisingly, larger sliding windows can …
-
xLSTM networks enhance deep reinforcement learning for automated stock trading
Researchers have developed a new automated stock trading system utilizing Extended Long Short-Term Memory (xLSTM) networks combined with deep reinforcement learning (DRL). This approach aims to overcome the limitations …