Mamba-2
PulseAugur coverage of Mamba-2 — every cluster mentioning Mamba-2 across labs, papers, and developer communities, ranked by signal.
5 day(s) with sentiment data
-
Selective State Space Models Show Token Consensus Similar to Transformers
Researchers have explored the dynamics of selective state space models (SSMs), comparing their token aggregation mechanisms to those in transformers. By analyzing SSMs from a dynamical systems perspective, they found th…
-
New research decomposes Mamba's associative recall, identifies training interventions
A new research paper published on arXiv explores the associative recall capabilities of fixed-state recurrent neural networks, specifically comparing Mamba and Mamba-2 architectures. The study decomposes recall performa…
-
New research proposes exact record omission for LLMs via checkpoint replay
A new research paper introduces a method for achieving exact record omission in large language models, a crucial aspect for privacy and data management. The study proposes a 'transport criterion' and a 'replay certifica…
-
AI's next frontier: Mamba, JEPA, and Diffusion Models poised to replace transformers
The AI landscape is experiencing a cyclical shift, with transformers, dominant since 2017, potentially being replaced by newer architectures like state space models (Mamba) and Joint Embedding Predictive Architectures (…
-
New 'Hidden State Poisoning Attack' targets Mamba-based AI models
Researchers have identified a new type of attack, termed Hidden State Poisoning Attack (HiSPA), that specifically targets state space models (SSMs) like Mamba. These attacks induce partial amnesia in the models by overw…
-
SSDi8 framework enhances Mamba-2 architecture with 8-bit quantization
Researchers have developed SSDi8, a novel 8-bit quantization framework specifically for the Mamba-2 architecture's Structured State Space Duality (SSD). This method aims to reduce the memory and latency overhead introdu…
-
NVIDIA releases Nemotron-Labs-Teacher models with 1M context · 4 sources tracked
NVIDIA has released a suite of Nemotron-Labs-Teacher models, each with 550 billion parameters, though only 55 billion are actively used. These models leverage a LatentMoE architecture incorporating Mamba-2, MoE, and Mul…
-
Nvidia's NeMo Switchyard cuts AI agent costs by 74%, overshadowing new model release
Nvidia has released two new technologies: Nemotron 3.5 Lightning, an open-weight language model, and NeMo Switchyard, an open-source routing library for AI agents. While Nemotron 3.5 Lightning is a standard 30B paramete…
-
State-Space Models: From S4 to Mamba Reviewed
This paper provides a comprehensive review of Structured State Space Models (SSMs), tracing their evolution from the initial S4 architecture to more advanced models like Mamba and Mamba-2. It analyzes key design dimensi…
-
New nGPT training method halves token requirements for MoE models
Researchers have developed a new training methodology called normalized Transformer (nGPT) that constrains model parameters and activations to a unit hypersphere for improved representation learning. This recipe, detail…
-
DART architecture enhances long-context sequence modeling by merging Transformers and SSMs
Researchers have introduced DART (Decoded Attention over Recurrent States), a novel architecture that combines the strengths of Transformers and State Space Models (SSMs) for efficient long-context sequence modeling. DA…
-
JAXBench launches to optimize AI kernels on Google TPUs
A new benchmark suite called JAXBench has been developed to specifically address the optimization of AI kernel performance on Google Cloud TPUs. This suite includes 50 JAX workloads derived from prominent AI models like…
-
New instrument precisely measures Mamba model state usage and input-driven migration
Researchers have developed a new instrument to precisely measure how selective state-space models, like Mamba, utilize their internal modes. This tool allows for the exact quantification of mode contributions to a model…
-
New instrument precisely measures state usage in Mamba models
Researchers have developed an exact instrument to measure state usage in selective state-space models like Mamba. This tool allows for the precise quantification of how trained models utilize their modes and predicts th…
-
NVIDIA Nemotron 3 Nano: Open Model for Efficient AI Agents
NVIDIA has released Nemotron 3 Nano, a 30-billion parameter open model designed for efficient reasoning and long-context applications. This model utilizes a hybrid Mixture-of-Experts architecture, activating only a frac…
-
NVIDIA unveils efficient Nemotron 3 LLM family with hybrid architecture
NVIDIA has released two new large language models, Nemotron 3 Nano and Nemotron 3 Ultra, focusing on efficiency and advanced capabilities. Nemotron 3 Nano is a 30B-class model designed for private inference and agentic …
-
Ternary Mamba achieves 3.61x compression via QAT with knowledge distillation
Researchers have developed a new method for compressing State Space Models (SSMs) like Mamba-2, significantly reducing their memory footprint for edge deployment. By employing grouped quantization-aware training (QAT) w…
-
New N-VSSM Model Outperforms Claude Opus 4.5 in Long-Form Narrative Consistency
Researchers have developed NarrativeWorldBench, a new benchmark designed to evaluate large language models (LLMs) on their ability to maintain narrative consistency in long-form audio dramas. Current frontier LLMs strug…
-
Compiler-first duality enables portable O(1) Mamba-2 inference
Researchers have developed a new method for optimizing Mamba-2 inference, focusing on compiler-first state space duality. This approach enables portable autoregressive caching with $O(1)$ complexity, eliminating the nee…
-
xLSTM outperforms Mamba-2 and DeltaNet in sequence modeling tasks
A new research paper compares three subquadratic architectures—xLSTM, Mamba-2, and Gated DeltaNet—for sequence modeling tasks. The study found that xLSTM outperformed the others in code-model pre-training, distillation,…