Mamba-2
PulseAugur coverage of Mamba-2 — every cluster mentioning Mamba-2 across labs, papers, and developer communities, ranked by signal.
- used by LatentMoE 90%
- instance of State Space Models 90%
- instance of Recurrent Neural Networks 90%
- uses LatentMoE 90%
- instance of Mamba-1 90%
- used by OpenMDW-1.1 90%
- competes with Gated DeltaNet 70%
- instance of alphaXiv 70%
- instance of ScienceCast 70%
- competes with Mamba-3 70%
- competes with Recurrent Neural Networks 70%
- affiliated with State Space Models 70%
8 day(s) with sentiment data
-
New SEED method optimizes hybrid LLM memory and compute efficiency
A new research paper introduces SEED, a method designed to optimize the memory and compute efficiency of hybrid language models. SEED addresses the inefficiencies in how these models handle memory by making the prefill …
-
New DAMI technique boosts long-context retrieval in linear attention models
Researchers have introduced Dual-Axis Memory Initialization (DAMI), a novel technique to improve long-context retrieval in linear attention models. DAMI focuses on optimizing the initialization of gate parameters that c…
-
NVIDIA Nemotron 3.5 Super VL model deployed on AWS for agentic AI
NVIDIA has released its Nemotron 3.5 Super VL model, a 120B-parameter mixture-of-experts model capable of multimodal understanding and agentic workflows. This model integrates reasoning and tool use with text, image, do…
-
NVIDIA releases Nemotron 3.5 Transcribe, a top-ranked open-source speech-to-text model
NVIDIA has released Nemotron 3.5 Transcribe, an open-source speech-to-text model that ranks highly on the Hugging Face Open ASR Leaderboard. The model, available under the permissive OpenMDW-1.1 license, can be used com…
-
Metis-1.6 models achieve Pareto-optimal efficiency with novel architectures
A single researcher has developed Metis-1.6, a new suite of language models, Praxis and Logos, that prioritize efficient training methods and novel architectures over sheer parameter count. Despite not competing with fr…
-
Lightweight Mamba-2 Model Achieves 74.75% Accuracy in EEG Cognitive Workload Classification
Researchers have developed FOCUS, a lightweight Mamba-2 based model designed for classifying cognitive workload using electroencephalography (EEG) data. This compact model, with only 0.04 million parameters, utilizes a …
-
AI developments span model compression, code optimization, and talent investment
The llama.cpp project has released version b11875, which includes enhancements to its hexagon tools for collapsing and reshaping operations across various platforms like macOS, Linux, and Windows. Separately, a research…
-
NoteBank enhances long-context recall in linear-attention models
Researchers have developed a novel technique called NoteBank to enhance long-context recall in language models that use linear attention or state-space architectures. This method acts as a training-free add-on, storing …
-
NVIDIA releases Nemotron 3.5 Super VL and edge models for enterprise AI agents · 8 sources tracked
NVIDIA has released its Nemotron 3.5 family of open multimodal models, designed for enterprise AI agents. The flagship Nemotron 3.5 Super VL, with 120 billion parameters, offers high accuracy and speed for specialized t…
-
ReplaySSM optimizes State Space Models, boosting decoding speed
Researchers have introduced ReplaySSM, a novel method designed to optimize the performance of State Space Models (SSMs) in AI applications. This technique addresses bottlenecks caused by frequent state input/output oper…
-
SketchSSM method boosts LLM state-read efficiency by 10x
Researchers have introduced SketchSSM, a novel method that enhances the efficiency of hybrid-attention models by approximating state reads. This technique reduces KV-cache growth and enables larger decode batches by buf…
-
New Exponentially Weighted Signature Model Extends State-Space Models
Researchers have introduced the exponentially weighted signature (EWS), a novel continuous-time model that extends state-space models (SSMs) by computing iterated integrals of a path with increments weighted by a learna…
-
New REALM framework enables causal LFP decoding for brain-computer interfaces
Researchers have developed REALM, a novel framework for causal local field potential (LFP) behavior decoding. This retrospective knowledge distillation approach transfers representational knowledge from a non-causal "te…
-
New attention mechanisms boost long-context sequence modeling
Two new papers introduce novel approaches to enhance long-context sequence modeling in recurrent neural networks. The first paper, "SMat-Attention," proposes Structured Matrix Attention, which uses structured causal mas…
-
Selective State Space Models Show Token Consensus Similar to Transformers
Researchers have explored the dynamics of selective state space models (SSMs), comparing their token aggregation mechanisms to those in transformers. By analyzing SSMs from a dynamical systems perspective, they found th…
-
New research decomposes Mamba's associative recall, identifies training interventions
A new research paper published on arXiv explores the associative recall capabilities of fixed-state recurrent neural networks, specifically comparing Mamba and Mamba-2 architectures. The study decomposes recall performa…
-
New research proposes exact record omission for LLMs via checkpoint replay
A new research paper introduces a method for achieving exact record omission in large language models, a crucial aspect for privacy and data management. The study proposes a 'transport criterion' and a 'replay certifica…
-
AI's next frontier: Mamba, JEPA, and Diffusion Models poised to replace transformers
The AI landscape is experiencing a cyclical shift, with transformers, dominant since 2017, potentially being replaced by newer architectures like state space models (Mamba) and Joint Embedding Predictive Architectures (…
-
New 'Hidden State Poisoning Attack' targets Mamba-based AI models
Researchers have identified a new type of attack, termed Hidden State Poisoning Attack (HiSPA), that specifically targets state space models (SSMs) like Mamba. These attacks induce partial amnesia in the models by overw…
-
SSDi8 framework enhances Mamba-2 architecture with 8-bit quantization
Researchers have developed SSDi8, a novel 8-bit quantization framework specifically for the Mamba-2 architecture's Structured State Space Duality (SSD). This method aims to reduce the memory and latency overhead introdu…