Mamba-Transformer
PulseAugur coverage of Mamba-Transformer — every cluster mentioning Mamba-Transformer across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
German consortium releases open hybrid Mamba-Transformer model Soofi S 30B-A3B
A German research consortium has released Soofi S 30B-A3B, an open-source, hybrid Mamba-Transformer Mixture-of-Experts foundation model. Trained on Deutsche Telekom's AI cloud using NVIDIA B200 GPUs, the model prioritiz…
-
Open-source Soofi S 30B-A3B model released for German and English
Researchers have introduced Soofi S 30B-A3B, a new open-source foundation model designed for German and English languages. This model utilizes a hybrid Mamba-Transformer architecture, activating only 3 billion of its 30…
-
NVIDIA unveils Audex, a unified audio-text LLM that preserves text intelligence
NVIDIA has introduced Audex (Nemotron-Labs-Audex-30B-A3B), a unified audio-text large language model capable of understanding and generating both audio and speech. Unlike many multimodal models that experience a decline…
-
Nemotron 3 Ultra: Open-Source LLM Boasts 1M Context, 6x Throughput
Researchers have introduced Nemotron 3 Ultra, a 550 billion parameter language model that utilizes a hybrid Mamba-Transformer architecture with a Mixture-of-Experts approach. The model was trained on 20 trillion tokens …
-
NVIDIA releases open 550B Nemotron 3 models for agents and ASR
NVIDIA has released its Nemotron 3 family of open-source models, including Nemotron 3 Ultra and Nemotron 3.5 ASR. Nemotron 3 Ultra is a 550 billion parameter model designed for long-running AI agents, featuring a hybrid…
-
NVIDIA unveils 4-bit pretraining method, NVFP4, for LLMs
NVIDIA has developed a new 4-bit pretraining methodology called NVFP4, designed to overcome the challenges of reduced dynamic range and increased quantization error in narrower floating-point formats. This method was su…
-
New nGPT architecture enables native 4-bit training for LLMs
Researchers have developed a new neural network architecture called nGPT that natively supports 4-bit precision training for large language models. This architecture constrains weights and hidden representations to a un…
-
Why Nvidia builds open models with Bryan Catanzaro
Nvidia is significantly expanding its open model program, releasing higher quality models and datasets. This strategy benefits Nvidia by capturing value from open language models, creating a sustainable advantage. The c…