Nvidia has released Nemotron-Labs-Sigma, a new family of language models that utilize a continuous diffusion approach for text generation. The 8-billion-parameter Nemotron-Labs-Sigma-8B-instruct model, fine-tuned for instruction following, generates text in parallel blocks by denoising Gaussian noise in a learned embedding space. This method allows for faster generation with fewer steps while maintaining competitive reasoning accuracy compared to traditional masked diffusion models. The models are available for commercial and non-commercial use and support a context length of up to 8,192 tokens. AI
IMPACT Introduces a novel diffusion-based generation method for LLMs, potentially improving inference speed and quality.
RANK_REASON Frontier-lab model release with system card [lever_c_demoted from frontier_release: ic=1 ai=1.0]
- Nemotron-Labs-Sigma
- Nemotron-Labs-Sigma-3B
- Nemotron-Labs-Sigma-3B-instruct
- Nemotron-Labs-Sigma-8B
- Nemotron-Labs-Sigma-8B-instruct
- Nvidia
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →