NVFP4
PulseAugur coverage of NVFP4 — every cluster mentioning NVFP4 across labs, papers, and developer communities, ranked by signal.
23 day(s) with sentiment data
BAAI's AREX-2 model will see official NVFP4 quantization support within 60 days
The BAAI AREX-2 model's serving notes mention pending NVFP4 support. Given that NVFP4 is already being adopted by other major model releases (Hugging Face Beam, StepFun), it's probable that the community or BAAI itself will prioritize developing and releasing NVFP4-optimized weights for AREX-2 to capitalize on hardware efficiencies.
NVFP4 quantization is a key enabler for recent large MoE model releases
NVFP4 quantization is explicitly mentioned as an optimization for NVIDIA hardware in both the StepFun and Hugging Face MoE model releases. This suggests that NVFP4 is a critical technology enabling the efficient deployment and inference of these massive models, particularly those with long context windows.
NVFP4 support will become a standard requirement for high-performance LLM serving frameworks
Given the increasing prevalence of NVFP4 in the release notes of major models and its mention as an optimization for NVIDIA hardware, it is likely that serving frameworks like vLLM will prioritize and standardize NVFP4 support. This could manifest as direct integration, performance benchmarks, or specific configuration options for NVFP4-quantized models.
-
NVIDIA Nemotron 3.5 Super VL model now available on Amazon SageMaker JumpStart
NVIDIA's Nemotron 3.5 Super VL, a multimodal vision-language model, is now accessible through Amazon SageMaker JumpStart. This model can process images, video, and text to generate text outputs, supporting context lengt…
-
New DiRotQ method enhances 4-bit quantization for Diffusion Transformers
Researchers have developed DiRotQ, a new 4-bit quantization framework for Diffusion Transformers (DiTs) that significantly reduces memory and computational costs while preserving image generation quality. The method uti…
-
New BitDecoding-FP4 method boosts LLM serving throughput with FP4 KV cache
Researchers have developed BitDecoding-FP4, a novel algorithm-system co-design for efficient and accurate native FP4 KV cache decoding in large language models. This approach addresses accuracy drops and throughput degr…
-
Thinking Machines Lab releases Inkling 2 Small with 1M context window
Thinking Machines Lab has released Inkling 2 Small, a 594B-parameter mixture-of-experts model featuring image input and a 1M-token context window. While priced competitively per token, its extensive output token usage f…
-
Inkling 2 Small model demands 200GB RAM, posing local deployment hurdles
Running the new Inkling 2 Small model locally presents significant hardware challenges, primarily due to its 594 billion parameters. While the model supports 17 billion active parameters per token, the entire model must…
-
TensorRT-LLM Production Deployment Guide Released for MoE Models
ServerMO has released a guide detailing the production deployment of TensorRT-LLM, focusing on advanced techniques for serving large Mixture-of-Experts (MoE) models. The guide covers quantization methods like FP8 for Ho…
-
llama.cpp adds NVFP4 scale support for Qwen2 model
The llama.cpp project has released version b12126, which includes an update for the Qwen2 model. This update addresses a critical issue where models converted to NVFP4 format would crash due to unapplied scales in the F…
-
llama.cpp b12122 release optimizes CUDA integration for MMQ operations
The llama.cpp project has released version b12122, which includes several updates focused on CUDA integration. These updates involve fusing NVFP4 gate/up matmuls and GLU operations within the MMQ (Mixed-Quantized Matrix…
-
Hydrogen-1.0 scientific model challenges GPT-6 Astra with lower costs
Deep Principle has released Hydrogen-1.0, an open-weight scientific model based on GLM-5.3, designed for chemistry, materials science, and life sciences. In evaluations, Hydrogen-1.0 reportedly outperformed GPT-6 Astra …
-
Mistral Large 4 open weights released; self-hosting requires substantial hardware
Mistral AI has released the open weights for its Mistral Large 4 model, a 1.05 trillion parameter mixture-of-experts architecture. This model, nicknamed "Le Chonk," features a 1 million token context window and supports…
-
FLAMIA kernel accelerates Transformer training on NVIDIA Blackwell GPUs
Researchers have developed FLAMIA, a new attention kernel designed to accelerate Transformer training on NVIDIA Blackwell GPUs. FLAMIA utilizes mixed-precision kernels, specifically MXFP8 and NVFP4, to reduce quantizati…
-
New calibration methods enhance HiFloat4 quantization accuracy for LLMs
Researchers have developed three new calibration approaches—HiGPTQ, HiAutoRound, and HiFlatQuant—to improve the accuracy of large language models quantized using the HiFloat4 (HiF4) format. These methods address the cha…
-
Nemotron Labs releases Dory1.0-4B looped language model
Nemotron Labs has released Dory1.0-4B, a 4-billion-parameter language model that utilizes a novel looped architecture. This design allows the model to achieve improved accuracy and token efficiency by applying a shared …
-
Anthropic's Numbat model checkpoint leaked online
A leaked checkpoint for Anthropic's unreleased "Mythos-class" model, codenamed Numbat, has surfaced online. The checkpoint, which is approximately 7.72 TB and contains 142 tensors, is described as a mid-pretraining arti…
-
Qwen 3.8 Flash Next runs at high speeds on consumer GPUs · 4 sources tracked
Users are sharing guides and benchmarks for running the Qwen 3.8 Flash Next model on consumer hardware, achieving impressive speeds and quality. One user details a setup on an RTX 5080 with 64GB RAM, reaching over 250 t…
-
Mistral AI releases open-weights Large 4 model for self-hosting
Mistral AI has released the weights for its flagship Mistral Large 4 model, allowing organizations to host it on their own servers. The model boasts approximately 1.05 trillion parameters and supports a 1 million token …
-
Developer creates local GPU benchmark server for open-weight LLMs
A developer has created a local GPU benchmark server to evaluate the performance of open-weight LLMs, particularly quantized versions, on consumer hardware. The server, equipped with two RTX 5060 Ti cards, tests models …
-
Triage method optimizes attention tiles in video diffusion transformers
Researchers have developed a new method called Triage for optimizing attention mechanisms in video diffusion transformers (DiTs). This training-free approach categorizes attention tiles into three tiers: skipped, skimme…
-
DiffusionGemma runs locally on RTX 5090 via vLLM, facing ecosystem hurdles
Running Google's DiffusionGemma model locally on an RTX 5090 GPU is feasible but comes with challenges due to the model's unique block diffusion architecture and the immaturity of the surrounding ecosystem. The author f…
-
Open-weight AI models released in October 2026 include Meta's 816B parameter Muse Spark Open
In October 2026, several open-weight AI models were released, highlighting the growing trend of making model parameters publicly available. Meta's Muse Spark Open, a massive 816B parameter model, requires approximately …