NVFP4
PulseAugur coverage of NVFP4 — every cluster mentioning NVFP4 across labs, papers, and developer communities, ranked by signal.
13 day(s) with sentiment data
-
Qwen3.6-27B model optimized for V100 GPUs hits 366 t/s
A developer has optimized the Qwen3.6-27B model for NVIDIA V100 GPUs, achieving up to 366 tokens per second in specific benchmarks. This optimization, named "v100-skinny," focuses on creating fast paths for NVFP4 weight…
-
TileRT AI boosts LLM decode interactivity on NVIDIA Blackwell GPUs
TileRT, a new technology from TileRT AI, promises to significantly boost decode interactivity for large language models on NVIDIA Blackwell GPUs. By statically compiling models into a persistent Engine Kernel, TileRT ai…
-
NVIDIA releases Nemotron 3.5 Lightning draft models for specialized decoding · 3 sources tracked
NVIDIA has released new draft models under the Nemotron 3.5 Lightning 30B-A3B series, designed for specialized decoding tasks. Nemotron-3.5-Lightning-30B-A3B-NVFP4-DFlash, with 833 million parameters, accelerates a 30B …
-
GGUF quants outperform NVFP4, AWQ, and FP8 for Qwen3.6 27B model
A comparison of various quantization methods for the Qwen3.6 27B model reveals that GGUF formats generally offer the best quality-to-size trade-offs. These GGUF models, which do not quantize activations, showed lower KL…
-
Quantized Qwen3-VL-32B-Heretic models released for MiniMax-H3 and H3 Healthcare
New quantized versions of the Qwen3-VL-32B-Heretic model are now available, specifically tailored for the MiniMax-H3 and H3 Healthcare Three Hop Index text encoders. These versions, including an NVFP4 quantization and a…
-
MiniMax H3 model quantized for local inference on consumer GPUs
The Abiray repository on Hugging Face offers quantized versions of the MiniMax H3 model, optimized for local inference on consumer GPUs. These collections include various formats like INT4, INT8, and NVFP4 for diffusion…
-
MiniMax H3 text encoder released, quantized to fit on 16GB GPU
The MiniMax H3 text encoder, quantized to NVFP4, has been released and is significantly smaller than its original 26.4 GB size, now fitting onto a single 16 GB graphics card. This model utilizes Qwen3-VL-32B as its text…
-
Laguna S 2.1 model updated with 1M context and new weights
Poolside has released updated weights for their Laguna S 2.1 model, specifically for FP8 and NVFP4 formats. These updates increase the default context size to 1 million tokens and include revised configurations. Users h…
-
vLLM v0.25.0 ships Model Runner V2, enhancing local LLM inference
The vLLM project has released version 0.25.0, featuring Model Runner V2 as the default for dense models, which enhances quantization support for more efficient local LLM inference. This update aims to improve throughput…
-
User shares vLLM optimizations for Qwen3.6-27B-NVFP4 model
A user on Reddit's r/LocalLLaMA shared their successful configuration for running the unsloth/Qwen3.6-27B-NVFP4 model using vLLM. They encountered Out-Of-Memory (OOM) errors and found that setting `MAX_JOBS=4` and `NVCC…
-
MXAttention framework optimizes MXFP4 attention for video generation
Researchers have developed MXAttention, a novel data-free post-training quantization framework designed to optimize MXFP4 attention in diffusion-based video generation models. This framework addresses numerical issues l…
-
Vivix launches A1, a real-time interactive multimodal model for virtual characters
Vivix has launched its first real-time interactive multimodal model, A1, designed to enable continuous video communication with virtual characters. The model allows characters to exhibit physical presence, interact with…
-
AMD challenges Nvidia with new Helios AI system and GPUs · 4 sources tracked
AMD is challenging Nvidia's dominance in AI computing with its new Helios rack-scale system, featuring 72 Instinct MI455X GPUs and next-generation EPYC processors. This integrated solution aims to provide a complete, re…
-
NVFP4 KV Cache Optimization with vLLM on Dual 5060 Ti
A user on Reddit shared a method for optimizing the KV cache using NVFP4 quantization with vLLM on a dual 5060 Ti setup. This approach, adapted from another user's work, appears to improve performance by leveraging spec…
-
New QUADS technique stabilizes NVFP4 RL for MoE LLMs
Researchers have developed a new technique called QUADS to stabilize reinforcement learning (RL) for Mixture-of-Experts (MoE) Large Language Models using the NVFP4 low-precision format. They identified activation error,…
-
Developer releases tools to visualize and analyze safetensors model quantization
A developer has created two Python tools to visualize and analyze safetensors files, commonly used for storing AI models. The first tool, `model_explorer.py`, displays the structure of a safetensors file in a tree forma…
-
Thinking Machines releases Inkling, a 1T-parameter multimodal LLM with 1M context
Thinking Machines has released Inkling, a large multimodal language model with approximately 1 trillion parameters and a 1 million token context window. The model natively processes text, image, and audio inputs, and fe…
-
Krea 2 Turbo model formats benchmarked for speed and quality in ComfyUI
A benchmark of Krea 2 Turbo model formats in ComfyUI reveals that the INT8 ConvRot format offers the best balance of speed and quality, particularly at higher resolutions. While BF16 provides the highest fidelity, INT8 …
-
RTX Pro 4500 GPU tested with new PrismaQuant LLM quantization methods
A user on Reddit's r/LocalLLaMA subreddit shared their experience testing new quantization methods for large language models on an RTX Pro 4500 GPU. They encountered issues with a specific Sakamakismile model, experienc…
-
Unsloth releases 2.5x faster Qwen3.6 quants with double context length
Unsloth has released new NVFP4 quantizations for the Qwen3.6 language model, achieving up to 2.5x faster performance compared to NVIDIA's standard NVFP4 quants without sacrificing accuracy. These optimizations utilize W…