MXFP8
PulseAugur coverage of MXFP8 — every cluster mentioning MXFP8 across labs, papers, and developer communities, ranked by signal.
7 day(s) with sentiment data
-
Inkling 2 Small model demands 200GB RAM, posing local deployment hurdles
Running the new Inkling 2 Small model locally presents significant hardware challenges, primarily due to its 594 billion parameters. While the model supports 17 billion active parameters per token, the entire model must…
-
Kimi K3.1 Pro model precision details revealed as inaccurate
A technical analysis of the Kimi K3.1 Pro model revealed discrepancies in its reported bit precision. While the model page indicated 8-bit precision and the download badge suggested the same, a detailed calculation base…
-
FLAMIA kernel accelerates Transformer training on NVIDIA Blackwell GPUs
Researchers have developed FLAMIA, a new attention kernel designed to accelerate Transformer training on NVIDIA Blackwell GPUs. FLAMIA utilizes mixed-precision kernels, specifically MXFP8 and NVFP4, to reduce quantizati…
-
Nemotron Labs releases Dory1.0-4B looped language model
Nemotron Labs has released Dory1.0-4B, a 4-billion-parameter language model that utilizes a novel looped architecture. This design allows the model to achieve improved accuracy and token efficiency by applying a shared …
-
MXFP8 Reinforcement Learning on Blackwell: Speed, Consistency, and Failure Modes Explored
A new research paper explores the use of MXFP8 precision for end-to-end reinforcement learning (RL) with large language models on NVIDIA's Blackwell platform. The study implements and measures this approach using variou…
-
MXFP8 Reinforcement Learning for LLMs: Gains, Pains, and Correctness
A new paper explores the use of MXFP8 precision for reinforcement learning (RL) in large language models, aiming to reduce computational costs. The research implements end-to-end MXFP8 RL on B200 GPUs using Qwen models …
-
NVFP4 quantization offers 2.3x faster decoding on Ada GPUs, matching BF16 quality
A technical blog post compares the performance of different quantization formats for large language models on an NVIDIA Ada GPU. The analysis focuses on NVFP4, FP8, and GGUF, finding that NVFP4 offers a significant spee…
-
Llama.cpp fixes minimax-m3 scaling, new research on geometric style transfer, and OpenAI model security concerns
A recent release for the llama.cpp project, specifically version b11861, addresses an issue where scale tensors for the minimax-m3 model were not correctly forwarded, leading to incorrect weight scaling and token soup. …
-
Imagination E-series GPUs integrate AI for faster rendering and LLM inference
Imagination Technologies has unveiled its new E-series GPU architecture, designed to integrate AI capabilities directly into graphics processing. This new architecture features a "Neural Core" that can perform AI upscal…
-
MiniMax M3 LLM Performance Tweaked in llama.cpp
A user is experimenting with the MiniMax M3 large language model on a Mac, specifically within the llama.cpp framework. They encountered occasional minor hallucinations and oddities with the model, which they suspect mi…
-
Kimi K3's 2.8T parameters highlight LLM deployment as a systems engineering challenge
The Kimi K3 model, with 2.8 trillion total parameters and approximately 104 billion active parameters per token, presents significant deployment challenges beyond its sheer size. Its architecture incorporates a mixture-…
-
Ollama v0.32.10-rc0 speeds up model prefill performance
Ollama has released version v0.32.10-rc0, introducing optimizations for double-scale NVFP4 models. This update compiles multiply and cast operations into a single kernel, reducing overhead from separate eager ops. Bench…
-
Motif Technologies unveils 314B parameter Motif 3 LLM
Motif Technologies has released Motif 3, a decoder-only Mixture-of-Experts language model with 314 billion total parameters and 13.2 billion activated per token. The model features a novel Grouped Differential Latent At…
-
New FP4 training method enables stable LLM training with reduced precision
Researchers have developed a novel method for training large language models (LLMs) using 4-bit floating-point precision (FP4), a significant reduction from the standard bfloat16 or FP8. This technique addresses the ins…
-
Moonshot releases Kimi K3, a 2.8T parameter multimodal model with 1M context
Moonshot has released Kimi K3, a new 2.8 trillion parameter multimodal model featuring a 1 million token context window and native vision capabilities. The model demonstrates impressive speed, achieving 460 tokens per s…
-
Krea 2 Turbo model formats benchmarked for speed and quality in ComfyUI
A benchmark of Krea 2 Turbo model formats in ComfyUI reveals that the INT8 ConvRot format offers the best balance of speed and quality, particularly at higher resolutions. While BF16 provides the highest fidelity, INT8 …
-
Krea2 AI Model Performance: INT8 and NVFP4 Show Fastest Generation Times
A user on Reddit has conducted a comparison of various numerical formats for the Krea2 AI model, evaluating their performance and image generation quality. The tests included BF16, FP8, INT8, GGUF, MXFP8, and NVFP4, wit…
-
DynamiQ framework accelerates LLM training with optimized gradient synchronization
Researchers have developed DynamiQ, a new framework designed to accelerate the training of large language models by optimizing gradient synchronization. This method addresses the network bottleneck issue in large-scale …
-
New LLM Quantization Methods Boost Speed and Accuracy
Two new research papers introduce novel quantization techniques to improve the efficiency of large language models (LLMs). FPTQuant focuses on function-preserving transforms for INT4 quantization, achieving up to 3.9X s…
-
Krea 2 image model released in multiple quantized formats for broader GPU access
The Krea 2 image generation model has been released in quantized versions, including FP8, MXFP8, NVFP4, and INT8 formats, making it accessible for a wider range of GPUs. The model comes in two variants: Krea 2 Raw for t…