Fp8
PulseAugur coverage of Fp8 — every cluster mentioning Fp8 across labs, papers, and developer communities, ranked by signal.
12 day(s) with sentiment data
-
New CurveFP datatypes promise lower cost and better performance for language models
Researchers have introduced CurveFP, a novel family of low-precision datatypes designed to reduce the cost of language models. CurveFP optimizes scalar fidelity and the arithmetic induced by products through a closed-pr…
-
GGUF quants outperform NVFP4, AWQ, and FP8 for Qwen3.6 27B model
A comparison of various quantization methods for the Qwen3.6 27B model reveals that GGUF formats generally offer the best quality-to-size trade-offs. These GGUF models, which do not quantize activations, showed lower KL…
-
New tools enable LLM fine-tuning on low-spec hardware
New tools and techniques are emerging to enable fine-tuning and running large language models (LLMs) on consumer-grade hardware. Soup CLI, an open-source Python tool, utilizes layer streaming to fine-tune an 8B LLM on a…
-
MiniMax H3 model released, runs on single DGX Spark
The MiniMax H3 model has been made publicly available, with details shared on Hugging Face and Arint.info. One user successfully ran the model on a single DGX Spark system, noting the need for experimentation with SM121…
-
NVIDIA Transformer Engine tutorial details GPU acceleration for LLMs
NVIDIA's Transformer Engine is detailed in a tutorial that explains how to accelerate transformer workloads. The engine combines fused GPU kernels, BF16 computation, and hardware-aware FP8 execution. The tutorial covers…
-
Laguna S 2.1 model updated with 1M context and new weights
Poolside has released updated weights for their Laguna S 2.1 model, specifically for FP8 and NVFP4 formats. These updates increase the default context size to 1 million tokens and include revised configurations. Users h…
-
INT4 Weight-Only Quantization: Decode Speedup, Prefill Stagnation Explained
Weight-only INT4 quantization, while effective for reducing memory traffic and speeding up the decoding phase of LLM inference, does not improve the prefill phase. This is because prefill is compute-bound, meaning it is…
-
New FP4 training method enables stable LLM training with reduced precision
Researchers have developed a novel method for training large language models (LLMs) using 4-bit floating-point precision (FP4), a significant reduction from the standard bfloat16 or FP8. This technique addresses the ins…
-
AMD MI4555X accelerator boasts 40 Petaflops and 432GB RAM
AMD's MI4555X accelerator reportedly offers impressive performance with 40 Petaflops of MXFP6 compute, a format close to FP8 quality. It also features a substantial 432 GB of RAM with a memory bandwidth of approximately…
-
Karpathy advocates for STEM-focused education; new AMD GPU specs revealed
Andrej Karpathy suggested that 80% of education should focus on math, physics, and computer science, implying that traditional schooling might be less relevant for future generations. Separately, the AMD MI4555X GPU was…
-
New QUADS technique stabilizes NVFP4 RL for MoE LLMs
Researchers have developed a new technique called QUADS to stabilize reinforcement learning (RL) for Mixture-of-Experts (MoE) Large Language Models using the NVFP4 low-precision format. They identified activation error,…
-
AMD ROCm users achieve faster Krea2 Turbo performance with INT8 Triton workaround
A user has successfully optimized the Krea2 Turbo model for AMD ROCm hardware by implementing a selective Triton workaround. This workaround allows INT8 ConvRot operations to run faster than FP8, achieving an 18% speed …
-
M+Adam optimizer improves low-precision LLM training
Researchers have introduced M+Adam, a novel optimization method designed to improve the accuracy of training large language models with low-precision weights. Standard optimizers can struggle with low precision, leading…
-
New research reveals "silent freeze" in low-precision AI training
Researchers have identified a phenomenon called "silent freeze" that occurs during low-precision training of deep learning models. This freeze happens when weight updates round away to zero, effectively halting learning…
-
Krea 2 Turbo model formats benchmarked for speed and quality in ComfyUI
A benchmark of Krea 2 Turbo model formats in ComfyUI reveals that the INT8 ConvRot format offers the best balance of speed and quality, particularly at higher resolutions. While BF16 provides the highest fidelity, INT8 …
-
Ostris AI Toolkit users seek FP8 weight loading optimization
A user on Reddit is seeking guidance on optimizing the Ostris AI Toolkit for StableDiffusion model training. They are encountering issues with the toolkit loading and quantizing full-sized model weights to FP8 every tim…
-
LoKA framework enables low-precision FP8 for large recommendation models
Researchers have developed LoKA, a framework designed to make low-precision arithmetic, specifically FP8, practical for large recommendation models (LRMs). Unlike LLMs, LRMs are sensitive to numerical precision and ofte…
-
Krea2 AI Model Performance: INT8 and NVFP4 Show Fastest Generation Times
A user on Reddit has conducted a comparison of various numerical formats for the Krea2 AI model, evaluating their performance and image generation quality. The tests included BF16, FP8, INT8, GGUF, MXFP8, and NVFP4, wit…
-
GIFT method speeds up LLM pretraining with geometry-informed gradients · 2 sources tracked
Researchers have developed GIFT, a novel method for optimizing large language model (LLM) pretraining by improving gradient communication. GIFT transforms gradients into a geometry-aware coordinate system before quantiz…
-
GLM-5.2 deployment on 8x B200 GPUs favors NVFP4 for optimal throughput
A technical analysis reveals that deploying the GLM-5.2 model on 8x NVIDIA B200 GPUs is most efficient using NVFP4 precision across four GPUs, rather than the more intuitive FP8 precision across all eight. This configur…