New 4/6 quantization method boosts LLM accuracy with adaptive scaling

Researchers have developed a new quantization method called Four Over Six (4/6) to improve the accuracy of low-precision numerical formats like NVFP4 for large language models. This technique adaptively scales blocks to smaller FP4 values, reducing quantization error, particularly for near-maximal values. Experiments with the Nemotron 3 Nano 30B-A3B model architecture showed that 4/6 brings training loss closer to BF16 compared to existing NVFP4 methods, with minimal computational overhead. AI

IMPACT Improves efficiency of LLMs by reducing memory usage and increasing speed with minimal accuracy loss.

RANK_REASON Academic paper detailing a new method for model quantization. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Four Over Six
• Nemotron 3 Nano 30B-A3B
• NVFP4

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →