New framework enables genuine 1-bit quantization for LLMs

Researchers have developed a novel framework called All for 1-Bit (AF1) to achieve genuine 1-bit post-training quantization for large language models (LLMs). AF1 addresses the issue of existing binarization methods exceeding the nominal 1-bit storage target by incorporating Null-space-Aware Binary Factorization (NABF) and Hierarchical Shapley Allocation (HiSA). Experiments on models like LLaMA, Qwen, and Gemma demonstrate that AF1 significantly outperforms other binarization-based PTQ methods in terms of perplexity and accuracy, while also offering substantial inference speedups and memory reductions compared to bfloat16. AI

IMPACT Enables more efficient deployment of LLMs by drastically reducing storage and memory requirements.

RANK_REASON Research paper detailing a new method for LLM quantization. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• All for 1-Bit
• bfloat16
• Gemma
• Hierarchical Shapley Allocation
• LLaMA
• LLMs
• Null-space-Aware Binary Factorization
• Qwen

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →