OmniQuant

New method slashes LLM quantization bit-width with spectral rotations

Researchers have developed a novel method called BBT-spectral for quantizing large language models (LLMs) to extremely low bit-widths, specifically W2A16 (2-bit weights, 16-bit activations). This technique utilizes infl…