Chong et al

NanoQuant implementation enables sub-1-bit model quantization

A new implementation of the NanoQuant method allows for flexible binary quantization of transformer models, reducing model size to sub-1-bit per weight. This approach factorizes matrices into scaling vectors and binary …