MatGPTQ

New Recurrent Residual Quantization method speeds up LLM deployment

Researchers have developed Recurrent Residual Quantization (RRQ), a novel post-training quantization framework for large language models (LLMs). RRQ allows for multiple effective precisions from a single checkpoint by r…