Q5_K

New ExTernD method approaches bfloat16 accuracy for LLMs at low bit-widths

Researchers have developed ExTernD, a novel post-training quantization method for large language models that decomposes weight matrices into ternary factors. This technique allows for accuracy levels approaching bfloat1…