CKA-QAD

New distillation method preserves LLM internal geometry for better low-precision accuracy

Researchers have developed a new method called CKA-QAD to improve the accuracy of low-precision large language models (LLMs). Traditional methods like quantization-aware distillation (QAD) focus on matching output distr…