CoT Token Inflation Ratio

Quantization of LLMs inflates reasoning token usage, researchers find

A new research paper highlights that while quantization techniques like INT4 and INT3 are effective at reducing the inference costs of large language models, they can unexpectedly inflate reasoning token usage. This phe…