Quantization impacts code generation models differently, study finds

A new study investigates the impact of various quantization methods on the performance of large code generation models when run on resource-constrained hardware. Researchers evaluated six state-of-the-art techniques, including GPTQ, AWQ, and AQLM, on Qwen2.5-Coder and CodeLlama models using Python and Java benchmarks. The findings indicate that quantization methods have a significant and varied effect on code correctness and quality, with AQLM performing comparably to full-precision models and QuIP# showing the largest degradation. AI

IMPACT Provides guidance for deploying large code models on resource-constrained devices by evaluating the trade-offs of different quantization techniques.

RANK_REASON The cluster contains a research paper detailing an empirical study on quantization methods for code generation models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• Activation Aware Quantization
• AQLM
• bitsandbytes
• CodeLlama
• CoderEval: A Benchmark of Pragmatic Code Generation with Generative Pre-trained Models
• GGUF
• GPTQ
• Java
• McEval
• Ollama
• Python
• Quip#
• Qwen2.5 Coder

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →