DopQ-ViT improves Vision Transformer quantization efficiency

Researchers have introduced DopQ-ViT, a novel post-training quantization method designed to improve the efficiency of Vision Transformers (ViTs). This method addresses performance degradation in low-bit quantization by aligning with the power-law distribution of post-Softmax activations and mitigating the impact of outliers in scaling factors. DopQ-ViT utilizes a new Tan Quantizer (TanQ) and a MAD-guided Optimal Scaling Factor (MOSF) to achieve better performance on classification and detection tasks compared to existing methods. AI

IMPACT Enhances the efficiency of Vision Transformers, potentially enabling wider deployment in resource-constrained environments.

RANK_REASON The cluster contains a research paper detailing a new method for model quantization. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• arXiv
• DopQ-ViT
• Hugging Face
• Lianwei Yang
• MAD-guided Optimal Scaling Factor
• Tan Quantizer
• Vision Transformers
• Vits

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →