Researchers have developed a new post-training quantization (PTQ) framework called Token-Group Quantization for Mamba-based super-resolution models. This method addresses the challenge of varying activation ranges across spatial tokens in Mamba models, which complicates low-bit quantization. The framework approximates a hardware-unfriendly grouping oracle by assigning tokens with similar clipping ranges to the same group, using a lightweight predictor to infer these groups from pre-scan features. Additionally, a Frequency-Preserving Refinement objective is introduced to maintain high-frequency structures during calibration, leading to improved performance and a 2.23x speedup on NVIDIA Jetson Orin Nano over FP16. AI
IMPACT This research could lead to more efficient deployment of Mamba-based models on edge devices, improving performance and reducing computational costs.
RANK_REASON The cluster contains a research paper detailing a novel method for optimizing Mamba models. [lever_c_demoted from research: ic=1 ai=1.0]
- arXiv
- Frequency-Preserving Refinement
- Hugging Face
- Mamba
- NVIDIA Jetson Orin Nano 8GB
- Token-Group Quantization
- transformer
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →