Brief

Researchers have developed a new technique called Module-Adaptive Residual Reconstruction (MARR) to improve low-bit post-training quantization for large language models and vision transformers. MARR addresses limitations in existing methods by adaptively balancing error correction and bias across different model modules. This approach uses a module-specific scaling coefficient and a PID-based update strategy to refine coefficients, leading to significant performance gains, particularly at quantization levels of 4-bit or lower. AI

Researchers have developed UniRefiner, a framework designed to improve the spatial accuracy of Vision Transformer (ViT) models. This method teaches pre-trained ViTs to identify and discard irrelevant or spurious tokens that can degrade performance on spatially sensitive tasks. By using contrastive registers and a dual objective, UniRefiner refines diverse ViTs with minimal fine-tuning, leading to significant improvements in tasks like semantic segmentation. AI