Brief

Researchers have introduced MODE, a novel quantization framework designed to reduce the significant memory costs associated with Mixture-of-Experts Multimodal Large Language Models (MoE-MLLMs). The framework addresses biases in expert importance estimation that hinder performance in existing methods. By decomposing expert selection frequency by modality and filtering redundant vision tokens, MODE aims to improve the accuracy of quantization, especially for text-critical experts. Experiments demonstrate that MODE achieves substantial compression, with minimal performance loss even at extreme bit-width settings. AI

Researchers have developed a novel solver-free framework for tackling Integer Linear Programming (ILP) problems, which are common in combinatorial optimization. This new method directly explores feasible regions without relying on traditional solvers or machine learning training. It utilizes a Locally-Balanced Proposal for its transition kernel and incorporates Parallel Tempering, including a new penalty tempering technique that adjusts constraint barriers. The framework demonstrates superior performance compared to established solvers like SCIP and Gurobi on several benchmarks, showing greater robustness to distribution shifts than learning-based approaches. AI