TORQ: Two-Level Orthogonal Rotation for MXFP4 Quantization
Researchers have developed TORQ, a new framework for quantizing Large Language Models (LLMs) using the MXFP4 format. This method addresses accuracy degradation issues by analyzing and correcting imbalances in activation quantization. TORQ employs a two-level orthogonal rotation strategy to optimize the activation space, significantly improving LLM accuracy with 4-bit floating-point quantization. AI
IMPACT Improves LLM efficiency and accuracy by enabling better low-bit quantization, potentially reducing inference costs.