MX-SAFE: Versatile Inference- and Training-Proof Microscaling Format with On-the-Fly Exponent and Mantissa Bit Allocation
Researchers have introduced MX-SAFE, a novel dynamic quantization format designed to reduce computational costs in deep learning. This format enhances the existing microscaling (MX) standard by adaptively allocating bits for exponents and mantissas, supporting both training and inference with improved accuracy. The proposed MX-SAFE format demonstrated an average accuracy improvement of up to 3.55% over existing MXFP formats and achieved comparable accuracy to BF16 baselines while consuming 24.9% less energy in a dedicated accelerator. AI
IMPACT This new quantization format could significantly reduce the energy consumption and computational cost of training and running AI models.
