New Muon-C optimizer enhances convolutional kernel performance

Researchers have developed Muon-C, a novel optimizer for convolutional kernels that improves upon existing methods like Adam and unfolded Muon. By representing kernel momentum as frequency-wise channel-transfer matrices and polarizing these blocks independently, Muon-C ensures updates remain within the original finite kernel support. This new geometry leads to superior performance, achieving a 9.87 FID on CIFAR-10 flow matching with fewer FLOPs compared to its predecessors, and a 3.42 FID under equal tuning budgets. AI

IMPACT Introduces a more efficient optimization method for convolutional neural networks, potentially leading to faster training and better performance in image-related tasks.

RANK_REASON This is a research paper detailing a new optimization technique for machine learning models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• Adam
• arXiv
• CIFAR-10
• convolutional kernels
• Muon

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →