New derivative-free framework adapts Muon for optimization without gradients

Researchers have developed a derivative-free framework to adapt the Muon optimization method for scenarios where gradients are unavailable or unreliable. This new approach uses structured finite differences to construct Muon-style updates, offering four variants including random low-rank surrogates and direct structured search. Experiments on matrix-regression and noisy-gradient tasks demonstrate that structured probing can significantly reduce the number of function evaluations needed, potentially compensating for unreliable gradient oracles in specific black-box problems. AI

IMPACT This research offers a new method for optimization in machine learning, potentially enabling the use of gradient-free techniques in scenarios where traditional methods are not applicable.

RANK_REASON The item is an academic paper detailing a new method for optimization. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• alphaXiv
• arXiv
• CartPole
• CatalyzeX
• DagsHub
• Gotit.pub
• Hugging Face
• Muon
• ScienceCast

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →