smuon

New sMuon method enhances low-rank fine-tuning for neural networks

Researchers have developed a new method called sMuon to improve parameter-efficient fine-tuning (PEFT) of neural networks. This technique addresses the incompatibility between the Muon optimizer and the common LoRA PEFT…