A discussion on Reddit's r/MachineLearning subreddit questions why OKLS (Optimized Kernel Low-rank Adaptation) is not outperforming Muon in step efficiency, despite OKLS being a zero-staleness implementation of KL-Shampoo. The original poster notes that KL-Shampoo is generally more step-efficient than SOAP, and OKLS is expected to follow suit. While OKLS reportedly surpasses Muon in parameter efficiency during very long training runs, its performance in terms of steps per iteration is a point of confusion. AI
RANK_REASON Reddit discussion questioning the performance of two methods.
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →