New R-SGD-Mini method tackles heavy-tailed noise in optimization

Researchers have introduced a new optimization method called Robust Stochastic Gradient Descent with medoid mini-batch gradient sampling (R-SGD-Mini). This method is designed to handle heavy-tailed noise in gradient calculations, which can have infinite variances. R-SGD-Mini works by dividing data batches into smaller chunks, calculating gradients for each, and then using the medoid of these gradients to update the solution estimate. The approach has demonstrated improved performance over existing methods like standard SGD and Median-of-Means in experimental settings. AI

影响 Introduces a novel method for handling noisy gradients in optimization, potentially improving the stability and performance of machine learning training processes.

排序理由 The cluster contains an arXiv paper detailing a new optimization method. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

• R-SGD-Mini
• SGD

• 论文
• 其他

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →