SGD with momentum

New $\ell_p$-norm scheme enhances deep learning optimization

Researchers have introduced a new optimization scheme for deep neural networks that utilizes a dynamic $\ell_p$-norm, moving beyond the limitations of fixed $\ell_2$ and $\ell_\infty$ norms. This novel approach, termed …