Alexander Tyurin

Gradient Descent Mimics Perceptron Algorithm in Neural Networks

Researchers have demonstrated that gradient descent steps in neural networks trained with logistic loss can be simplified to resemble generalized perceptron algorithms. This new perspective, using classical linear algeb…