Fisher-information flow

New framework bounds information acquisition in neural networks

Researchers have developed a new framework to understand how neural networks acquire information during the learning process. By modeling stochastic gradient descent (SGD) as a Markovian stochastic process, they derived…