Egor Shvetsov

Neural network weight drift identified as a training dynamic issue

Researchers have identified a phenomenon called "weight drift" in neural networks, where optimization processes inadvertently push weights towards negative values. This drift, independent of the training data, occurs wi…