Akira Sakai

Neural network weight signs persist, limiting compression

Researchers have identified a phenomenon called "sign lock-in" in neural networks, where the initial random signs of weights tend to persist throughout training. This persistence acts as a bottleneck for sub-bit model c…