Dmitrii Andriianov

Spectral weight decay improves LLM compression and accuracy

Researchers have developed a new technique called spectral weight decay, which applies a nuclear-norm update to neural network weights to induce low-rank structure. This method improves compression and inference speed f…