RepNet: Tackling spectral bias in deep neural networks via parameter reparameterization
Researchers have introduced RepNet, a novel deep neural network architecture designed to address spectral bias, a common limitation in capturing high-frequency and oscillatory behaviors. By reparameterizing the weights and biases in the first hidden layer, RepNet effectively controls the initial slope scale and distribution of partition points. This adaptive frequency scaling during training allows RepNet to improve accuracy in approximating complex functions and solving PDE problems, particularly when combined with physics-informed neural networks, with only a marginal increase in computational cost. AI
IMPACT RepNet offers a new method for improving the accuracy of deep neural networks in capturing high-frequency data, potentially benefiting scientific computing and complex system modeling.