Fourier fractal dimension to predict the generalization of deep neural networks
Researchers have developed a new method to predict how well deep neural networks will generalize without needing separate validation data. This approach uses the Fourier fractal dimension of the network's weight variations, analyzing learning trajectories in the frequency domain. The proposed measure shows a strong correlation with the actual generalization gap and outperforms existing methods on standard datasets like CIFAR-10 and MNIST. The work also introduces a Fourier-based optimizer to help regularize this fractal dimension during training. AI
IMPACT Provides a novel way to assess model performance without validation sets, potentially streamlining the ML development process.