The Importance of Phase in Neural Representations: An Internal Oppenheim-Lim Test of Image Classifiers
A new research paper explores the role of phase in neural representations within image classifiers, drawing parallels to the Oppenheim-Lim test which demonstrated that natural images can be reconstructed from their Fourier phase alone. The study found that models like PRISM2D, GFNet, and ViT-B/16 rely heavily on phase information for identity, with magnitude playing a less critical role for readout. While ResNet-50 initially appeared to deviate, further analysis revealed a latent sign code exposed in different bases depending on the architecture's rectification and readout geometry. AI
IMPACT This research offers a deeper understanding of how image classifiers process information, potentially influencing future model architectures and interpretability methods.