Structured Features Overfit in ML Models, New Research Shows

Researchers have demonstrated that structured feature maps in over-parameterized ridge regression can lead to overfitting, contrasting with unstructured random feature maps that generalize. Specifically, a band-limited Fourier feature map over $\mathbb{Z}_p^2$ showed degraded accuracy as the band increased, even below the interpolation threshold. The study found that the number of active modes, rather than the capacity ratio, significantly impacts performance, highlighting the importance of feature geometry in generalization. AI

IMPACT Highlights the critical role of feature geometry in model generalization, potentially guiding future model design.

RANK_REASON This is a research paper detailing theoretical findings in machine learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• alphaXiv
• CatalyzeX
• DagsHub
• Fourier
• Gotit.pub
• Hugging Face
• IArxiv
• Influence Flower
• International Conference on Machine Learning
• Safran
• ScienceCast
• Vardi
• Xu
• Z_p^2

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →