New statistical framework analyzes overfitting in high-dimensional regression

This paper delves into the statistical behavior of high-dimensional ridgeless least-squares estimators, examining how prediction risk is affected by sample size and feature dimension growth. It introduces a generalized spiked population covariance model to analyze the influence of latent factors and their eigenvalues on overfitting. The research highlights that the alignment between regression coefficients and the covariance matrix's eigenspaces is crucial for determining whether interpolation leads to benign, tempered, or catastrophic overfitting, establishing sharp prediction risk limits under minimal moment conditions. AI

IMPACT Provides a theoretical framework for understanding generalization in overparameterized models, relevant for developing more robust AI systems.

RANK_REASON Academic paper published on arXiv detailing statistical theory. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• arXiv
• High-dimensional ridgeless least squares interpolation under spiked covariance structures
• Hugging Face
• Statistics Theory

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →