High-dimensional ridge regression with random features for non-identically distributed data with a variance profile
Two recent arXiv preprints explore high-dimensional ridge regression for non-identically distributed data, moving beyond standard assumptions of independent and identically distributed samples. The papers introduce variance profile models to analyze the predictive risk of ridge estimators, particularly focusing on the double descent phenomenon. Researchers used tools from random matrix theory and operator-valued free probability to derive asymptotic equivalents for risk and degrees of freedom, with numerical experiments validating their findings and highlighting how heterogeneous variance profiles can alter generalization behavior. AI
IMPACT These papers advance theoretical understanding of regression models, potentially informing future AI development by clarifying generalization properties under non-standard data distributions.