New statistical framework explains deep neural network quality prediction

Researchers have developed a statistical framework to explain why analyzing the weight matrices of deep neural networks can predict their quality. This work derives the marginal likelihood for a DNN using only the final layer weights, demonstrating its predictive power across large-scale convolutional DNNs and LLMs. The study further investigates the role of heavy-tailed spectra, specifically the High-Temperature Marchenko-Pastur distribution, showing how an optimal heaviness parameter can empirically predict overfitting in large-scale DNNs. AI

IMPACT Provides a theoretical basis for understanding and predicting the performance of deep neural networks, potentially leading to more efficient model development.

RANK_REASON The item is an academic paper published on arXiv detailing a new statistical framework for predicting model quality. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• alphaXiv
• arXiv
• CatalyzeX Code Finder for Papers
• DagsHub
• Gotit.pub
• High-Temperature Marchenko-Pastur
• Hugging Face
• Influence Flower
• ScienceCast

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →