New bounds established for ReLU NTK Gram matrices

Researchers have established tight worst-case bounds for the smallest eigenvalue of ReLU neural tangent kernel (NTK) Gram matrices. The study focuses on unit vectors in a d-dimensional space, averaging pairwise gated inner products over a standard Gaussian direction. The findings provide a universal, dimension-free lower bound of $\Omega(\Delta_\pm/\sqrt{\log n})$ for the smallest eigenvalue, where $\Delta_\pm$ represents the projective separation of the vectors. This bound is shown to be tight, with matching upper bounds constructed for worst-case scenarios. AI

IMPACT This research provides theoretical insights into the behavior of neural tangent kernels, potentially informing the design and understanding of deep learning models.

RANK_REASON The cluster contains a single academic paper detailing theoretical research findings in machine learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Gaussian function
• Hugging Face
• machine learning
• rectifier

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →