Neural tangent kernel
PulseAugur coverage of Neural tangent kernel — every cluster mentioning Neural tangent kernel across labs, papers, and developer communities, ranked by signal.
- 2026-05-13 research_milestone Publication of a paper introducing a force-aware Neural Tangent Kernel for active learning of MLIPs. 来源
3 天有情绪数据
-
New optimization technique boosts accuracy for complex physics neural networks
Researchers have developed a new optimization technique called SOAP+GN to improve the accuracy of physics-informed neural networks (PINNs) when dealing with complex, coupled multiphysics systems. This method addresses a…
-
New method enhances neural network uncertainty estimation
Researchers have developed a new method to improve Bayesian Last Layers (BLLs) for estimating uncertainty in neural networks. Their approach leverages a projection of Neural Tangent Kernel (NTK) features to account for …
-
GLU structures accelerate LLM optimization by reshaping NTK spectrum
Researchers have investigated why Gated Linear Units (GLU) are superior to non-GLU structures in large language models. Their analysis in the neural tangent kernel regime indicates that GLU reshapes the NTK spectrum, re…
-
New Math Framework Explains Transformer Training Dynamics
A new paper introduces a mathematical framework for understanding how Transformers train, particularly in the mean-field regime where both depth and width approach infinity. Unlike ResNets which can be modeled by ODEs, …
-
New framework enables scalable, robust active learning for MLIPs
Researchers have developed a new active learning framework for machine-learning interatomic potentials (MLIPs) that addresses scalability and robustness challenges. This framework utilizes a force-aware Neural Tangent K…
-
Paper explores preconditioned gradient descent's impact on neural network learning regimes
This paper investigates how preconditioned gradient descent (PGD) methods, like Gauss-Newton, influence spectral bias and the phenomenon of grokking in neural networks. Researchers propose that PGD can mitigate spectral…
-
New research explains why Zeroth-Order Optimization scales to LLMs
Two new papers explore zeroth-order (ZO) optimization for fine-tuning large language models (LLMs). The first paper introduces a kernel perspective, showing that the approximation error depends on output size rather tha…
-
New theories explore how pre-training and sparse connectivity enhance deep learning generalization
Three new papers explore the theoretical underpinnings of generalization in deep learning. One paper identifies pre-training as a critical factor for weak-to-strong generalization, demonstrating its emergence through a …