New research analyzes convergence of semi-gradient TD learning in neural networks

Researchers have published a paper on arXiv detailing the convergence properties of semi-gradient temporal-difference (TD) learning, a fundamental algorithm in reinforcement learning. The study focuses on two-layer neural networks within a mean-field regime, identifying a stability mechanism where entropy-induced relaxation overcomes the sensitivity of the moving bootstrap target. This analysis proves exponential convergence to a unique fixed point and quantifies its bias relative to the unregularized Bellman solution, with implications for entropy-regularized off-policy control via soft Q-learning. AI

IMPACT Provides theoretical insights into the stability and convergence of core reinforcement learning algorithms, potentially informing future algorithm design.

RANK_REASON The cluster contains a single academic paper published on arXiv. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Bellman solution
• Hugging Face
• mean-field regime
• reinforcement learning
• Semi-gradient TD
• Soft Q-learning
• two-layer neural networks

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →