This paper delves into the stationary bias present in nonlinear two-timescale stochastic approximation methods. The research focuses on recursions driven by finite-state Markov chains, deriving a first-order bias expansion that remains uniform even when the slow step size is significantly smaller than the fast step size. The findings indicate that the bias reduction strategy requires careful matching of Richardson-Romberg extrapolation weights to the specific step-size path, as bias exponents may not always be integers. The study includes an exactly solvable nonlinear Markov example for verification and examines temporal-difference learning and finite-run extrapolation. AI
RANK_REASON The cluster contains a single academic paper published on arXiv. [lever_c_demoted from research: ic=1 ai=1.0]
Read on Hugging Face Daily Papers →
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →