Uncertainty quantification for Markov chain induced martingales with application to temporal difference learning
Researchers have developed new high-dimensional concentration inequalities and Berry-Esseen bounds for martingales induced by Markov chains. These findings are applied to analyze Temporal Difference (TD) learning with linear function approximations, a key method in Reinforcement Learning (RL). The study provides a strong consistency guarantee for TD learning and establishes an $O(T^{- rac{1}{4}}\log T)$ distributional convergence rate for the TD estimator. AI
IMPACT Advances theoretical understanding of RL algorithms, potentially leading to more robust and reliable AI agents.