New Q-learning algorithms achieve fast rates with KL regularization

Researchers have developed new algorithms for offline Q-learning in finite-horizon Markov decision processes, incorporating Kullback-Leibler (KL) regularization. The proposed KL-regularized Q-Learning (KLQ) algorithm achieves a sample complexity of \(\widetilde{O}(1/\epsilon)\) without explicit pessimism, though with an exponential dependence on the regularization parameter and horizon. A subsequent algorithm, KL-regularized Pessimistic Q-Learning (KLPQ), maintains the same rate while removing this dependence, improving overall sample complexity. These findings represent the first fast-rate guarantees for offline Q-learning in tabular finite-horizon KL-regularized MDPs. AI

IMPACT Establishes new theoretical foundations for offline reinforcement learning, potentially improving efficiency in real-world sequential decision-making tasks.

RANK_REASON Academic paper detailing new algorithms and theoretical guarantees. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• KLPQ
• KL regularization
• Kullback--Leibler divergence
• Markov decision processes
• Q-learning

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →