New Q-learning framework tackles average-reward reinforcement learning challenges

Researchers have developed a new framework for average-reward reinforcement learning that addresses challenges posed by non-mixing Markov chains. The proposed method uses a reward-agnostic m-step mixing policy to regularize the contraction and exploration properties of learned policies. The analysis provides finite-time guarantees for Q-learning algorithms, matching existing rates for one-step settings and deriving new rates for m-step mixing. The study also extends to Q-planning variants, demonstrating that planning acts as time-localized learning, with the planning horizon balancing boundary value bias against sample complexity. AI

IMPACT Introduces a novel theoretical framework for reinforcement learning, potentially improving agent performance in average-reward settings.

RANK_REASON Academic paper detailing a new theoretical framework and algorithms for reinforcement learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• alphaXiv
• CatalyzeX
• DagsHub
• Gotit.pub
• Hugging Face
• IArxiv
• Influence Flower
• Markov chain
• Q-learning
• Q-Planning
• ScienceCast

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →