New Q-learning algorithm offers robustness against corrupted data

Researchers have developed BR-Async-Q, a novel algorithm designed to improve the robustness of Q-learning in environments with corrupted rewards and states. This epoch-based approach utilizes data batching to reduce variance and constructs robust estimates of the Bellman optimality operator. The algorithm achieves a high-probability error bound that closely matches standard Q-learning, with a minor additive term dependent on the corruption fraction. This work offers the first known robustness guarantee for asynchronous Q-learning under both reward and state corruption, and demonstrates minimax optimality when only rewards are corrupted. AI

IMPACT Introduces a more resilient Q-learning method for environments with adversarial data corruption.

RANK_REASON The cluster contains an academic paper detailing a new algorithm for reinforcement learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• BR-Async-Q
• Huber contamination model

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →