New framework unifies bisimulation metrics for reinforcement learning

Researchers have developed a new framework for understanding bisimulation metrics in reinforcement learning, extending their application beyond discounted Markov decision processes (MDPs) to total- and average-reward settings. This approach reframes metric operators as Bellman-Shapley operators in zero-sum games, defining metrics as game value functions. This game-theoretic foundation allows for the transfer of dynamic programming theories and algorithms, offering unique definitions even when the discount factor is one. The research also characterizes metric finiteness based on MDP structure and introduces a general framework for modifying game action spaces to recover and inspire new metric constructions, providing error and suboptimality guarantees for planning. AI

IMPACT This research provides a theoretical foundation for more robust state similarity measures in reinforcement learning, potentially improving planning and decision-making in complex, undiscounted environments.

RANK_REASON The item is an academic paper on a theoretical advancement in reinforcement learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• alphaXiv
• arXivLabs
• Bellman-Shapley
• CatalyzeX
• DagsHub
• Gotit.pub
• Hugging Face
• Markov decision process
• ScienceCast

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →