No Coin Left Behind: Maximizing Strategic Surplus Against No-Regret Dynamics
This paper investigates the strategic surplus achievable against a Follow-the-Regularized-Leader (FTRL) learning algorithm in two-player zero-sum games. The research demonstrates that the extraction of this surplus is an inherent characteristic of the FTRL family, scaling with the number of suboptimal actions taken by the learner. The analysis reveals a dichotomy based on regularizer steepness, where non-steep regularizers allow for maximal transient surplus through finite-time elimination of suboptimal actions, while steep regularizers introduce a delay in surplus saturation. AI
IMPACT Provides theoretical insights into the behavior of learning algorithms in game theory, potentially influencing the design of future AI agents.