Well-Posed KL-Regularized Control via Wasserstein and Kalman-Wasserstein KL Divergences
Researchers have introduced new divergences that act as analogs to Kullback-Leibler (KL) divergence, addressing its limitations in reinforcement learning, particularly when distributions do not match or in low-noise scenarios. These novel divergences, based on Wasserstein and Kalman-Wasserstein geometries, remain finite even as distributions degenerate. The study demonstrates their effectiveness in KL-regularized optimal control for linear systems with Gaussian noise, showing they prevent singularity and improve performance in examples like the double integrator and cart-pole. AI
IMPACT Introduces mathematical tools that could improve the stability and performance of reinforcement learning agents in complex control tasks.