Completion vs Optimality: Policy Gradient in Long-Horizon Cumulative-Damage Problems
Researchers have explored policy gradient methods for long-horizon decision problems where immediate rewards can lead to significant future negative consequences. They identified two distinct failure modes: completion, which is reaching the end of the decision horizon, and optimality, which is making the best possible decisions given that the horizon is reached. The study proposes a method to separate these two issues and tested it on simulated scenarios like a bricklayer's career and an NBA player's career, finding that their approach improved performance. AI
IMPACT This research offers a framework for understanding and improving AI decision-making in complex, long-term scenarios.