The Algorithm Is Not the Behavior: Learned Priors Override Look-Ahead in a Chess-Playing Neural Network
Researchers have discovered that a sophisticated neural network, Leela Chess Zero, can internally compute correct solutions to chess puzzles but ultimately override them in favor of safer, less aggressive moves. This phenomenon, termed "forgotten puzzles," demonstrates that the presence of an algorithm within a neural network does not guarantee its behavioral output. The study found that while the network's look-ahead capabilities correctly identified optimal moves, later layers prioritized defensive strategies, leading to the incorrect final output. By intervening to counteract this preference, researchers were able to recover a significant percentage of these "forgotten puzzles." AI
IMPACT Reveals a potential disconnect between an AI's internal reasoning and its final output, impacting trust and interpretability in complex decision-making systems.