Researchers have observed that deep reinforcement learning agents, when using frozen, randomly initialized Convolutional Neural Network (CNN) feature extractors, spontaneously develop highly sparse fully-connected representations. This phenomenon occurs without any explicit sparsity-inducing objective in the training process. The study found that the number of active neurons in the first fully-connected layer scales with task complexity, with simpler tasks like deterministic Pong requiring only 1-3 neurons out of 64, while more complex games like Space Invaders utilized around 42 neurons. Furthermore, the research indicated that this emergent sparsity is crucial for performance, as removing these active neurons significantly degrades agent capabilities across different implementations and games. AI
IMPACT Suggests a potential pathway to achieving efficient representations in RL without explicit sparsity objectives.
RANK_REASON Academic paper detailing a novel emergent phenomenon in deep reinforcement learning. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
