New Theory Explores Learning Plasticity Using Control Theory

A new paper, "A Thermodynamic Theory of Learning Part II: History-Dependent Reachability and Continual Learning," has been published on arXiv. This work extends previous research by formulating plasticity in machine learning as a problem of finite-horizon reachability under history-dependent dynamics, utilizing minimum-energy control theory. The paper introduces an extended state that includes parameters and internal variables influencing future updates, and proposes that the minimum energy required for a given displacement determines adaptation costs. It also explores the relationship between learning and retention, suggesting that costs can increase without a loss in rank and distinguishing between accessible and inaccessible directions for learning. AI

IMPACT Introduces a novel theoretical framework for understanding learning plasticity and continual learning, potentially influencing future model architectures and training methodologies.

RANK_REASON Publication of an academic paper on arXiv detailing a new theoretical framework for machine learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• A Thermodynamic Theory of Learning Part II: History-Dependent Reachability and Continual Learning
• Daisuke Okanohara
• Hugging Face
• minimum-energy control theory

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →