Predictive Coding Networks match Backpropagation in theory

Researchers have theoretically analyzed the infinite width and depth limits of Predictive Coding Networks (PCNs), an alternative to standard backpropagation. Their findings indicate that for linear residual networks, PCNs can achieve the same gradient computations as backpropagation under specific parameterizations. This convergence to backpropagation's loss function occurs when the model's width significantly exceeds its depth, suggesting a potential for local updates in brain-like network architectures. AI

IMPACT Provides theoretical grounding for alternative training methods, potentially influencing future neural network architectures.

RANK_REASON Academic paper detailing theoretical limits and convergence properties of a neural network training method. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• backpropagation
• Francesco Innocenti
• Predictive Coding Networks

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →