New Weight Feedback Method Enhances Local Updates in Deep Networks

Researchers have developed a new method called Weight Feedback with Activation-based Predictive Coding (WF-Act-PC) that allows for more localized weight updates in deep neural networks. This approach aims to overcome the reliance on non-local operations, specifically the Jacobian transpose, which is a bottleneck in traditional predictive coding methods. By factoring the Jacobian transpose into locally available terms, WF-Act-PC removes the need for a full autograd backward pass for error transport. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets show that WF-Act-PC improves accuracy with network depth and matches or exceeds the performance of backpropagation on certain architectures. AI

IMPACT This research could lead to more efficient and biologically plausible training methods for deep neural networks.

RANK_REASON The cluster contains a research paper detailing a new method for deep learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• backpropagation
• CIFAR-10
• CIFAR-100
• Jacobian transpose
• Predictive Coding
• ResNet-18
• Tiny-ImageNet
• VGG-9
• Weight Feedback Computes the Jacobian Transpose Locally in Modern Deep Networks
• WF-Act-PC

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →