backpropagation
PulseAugur coverage of backpropagation — every cluster mentioning backpropagation across labs, papers, and developer communities, ranked by signal.
8 day(s) with sentiment data
-
How Large Language Models Work: Prediction, Tokens, Training, and Attention
Large language models (LLMs) fundamentally operate by predicting the next word in a sequence, a process that implicitly teaches them grammar, facts, and reasoning. Before prediction, text is broken into tokens and conve…
-
New protocol reveals silent failures in deep learning feedback alignment methods
Researchers have identified significant limitations in the standard evaluation methods for feedback alignment (FA) techniques in deep learning. Current assessments rely on task accuracy and gradient cosine similarity, b…
-
AI researcher codes biologically plausible network training algorithm
A user on Reddit shared their experience coding a biologically plausible network training algorithm inspired by Nobel laureate Geoffrey Hinton's work. This exploration delved into research papers that propose alternativ…
-
New paper proposes biologically inspired neuron model for efficient online learning
A new paper introduces a novel mechanistic model for multilayer neuronal networks that draws inspiration from biological computation. This model offers a practical alternative to traditional backpropagation, enabling ef…
-
Predictive coding shows higher sample efficiency than backpropagation
Researchers have developed a new metric called "target alignment" to theoretically understand why predictive coding (PC) is more sample-efficient than backpropagation (BP) in neural networks. Their analysis, particularl…
-
Feedback Alignment training method improved with new dimensionality techniques
Researchers have identified a key limitation in Feedback Alignment (FA), a method for training neural networks that bypasses the biological implausibility of backpropagation. They found that FA's error signals have a lo…
-
New ePC method accelerates neural network training
Researchers have developed a new method called error-based Predictive Coding (ePC) that significantly speeds up neural network training on digital hardware. Traditional Predictive Coding (PC) methods suffer from signal …
-
Forward-Forward learning falls short of backpropagation on real-world tasks
A new research paper challenges the scalability of the Forward-Forward (FF) learning algorithm, a layer-local training method proposed by Geoffrey Hinton. The study introduces a new instrument, DTG-FF, which sets a new …
-
New training method matches backpropagation with local updates
Researchers have developed a new training method called Augmented Lagrangian Predictive Coding (PC-ALM) that aims to bridge the gap between local learning and backpropagation in deep neural networks. PC-ALM maintains th…
-
Backpropagation degrades neural network brain alignment within one epoch
A new research paper reveals that standard supervised training methods, particularly backpropagation, can rapidly degrade the alignment of artificial neural networks with the early visual cortex of the human brain. This…
-
AI learning mechanism diverges from human brain processing
A new research paper explores the differences between how artificial neural networks learn and how the human brain processes visual information. While both deep learning models and the brain show similarities in represe…
-
Predictive Coding Networks match Backpropagation in theory
Researchers have theoretically analyzed the infinite width and depth limits of Predictive Coding Networks (PCNs), an alternative to standard backpropagation. Their findings indicate that for linear residual networks, PC…
-
AI learning rules align with early primate vision, diverge in higher areas
Researchers have published a study comparing how different learning rules in artificial neural networks align with visual processing in both humans and macaques. The study found that early visual cortex alignment was co…
-
New predictive coding method matches backpropagation speed
Researchers have developed a new method for predictive coding networks that addresses their historical limitations in speed and performance with increasing depth. By treating these networks as deep hierarchical Gaussian…
-
Brain-inspired FRE-RNN makes Equilibrium Propagation more practical for AI
Researchers have developed a new recurrent neural network architecture, the Feedback-regulated REsidual recurrent neural network (FRE-RNN), designed to improve the practicality of Equilibrium Propagation (EP) for brain-…
-
Mono-Forward algorithm offers local learning alternative to backpropagation
Researchers have introduced Mono-Forward (MF), a new algorithm designed to improve upon the Forward-Forward (FF) method for training deep neural networks. MF maintains the local learning and reduced memory footprint of …
-
Untrained CNNs match human visual cortex at V1, research finds
A new study published on arXiv investigates how different learning rules in neural networks compare to human brain activity in visual processing. Researchers found that for early visual areas like V1 and V2, the network…