Synthetic Benchmarks Overstate Forward-Forward Scaling: Real-Data Limits of Layer-Local Training
A new research paper challenges the scalability of the Forward-Forward (FF) learning algorithm, a layer-local training method proposed by Geoffrey Hinton. The study introduces a new instrument, DTG-FF, which sets a new state-of-the-art for FF on several real-world benchmarks, including ImageNet-100. However, the research demonstrates that FF significantly underperforms standard backpropagation (BP) on larger datasets and higher class counts, indicating a real-world performance ceiling. Furthermore, the paper debunks the memory efficiency argument for FF at scale, showing that BP is more efficient on commodity hardware. AI
IMPACT Demonstrates that layer-local training methods like Forward-Forward have significant limitations on real-world data compared to backpropagation.