PURe: A Plug-and-Play Product-Unit Residual Module for Vision Networks
Researchers have introduced PURe, a novel module designed to enhance vision networks by incorporating multiplicative local interactions. This module, built around a 2D Product Unit with a log-domain formulation, addresses optimization instability issues that have previously limited the use of product units in deep architectures. PURe can be seamlessly integrated as a replacement for existing residual units, demonstrating improved performance and a better accuracy-parameter trade-off on datasets like ImageNet and CIFAR-10, and also showing benefits in CT segmentation tasks. AI
IMPACT Introduces a new module for vision networks that improves accuracy-parameter trade-offs and enables multiplicative interactions.