Low-rank surrogate modeling and stochastic zero-order optimization for training of neural networks with black-box layers
Researchers have developed a new framework for training hybrid neural networks that combine digital components with physical, black-box layers. This approach uses stochastic zero-order optimization and a dynamic low-rank surrogate model to enable gradient propagation through non-differentiable physical devices. The method has demonstrated effectiveness across computer vision, audio classification, and language modeling tasks, achieving accuracy comparable to purely digital baselines. AI
IMPACT Offers a practical pathway for integrating non-differentiable physical components into scalable, end-to-end trainable AI systems.