Inheritance Between Feedforward and Convolutional Networks via Model Projection
Researchers have introduced a method called model projection to enable effective transfer learning between different neural network architectures. This technique allows properties from generalized convolutional networks (GCNNs) to be inherited by generalized feedforward networks (GFFNs). By freezing convolutional sub-functions and learning scalar coefficients, model projection creates a trainable structure similar to GFFNs, facilitating parameter-efficient transfer learning. Experiments on ImageNet-pretrained models show this approach is competitive with existing transfer learning methods and serves as a strong initialization for fine-tuning on downstream tasks. AI
IMPACT Enables more efficient transfer learning between different neural network types, potentially speeding up model development and reducing computational costs.