One-for-All Model Initialization with Frequency-Domain Knowledge
Researchers have developed a new framework called FRONT that leverages frequency-domain knowledge for more efficient model initialization. This method isolates a model's foundational knowledge, termed "learngene," from the low-frequency components of its weights. The learngene can then be adapted to initialize models of any size without retraining, significantly accelerating convergence and reducing computational costs. AI
IMPACT Enables faster and more efficient model training by reusing foundational knowledge across different model sizes.
