New FRONT framework enables training-free model initialization

Researchers have developed a new framework called FRONT that leverages frequency-domain knowledge for more efficient model initialization. This method isolates a model's foundational knowledge, termed "learngene," from the low-frequency components of its weights. The learngene can then be adapted to initialize models of any size without retraining, significantly accelerating convergence and reducing computational costs. AI

IMPACT Enables faster and more efficient model training by reusing foundational knowledge across different model sizes.

RANK_REASON This is a research paper detailing a new method for model initialization. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• Discrete Cosine Transform
• Jianlu Shen

• paper
• model release
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →