A user has successfully extended the Gemma 4.31B model to 40.5B parameters by adding new layers, overcoming an initial failure where the new layers did not learn. The key to this success was initializing the new layers to perform a blend of the tasks of adjacent layers, allowing them to receive feedback and learn from the outset. The user also found that strategic placement of these new layers and maintaining original volume settings were crucial for the model's performance. AI
IMPACT Demonstrates a novel method for extending existing fine-tuned models, potentially enabling greater customization and capacity.
RANK_REASON User-driven modification and extension of an existing open-source model. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
