Huawei has released the training code for its openPangu-2.0 model, which is designed for its Ascend-native Mixture-of-Experts (MoE) stack. This release includes code for pre-training, supervised fine-tuning (SFT), and reinforcement learning (RL) from human feedback. The open-sourced code complements previous releases of the model's weights, specifically the Pro (505B) and Flash (92B) versions. AI
IMPACT This release provides researchers and developers with the tools to train and fine-tune large-scale MoE models on Huawei's Ascend hardware, potentially fostering innovation in AI development.
RANK_REASON Open-source release of a large language model's training code. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →