Tencent has unveiled its Hy4 AI model, a 220-billion-parameter system that utilizes a sparse expert architecture. This model incorporates learned n-gram memory and long-context attention mechanisms. Notably, despite its large total parameter count, Hy4 efficiently operates by activating only 12 billion parameters at a time. AI
IMPACT This model's efficient parameter usage could influence future large model architectures and training strategies.
RANK_REASON Frontier-lab model release with system card. [lever_c_demoted from frontier_release: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →