Modal has partnered with Sakana AI to deploy Sakana Namazu, a trillion-parameter Mixture-of-Experts (MoE) model optimized for Japanese business contexts, web search, and code execution. The collaboration focused on overcoming the significant memory and quality challenges associated with quantizing such a large model. Modal implemented custom quantization techniques, including selective expert quantization and domain-representative calibration data, to maintain model quality while enabling deployment on NVFP4 precision for NVIDIA Blackwell GPUs. Additionally, a specialized speculative decoding model was trained on raw token streams from Sakana Namazu's traffic, resulting in improved token acceptance rates and reduced latency. AI
IMPACT Demonstrates advanced techniques for deploying massive LLMs efficiently, enabling specialized models for non-English markets.
RANK_REASON Deployment of a large-scale, specialized LLM by a cloud provider and AI lab. [lever_c_demoted from significant: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
