New Polar Express method accelerates matrix decomposition for deep learning

Researchers have developed a new GPU-friendly algorithm called Polar Express for computing matrix decompositions, which is crucial for the Muon optimizer used in training deep neural networks. This method optimizes for high throughput on GPUs and achieves rapid convergence by minimizing error in a worst-case sense. When integrated with the Muon optimizer, Polar Express demonstrated improved validation loss for a GPT-2 model trained on a large dataset, outperforming existing alternatives. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Introduces a more efficient GPU-based method for deep learning optimization, potentially speeding up training for models like GPT-2.

RANK_REASON Academic paper introducing a new numerical method for deep learning optimization. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• Polar Express
• Muon
• GPT-2
• FineWeb
• arXiv
• GPU
• bfloat16
• Chen & Chow
• Nakatsukasa & Freund

• paper
• infra