New MLCC technique accelerates DNN training by up to 2.7x

Researchers have developed MLCC, a new congestion control technique designed to speed up deep neural network (DNN) training in shared GPU clusters. MLCC works by allowing DNN training flows to adjust their sending rates to interleave with other communication during compute periods. This method requires minimal code changes to existing protocols and has demonstrated significant improvements in job completion times, with average and 99th percentile training iteration times reduced by up to 1.9x and 2.7x, respectively. Simulations also indicate a 1.35x increase in training throughput on a large-scale fat-tree topology. AI

IMPACT Potential to significantly reduce training times and improve resource utilization in shared GPU environments.

RANK_REASON Research paper detailing a new technique for accelerating ML training. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• DagsHub
• deep neural network
• GPU
• Hugging Face
• Sanjoli Narang

• infra
• paper

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →