New method uses idle inference resources to cut LLM training costs

Researchers have developed a novel method to reduce the computational cost of training large language models by utilizing idle inference resources. The algorithm predicts gradients using a reduced-precision, reverse-mode program and combines these predictions with a few exact gradients via a control variate. This approach aims to mitigate approximation errors by managing variance rather than bias. Experiments on models ranging from 10 million to 774 million parameters have shown potential cost reductions under specific conditions where fleet work is sufficiently inexpensive, though performance varies across different model sizes and training scenarios. AI

IMPACT Could significantly reduce the cost of training large language models by leveraging existing inference hardware.

RANK_REASON Academic paper detailing a new method for LLM training cost reduction. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• alphaXiv
• arXiv
• CatalyzeX
• Connected Papers
• CORE Recommender
• DagsHub
• Gotit.pub
• Hugging Face
• IArxiv Recommender
• Influence Flower
• Litmaps
• Nicolò Felicioni
• ScienceCast
• scite Smart Citations

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →