New scaling laws detail mini-batch impact on linear regression

Researchers have developed new scaling laws for sketched linear regression that specifically address the impact of mini-batching. Their analysis covers one-pass batch SGD, multi-pass batch SGD with replacement, and multi-pass batch SGD without replacement. The findings reveal how mini-batching affects bias and variance terms, offering a theoretical framework that places batch size alongside compute, data, and model dimension in regression analysis. AI

IMPACT Provides a theoretical framework for understanding mini-batching in linear regression, potentially informing future algorithm design.

RANK_REASON Academic paper detailing theoretical advancements in machine learning algorithms. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• SGD

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →