Deep Ensembles Show Linear Mode Connectivity Under Data Shifts

Researchers have investigated the phenomenon of linear mode connectivity (LMC) in deep learning, particularly how it is affected by data shifts in ensembles of image classifiers. The study suggests that data shifts can be treated as a form of stochastic gradient noise, which can be mitigated by using smaller learning rates and larger batch sizes. These parameters influence whether models converge to similar or varied regions of the loss landscape, impacting the trade-off between training efficiency and ensemble diversity. AI

IMPACT Provides insights into training stability and generalization for deep learning models, potentially improving ensemble methods.

RANK_REASON This is a research paper published on arXiv detailing experimental findings on deep learning phenomena. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• Carolyna Hepburn
• Deep Ensembles
• Hugging Face
• image-classifiers
• Institute of Electrical and Electronics Engineers
• linear mode connectivity

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →