Linear Mode Connectivity under Data Shifts for Deep Ensembles of Image Classifiers
Researchers have investigated the phenomenon of linear mode connectivity (LMC) in deep learning, particularly how it is affected by data shifts in ensembles of image classifiers. The study suggests that data shifts can be treated as a form of stochastic gradient noise, which can be mitigated by using smaller learning rates and larger batch sizes. These parameters influence whether models converge to similar or varied regions of the loss landscape, impacting the trade-off between training efficiency and ensemble diversity. AI
IMPACT Provides insights into training stability and generalization for deep learning models, potentially improving ensemble methods.