On the Relationship Between CoCoA and ADMM for Distributed Empirical Risk Minimization
A new research paper explores the relationship between two families of distributed optimization algorithms, CoCoA and ADMM. By unifying them through a primal-dual perspective, the study reveals that certain ADMM variants can perform comparably to or better than CoCoA for ridge-regularized empirical risk minimization problems. The unified view also provides a new primal-dual gap stopping criterion for consensus ADMM and a consistent convergence analysis for ADMM-type methods. AI
IMPACT Provides a unified theoretical framework for distributed optimization, potentially improving efficiency in training large-scale machine learning models.