Near-Optimal Decentralized Stochastic Convex Optimization over Networks
Researchers have developed a new decentralized stochastic convex optimization method for networks. This method allows for a significantly larger number of workers to be used under a fixed gradient sample budget while maintaining optimal statistical rates. The approach interleaves minibatching with an accelerated gossip scheme to control disagreement and is proven to be optimal up to logarithmic factors for linear-span decentralized first-order methods. AI