Section: New Results
An Accelerated Decentralized Stochastic Proximal Algorithm for Finite Sums.
Modern large-scale finite-sum optimization relies on two key aspects: distribution
and stochastic updates. For smooth and strongly convex problems, existing
decentralized algorithms are slower than modern accelerated variance-reduced
stochastic algorithms when run on a single machine, and are therefore not efficient.
Centralized algorithms are fast, but their scaling is limited by global aggregation
steps that result in communication bottlenecks. In this work, we propose an
efficient Accelerated Decentralized stochastic algorithm for Finite Sums named
ADFS, which uses local stochastic proximal updates and randomized pairwise
communications between nodes. On n machines, ADFS learns from