Eco-Fedsplit: Federated Learning with Error-Compensated Compression
- Resource Type
- Conference
- Authors
- Khirirat, Sarit; Magnusson, Sindri; Johansson, Mikael
- Source
- ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) Acoustics, Speech and Signal Processing (ICASSP), ICASSP 2022 - 2022 IEEE International Conference on. :5952-5956 May, 2022
- Subject
- Bioengineering
Communication, Networking and Broadcast Technologies
Computing and Processing
Signal Processing and Analysis
Signal processing algorithms
Machine learning
Signal processing
Collaborative work
Approximation algorithms
Convex functions
Speech processing
Optimization methods
operator splitting schemes
quantization
federated learning
- Language
- ISSN
- 2379-190X
Federated learning is an emerging framework for collaborative machine-learning on devices which do not want to share local data. State-of-the art methods in federated learning reduce the communication frequency, but are not guaranteed to converge to the optimal model parameters. These methods also experience a communication bottleneck, especially when the devices are power-constrained and communicate over a shared medium. This paper presents ECO-FedSplit, an algorithm that increases the communication efficiency of federated learning without sacrificing solution accuracy. The key is to compress inter-device communication and to compensate for information losses in a theoretically justified manner. We prove strong convergence properties of ECO-FedSplit on strongly convex optimization problems and show that the algorithm yields a highly accurate solution with dramatically reduced communication. Extensive numerical experiments validate our theoretical result on real data sets.