CISPA
Browse
- No file added yet -

ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training

Download (3.22 MB)
Version 2 2023-12-14, 12:32
Version 1 2023-11-29, 18:21
conference contribution
posted on 2023-12-14, 12:32 authored by Hui-Po Wang, Yang He, Mario FritzMario Fritz, Sebastian StichSebastian Stich
Federated learning is a powerful distributed learning scheme that allows numerous edge devices to collaboratively train a model without sharing their data. However, training is resource-intensive for edge devices, and limited network bandwidth is often the main bottleneck. Prior work often overcomes the constraints by condensing the models or messages into compact formats, e.g., by gradient compression or distillation. In contrast, we propose ProgFed, the first progressive training framework for efficient and effective federated learning. It inherently reduces computation and two-way communication costs while maintaining the strong performance of the final models. We theoretically prove that ProgFed converges at the same asymptotic rate as standard training on full models. Extensive results on a broad range of architectures, including CNNs (VGG, ResNet, ConvNets) and U-nets, and diverse tasks from simple classification to medical image segmentation show that our highly effective training approach saves up to 20 computation and up to 63 communication costs for converged models. As our approach is also complimentary to prior work on compression, we can achieve a wide range of trade-offs by combining these techniques, showing reduced communication of up to 50× at only 0.1 loss in utility. Code is available at https://github.com/a514514772/ProgFed.

History

Preferred Citation

Hui-Po Wang, Sebastian Stich, Yang He and Mario Fritz. ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training. In: International Conference on Machine Learning (ICML). 2022.

Primary Research Area

  • Trustworthy Information Processing

Name of Conference

International Conference on Machine Learning (ICML)

Legacy Posted Date

2022-07-22

Open Access Type

  • Gold

BibTeX

@inproceedings{cispa_all_3734, title = "ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training", author = "Wang, Hui-Po and Stich, Sebastian U. and He, Yang and Fritz, Mario", booktitle="{International Conference on Machine Learning (ICML)}", year="2022", }

Usage metrics

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC