Skip to content
/ FCCL Public
forked from WenkeHuang/FCCL

CVPR2022 - Learn From Others and Be Yourself in Heterogeneous Federated Learning

Notifications You must be signed in to change notification settings

Jean0928/FCCL

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

21 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Learn from others and Be yourself in Heterogeneous Federated Learning

Learn from others and Be yourself in Heterogeneous Federated Learning,
Wenke Huang, Mang Ye, Bo Du CVPR, 2022 Link

Abstract

Federated learning has emerged as an important distributed learning paradigm, which normally involves collaborative updating with others and local updating on private data. However, heterogeneity problem and catastrophic forgetting bring distinctive challenges. First, due to non-i.i.d (identically and independently distributed) data and heterogeneous architectures, models suffer performance degradation on other domains and communication barrier with participants models. Second, in local updating, model is separately optimized on private data, which is prone to overfit current data distribution and forgets previously acquired knowledge, resulting in catastrophic forgetting. In this work, we propose FCCL (Federated Cross-Correlation and Continual Learning). For heterogeneity problem, FCCL leverages unlabeled public data for communication and construct cross-correlation matrix to learn a generalizable representation under domain shift. Meanwhile, for catastrophic forgetting, FCCL utilizes knowledge distillation in local updating, providing inter and intra domain information without leaking privacy. Empirical results on various image classification tasks demonstrate the effectiveness of our method and the efficiency of modules.

Citation

@inproceedings{FCCL_CVPR22,
    title={Learn from others and be yourself in heterogeneous federated learning},
    author={Huang, Wenke and Ye, Mang and Du, Bo},
    booktitle={CVPR},
    year={2022}
}

Relevant Projects

[1] Rethinking Federated Learning with Domain Shift: A Prototype View - CVPR 2023 [Link] [Code]

[2] Federated Graph Semantic and Structural Learning - IJCAI 2023 [Link][Code]

About

CVPR2022 - Learn From Others and Be Yourself in Heterogeneous Federated Learning

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 64.2%
  • Jupyter Notebook 32.7%
  • Shell 3.1%