Pfedeff: an Efficient and Personalized Federated Cognitive Learning Framework in Multi-agent Systems

Hongjian Shi,Jianqing Zhang,Shuming Fan,Ruhui Ma,Haibing Guan
DOI: https://doi.org/10.1109/tcds.2023.3288985
IF: 4.546
2023-01-01
IEEE Transactions on Cognitive and Developmental Systems
Abstract:With the increase in data volume and environment complexity, real-world problems require more advanced algorithms to acquire useful information for further analysis or decision making. Cognitive learning (CL) effectively handles incomplete information, and multiagent systems can provide enough data for analysis. Inspired by distributed machine learning, federated learning (FL) has become an efficient framework for implementing CL algorithms in multiagent systems while preserving user privacy. However, traditional communication optimizations on the FL framework suffer from either large communication volumes or large accuracy loss. In this article, we propose pFedEff, a personalized FL framework with efficient communication that can reduce communication volume and preserve training accuracy. pFedEff uses two magnitude masks, two importance masks, and a personalized aggregation method to reduce the model and update size while maintaining the training accuracy. Specifically, we use a pretraining magnitude mask for approximated regularization to reduce the magnitude of ineffective parameters during training. We also use a post-training magnitude mask to eliminate the low-magnitude parameters after training. Furthermore, we use uploading and downloading importance masks to reduce the communication volume in both upload and download streams. Our experimental results show that pFedEff reduces up to 94% communication volume with only a 1% accuracy loss over other state-of-the-art FL algorithms. In addition, we conduct multiple ablation studies to evaluate the influence of hyperparameters in pFedEff, which shows the flexibility of pFedEff and its applicability in different scenarios.
What problem does this paper attempt to address?