Abstract:With the rapid development of science and technology, machine learning and deep learning technology have more and more applications in our daily life. At the same time, people pay more and more attention to the protection of their own privacy. And in recent years, various countries have introduced a series of laws and regulations to protect people's data privacy. In this case, the traditional method of pooling the data of each user for model training is no longer applicable, and the data of each user can only be saved in the hands of the user himself, which has a profound impact on the development of artificial intelligence technology. To solve this dilemma, the industry has proposed the concept of federated learning. Due to the data imbalance problem between each user, the prediction ability of the global aggregation model obtained by federated learning on the local specific data of each user needs to be improved. Therefore, personalization is an issue that federated learning needs to pay attention to.In order to improve the personalization ability, this paper proposes a federated transfer learning algorithm with attention mechanism. After each user obtains the global aggregation model of federated learning, the attention module is added to the local model, and then the parameters of the low-level neurons are frozen during training, and only the parameters of the high-level neurons and the attention module are updated. Finally, each user obtains a unique model that is more suitable for local data. In this paper, we conduct experiments to analyze the performance of this algorithm and the federated transfer algorithm and federated learning algorithm in terms of accuracy. The experimental results show that on the convolutional neural network model, the federated transfer learning algorithm applying the attention mechanism has improved the accuracy compared with the federated transfer algorithm and the federated learning algorithm.

Personalized Federated Learning with Local Attention

ChannelFed: Enabling Personalized Federated Learning via Localized Channel Attention

ChannelFed: Enabling Personalized Federated Learning via Localized Channel Attention Weights

Personalized Federated Learning with Attention-based Client Selection

Personalized federated learning based on multi-head attention algorithm

A personalized federated learning method based on the residual multi-head attention mechanism

Bilateral Improvement in Local Personalization and Global Generalization in Federated Learning.

Personalized federated learning based on feature fusion

Personalized Federated Learning on Long-Tailed Data Via Adversarial Feature Augmentation.

Personalized Federated Learning with Local Optimization Models

FedALA: Adaptive Local Aggregation for Personalized Federated Learning

FedPer++: Toward Improved Personalized Federated Learning on Heterogeneous and Imbalanced Data

Personalized Federated Learning with Attention Mechanisms

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

Energizing Federated Learning via Filter-Aware Attention

Pfedlhns: Personalized Federated Learning Via Local Hypernetworks

FedL2P: Federated Learning to Personalize

FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Data

FedMCSA: Personalized federated learning via model components self-attention

FedCR: Personalized Federated Learning Based on Across-Client Common Representation with Conditional Mutual Information Regularization.

Improving Global Generalization and Local Personalization for Federated Learning