Abstract:Addressing the privacy protection and data sharing issues in Chinese medical texts, this paper introduces a federated learning approach named FLCMC for Chinese medical text classification. The paper first discusses the data heterogeneity issue in federated language modeling. Then, it proposes two perturbed federated learning algorithms, FedPA and FedPAP, based on the self-attention mechanism. In these algorithms, the self-attention mechanism is incorporated within the model aggregation module, while a perturbation term, which measures the differences between the client and the server, is added to the local update module along with a customized PAdam optimizer. Secondly, to enable a fair comparison of algorithms' performance, existing federated algorithms are improved by integrating a customized Adam optimizer. Through experiments, this paper first conducts experimental analyses on hyperparameters, data heterogeneity, and validity on synthetic datasets, which proves that the proposed federated learning algorithm has significant advantages in classification performance and convergence stability when dealing with heterogeneous data. Then, the algorithm is applied to Chinese medical text datasets to verify its effectiveness on real datasets. The comparative analysis of algorithm performance and communication efficiency shows that the algorithm exhibits strong generalization ability on deep learning models for Chinese medical texts. As for the synthetic dataset, upon comparing with comparison algorithms FedAvg, FedProx, FedAtt, and their improved versions, the experimental results show that for data with general heterogeneity, both FedPA and FedPAP show significantly more accurate and stable convergence behavior. On the real Chinese medical dataset of doctor–patient conversations, IMCS-V2, with logistic regression and long short-term memory network as training models, the experiment results show that in comparison to the above three comparison algorithms and their improved versions, FedPA and FedPAP both possess the best accuracy performance and display significantly more stable and accurate convergence behavior, proving that the method in this paper has better classification effects for Chinese medical texts.

FACMIC: Federated Adaptative CLIP Model for Medical Image Classification

FedCL: Federated Contrastive Learning for Multi-center Medical Image Classification

ACFL: Communication-Efficient Adversarial Contrastive Federated Learning for Medical Image Segmentation

FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learning

FLCMC: Federated Learning Approach for Chinese Medicinal Text Classification

Unifying and Personalizing Weakly-supervised Federated Medical Image Segmentation via Adaptive Representation and Aggregation

Distributed contrastive learning for medical image segmentation

FedIIC: Towards Robust Federated Learning for Class-Imbalanced Medical Image Classification

Customized Federated Learning for Multi-Source Decentralized Medical Image Classification

FCA: Taming Long-tailed Federated Medical Image Classification by Classifier Anchoring

Federated Fusion Learning with Attention Mechanism for Multi-Client Medical Image Analysis

MH-pFLGB: Model Heterogeneous personalized Federated Learning via Global Bypass for Medical Image Analysis

Federated Distillation for Medical Image Classification: Towards Trustworthy Computer-Aided Diagnosis

FedCCE: A class-level contribution explainable federated learning based on comparable prototypes collaboration for multi-site medical image classification.

MedCLIP: Contrastive Learning from Unpaired Medical Images and Text

Contrastive Centroid Supervision Alleviates Domain Shift in Medical Image Classification

FedATA: Adaptive Attention Aggregation for Federated Self-supervised Medical Image Segmentation

CLIP in Medical Imaging: A Comprehensive Survey

CCVA-FL: Cross-Client Variations Adaptive Federated Learning for Medical Imaging

PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documents

Dense Contrastive-based Federated Learning for Dense Prediction Tasks on Medical Images