Abstract:Medical image reports are integral to clinical decision-making and patient management. Despite their importance, the confidentiality and private nature of medical data pose significant issues for the sharing and analysis of medical image data. This paper addresses these concerns by introducing a multimodal federated learning-based methodology for medical image reporting. This methodology harnesses distributed computing for co-training models across various medical institutions. Under the federated learning framework, every medical institution is capable of training the model locally and aggregating the updated model parameters to curate a top-tier medical image report model. Initially, we advocate for an architecture facilitating multimodal federated learning, including model creation, parameter consolidation, and algorithm enhancement steps. In the model selection phase, we introduce a deep learning-based strategy that utilizes multimodal data for training to produce medical image reports. In the parameter aggregation phase, the federal average algorithm is applied to amalgamate model parameters trained by each institution, which leads to a comprehensive global model. In addition, we introduce an evidence-based optimization algorithm built upon the federal average algorithm. The efficacy of the proposed architecture and scheme is showcased through a series of experiments. Our experimental results validate the proficiency of the proposed multimodal federated learning approach in generating medical image reports. Compared to conventional centralized learning methods, our proposal not only enhances the protection of patient confidentiality but also enriches the accuracy and overall quality of medical image reports. Through this research, we offer a novel solution for the privacy issues linked with the sharing and analyzing of medical data. Expected to assume a crucial role in medical image report generation and other medical applications, the multimodal federated learning method is set to deliver more precise, efficient, and privacy-secured medical services for healthcare professionals and patients.

Medical Federated Model with Mixture of Personalized and Sharing Components

Medical Federated Model with Mixture of Personalized and Shared Components

A Federated Learning Framework Via Decentralized Data Valuation for Chronic Disease Healthcare

PedFed: A performance evaluation-driven federated learning framework for efficient communication

Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data

Medical report generation based on multimodal federated learning

FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learning

MH-pFLGB: Model Heterogeneous personalized Federated Learning via Global Bypass for Medical Image Analysis

Federated Distillation for Medical Image Classification: Towards Trustworthy Computer-Aided Diagnosis

Multi-task Federated Learning Medical Analysis Algorithm Integrated Into Adapter

Towards Efficient Federated Learning-Based Scheme in Medicalcyber-Physicalsystems for Distributed Data

FedFC: an Efficient Personalized Federated Learning Method on Non-iid Data

Improving the Classification Effect of Clinical Images of Diseases for Multi-Source Privacy Protection

A federated learning system with data fusion for healthcare using multi-party computation and additive secret sharing

Privacy-Preserving Heterogeneous Federated Learning for Sensitive Healthcare Data

Stochastic Channel-Based Federated Learning for Medical Data Privacy Preserving

Federated Learning for Medical Image Analysis: A Survey

A Comprehensive Privacy-Preserving Federated Learning Scheme with Secure Authentication and Aggregation for Internet of Medical Things

Personalized Retrogress-Resilient Framework for Real-World Medical Federated Learning

Channel Adaptive and Sparsity Personalized Federated Learning for Privacy Protection in Smart Healthcare Systems

Personalized Federated Learning with Adaptive Batchnorm for Healthcare