Federated Distillation for Medical Image Classification: Towards Trustworthy Computer-Aided Diagnosis

Sufen Ren,Yule Hu,Shengchao Chen,Guanjun Wang

2024-07-03

Abstract:Medical image classification plays a crucial role in computer-aided clinical diagnosis. While deep learning techniques have significantly enhanced efficiency and reduced costs, the privacy-sensitive nature of medical imaging data complicates centralized storage and model training. Furthermore, low-resource healthcare organizations face challenges related to communication overhead and efficiency due to increasing data and model scales. This paper proposes a novel privacy-preserving medical image classification framework based on federated learning to address these issues, named FedMIC. The framework enables healthcare organizations to learn from both global and local knowledge, enhancing local representation of private data despite statistical heterogeneity. It provides customized models for organizations with diverse data distributions while minimizing communication overhead and improving efficiency without compromising performance. Our FedMIC enhances robustness and practical applicability under resource-constrained conditions. We demonstrate FedMIC's effectiveness using four public medical image datasets for classical medical image classification tasks.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper aims to address two main issues in medical image classification: 1. **Statistical Heterogeneity**: Due to significant differences in data distribution across different medical institutions, traditional federated learning methods perform poorly in handling medical image classification tasks. This non-independent and identically distributed (Non-IID) data leads to a decline in model performance. 2. **Communication Overhead**: Existing centralized training methods face network dependency and privacy issues in practical applications. Especially in resource-constrained environments, frequent data transmission not only increases communication burden but also potentially leaks sensitive information. To solve these problems, the paper proposes a new framework called FEDMIC. FEDMIC combines federated learning and personalized knowledge distillation techniques, enabling various medical institutions to collaboratively train without sharing raw data, while reducing communication costs and improving model robustness and practicality. Specifically: - **Dual Knowledge Distillation (Dual-KD)**: Each client maintains a teacher model and a student model. The teacher model is used for personalized learning, while the student model benefits from global knowledge. In this way, the local model can learn from both global and local knowledge, better adapting to different data distributions. - **Global Parameter Decomposition (GPD)**: To further reduce communication overhead, FEDMIC employs parameter decomposition techniques to compress local model parameters into low-rank matrices, thereby reducing the amount of data transmitted and improving efficiency. Through these methods, FEDMIC can achieve efficient and privacy-preserving medical image classification tasks under resource-constrained conditions.

Federated Distillation for Medical Image Classification: Towards Trustworthy Computer-Aided Diagnosis

Medical Federated Model with Mixture of Personalized and Sharing Components

Medical Federated Model with Mixture of Personalized and Shared Components

FedSLD: Federated Learning with Shared Label Distribution for Medical Image Classification

Federated Ophthalmic Disease Diagnosis Based on Weighted Averaging and Local Optimization

Customized Federated Learning for Multi-Source Decentralized Medical Image Classification

MH-pFLID: Model Heterogeneous Personalized Federated Learning Via Injection and Distillation for Medical Data Analysis

Federated Learning for Medical Image Analysis: A Survey

Improving the Classification Effect of Clinical Images of Diseases for Multi-Source Privacy Protection

Framework Construction of an Adversarial Federated Transfer Learning Classifier

Distributed Federated Learning-Based Deep Learning Model for Privacy MRI Brain Tumor Detection

Unifying and Personalizing Weakly-supervised Federated Medical Image Segmentation via Adaptive Representation and Aggregation

FedDP: Privacy-preserving method based on federated learning for histopathology image segmentation

Privacy Preserving Federated Learning in Medical Imaging with Uncertainty Estimation

FKD-Med: Privacy-Aware, Communication-Optimized Medical Image Segmentation via Federated Learning and Model Lightweighting Through Knowledge Distillation

Medical report generation based on multimodal federated learning

In-depth Analysis of Privacy Threats in Federated Learning for Medical Data

Dynamic-Fusion-Based Federated Learning for COVID-19 Detection

FedCL: Federated Contrastive Learning for Multi-center Medical Image Classification

Dynamic Bank Learning for Semi-supervised Federated Image Diagnosis with Class Imbalance

Vision Through the Veil: Differential Privacy in Federated Learning for Medical Image Classification