Dealing With Heterogeneous 3D MR Knee Images: A Federated Few-Shot Learning Method With Dual Knowledge Distillation

Xiaoxiao He,Chaowei Tan,Bo Liu,Liping Si,Weiwu Yao,Liang Zhao,Di Liu,Qilong Zhangli,Qi Chang,Kang Li,Dimitris N. Metaxas

2023-04-18

Abstract:Federated Learning has gained popularity among medical institutions since it enables collaborative training between clients (e.g., hospitals) without aggregating data. However, due to the high cost associated with creating annotations, especially for large 3D image datasets, clinical institutions do not have enough supervised data for training locally. Thus, the performance of the collaborative model is subpar under limited supervision. On the other hand, large institutions have the resources to compile data repositories with high-resolution images and labels. Therefore, individual clients can utilize the knowledge acquired in the public data repositories to mitigate the shortage of private annotated images. In this paper, we propose a federated few-shot learning method with dual knowledge distillation. This method allows joint training with limited annotations across clients without jeopardizing privacy. The supervised learning of the proposed method extracts features from limited labeled data in each client, while the unsupervised data is used to distill both feature and response-based knowledge from a national data repository to further improve the accuracy of the collaborative model and reduce the communication cost. Extensive evaluations are conducted on 3D magnetic resonance knee images from a private clinical dataset. Our proposed method shows superior performance and less training time than other semi-supervised federated learning methods. Codes and additional visualization results are available at <a class="link-external link-https" href="https://github.com/hexiaoxiao-cs/fedml-knee" rel="external noopener nofollow">this https URL</a>.

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

The paper aims to address several key issues faced by Federated Learning (FL) in the medical field:

1. **Limited Labeled Data Issue**: Due to the scarcity and high cost of high-quality annotations in clinical settings, the labeled data available locally at each medical institution is often insufficient. This leads to poor performance of each client model during joint model training.

2. **Data Heterogeneity Issue**: There are significant differences in data between different medical institutions, including variations in resolution and imaging parameters. This heterogeneity makes it difficult to directly apply models from public data repositories to local datasets.

3. **Large Data Transmission Issue**: When using large public data repositories for federated learning, a large amount of data needs to be transmitted, which is a challenge for medical institutions in remote areas.

To address the above issues, the authors propose a federated few-shot learning method that combines Dual Knowledge Distillation. Specifically, this method extracts both response and feature knowledge from a public data repository on the server side (such as the Osteoarthritis Initiative, OAI) and transfers it to the client models, thereby improving segmentation accuracy on local datasets. Additionally, this method addresses the challenges posed by data heterogeneity and reduces data transmission volume through pre-trained models. Experimental results show that this method outperforms other semi-supervised federated learning methods in the task of 3D magnetic resonance knee image segmentation, not only improving segmentation accuracy but also reducing training time and communication costs.