Abstract:Federated learning is an emerging research paradigm for enabling collaboratively training deep learning models without sharing patient data. However, the data from different institutions are usually heterogeneous across institutions, which may reduce the performance of models trained using federated learning. In this study, we propose a novel heterogeneity-aware federated learning method, SplitAVG, to overcome the performance drops from data heterogeneity in federated learning. Unlike previous federated methods that require complex heuristic training or hyper parameter tuning, our SplitAVG leverages the simple network split and feature map concatenation strategies to encourage the federated model training an unbiased estimator of the target data distribution. We compare SplitAVG with seven state-of-the-art federated learning methods, using centrally hosted training data as the baseline on a suite of both synthetic and real-world federated datasets. We find that the performance of models trained using all the comparison federated learning methods degraded significantly with the increasing degrees of data heterogeneity. In contrast, SplitAVG method achieves comparable results to the baseline method under all heterogeneous settings, that it achieves 96.2% of the accuracy and 110.4% of the mean absolute error obtained by the baseline in a diabetic retinopathy binary classification dataset and a bone age prediction dataset, respectively, on highly heterogeneous data partitions. We conclude that SplitAVG method can effectively overcome the performance drops from variability in data distributions across institutions. Experimental results also show that SplitAVG can be adapted to different base convolutional neural networks (CNNs) and generalized to various types of medical imaging tasks. The code is publicly available at https://github.com/zm17943/SplitAVG.

Split learning for health: Distributed deep learning without sharing raw patient data

Privacy-Preserving Collaborative Deep Learning with Unreliable Participants.

Split Learning for collaborative deep learning in healthcare

Split Learning for Distributed Collaborative Training of Deep Learning Models in Health Informatics

SplitNN-driven Vertical Partitioning

Distributed deep learning networks among institutions for medical imaging

Institutionally Distributed Deep Learning Networks

Split-U-Net: Preventing Data Leakage in Split Learning for Collaborative Multi-Modal Brain Tumor Segmentation

Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data

A novel decentralized federated learning approach to train on globally distributed, poor quality, and protected private medical data

Big Data Intelligence Using Distributed Deep Neural Networks

Splitfed learning without client-side synchronization: Analyzing client-side split network portion size to overall performance

Multi-Institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentation

SplitAVG: A Heterogeneity-Aware Federated Deep Learning Method for Medical Imaging

Spatio-Temporal Split Learning for Privacy-Preserving Medical Platforms: Case Studies with COVID-19 CT, X-Ray, and Cholesterol Data

Decentralized Proactive Model Offloading and Resource Allocation for Split and Federated Learning

Efficient federated learning for distributed neuroimaging data

Comparison of Privacy-Preserving Distributed Deep Learning Methods in Healthcare

FedSL: Federated Split Learning for Collaborative Healthcare Analytics on Resource-Constrained Wearable IoMT Devices

PrivyNet: A Flexible Framework for Privacy-Preserving Deep Neural Network Training