In Reply to Gopal and Prochazka and to Youngclaus.

T. Shanafelt,D. Satele,L. Dyrbye

DOI: https://doi.org/10.1097/ACM.0000000000001313

2016-09-01

Abstract:

What problem does this paper attempt to address?

Incremental Learning for Heterogeneous Structure Segmentation in Brain Tumor MRI

Xiaofeng Liu,Helen A. Shih,Fangxu Xing,Emiliano Santarnecchi,Georges El Fakhri,Jonghye Woo

2023-05-31

Abstract:Deep learning (DL) models for segmenting various anatomical structures have achieved great success via a static DL model that is trained in a single source domain. Yet, the static DL model is likely to perform poorly in a continually evolving environment, requiring appropriate model updates. In an incremental learning setting, we would expect that well-trained static models are updated, following continually evolving target domain data -- e.g., additional lesions or structures of interest -- collected from different sites, without catastrophic forgetting. This, however, poses challenges, due to distribution shifts, additional structures not seen during the initial model training, and the absence of training data in a source domain. To address these challenges, in this work, we seek to progressively evolve an ``off-the-shelf" trained segmentation model to diverse datasets with additional anatomical categories in a unified manner. Specifically, we first propose a divergence-aware dual-flow module with balanced rigidity and plasticity branches to decouple old and new tasks, which is guided by continuous batch renormalization. Then, a complementary pseudo-label training scheme with self-entropy regularized momentum MixUp decay is developed for adaptive network optimization. We evaluated our framework on a brain tumor segmentation task with continually changing target domains -- i.e., new MRI scanners/modalities with incremental structures. Our framework was able to well retain the discriminability of previously learned structures, hence enabling the realistic life-long segmentation model extension along with the widespread accumulation of big medical data.

Computer Vision and Pattern Recognition,Artificial Intelligence,Machine Learning,Medical Physics
A Classifier-Free Incremental Learning Framework for Scalable Medical Image Segmentation

Xiaoyang Chen,Hao Zheng,Yifang Xie,Yuncong Ma,Tengfei Li

DOI: https://doi.org/10.48550/arXiv.2405.16328

2024-05-26

Abstract:Current methods for developing foundation models in medical image segmentation rely on two primary assumptions: a fixed set of classes and the immediate availability of a substantial and diverse training dataset. However, this can be impractical due to the evolving nature of imaging technology and patient demographics, as well as labor-intensive data curation, limiting their practical applicability and scalability. To address these challenges, we introduce a novel segmentation paradigm enabling the segmentation of a variable number of classes within a single classifier-free network, featuring an architecture independent of class number. This network is trained using contrastive learning and produces discriminative feature representations that facilitate straightforward interpretation. Additionally, we integrate this strategy into a knowledge distillation-based incremental learning framework, facilitating the gradual assimilation of new information from non-stationary data streams while avoiding catastrophic forgetting. Our approach provides a unified solution for tackling both class- and domain-incremental learning scenarios. We demonstrate the flexibility of our method in handling varying class numbers within a unified network and its capacity for incremental learning. Experimental results on an incompletely annotated, multi-modal, multi-source dataset for medical image segmentation underscore its superiority over state-of-the-art alternative approaches.

Computer Vision and Pattern Recognition
Incremental Learning for Multi-organ Segmentation with Partially Labeled Datasets

Pengbo Liu,Li Xiao,S. Kevin Zhou

DOI: https://doi.org/10.1007/978-3-031-16440-8_68

2021-03-08

Abstract:There exists a large number of datasets for organ segmentation, which are partially annotated, and sequentially constructed. A typical dataset is constructed at a certain time by curating medical images and annotating the organs of interest. In other words, new datasets with annotations of new organ categories are built over time. To unleash the potential behind these partially labeled, sequentially-constructed datasets, we propose to learn a multi-organ segmentation model through incremental learning (IL). In each IL stage, we lose access to the previous annotations, whose knowledge is assumingly captured by the current model, and gain the access to a new dataset with annotations of new organ categories, from which we learn to update the organ segmentation model to include the new organs. We give the first attempt to conjecture that the different distribution is the key reason for 'catastrophic forgetting' that commonly exists in IL methods, and verify that IL has the natural adaptability to medical image scenarios. Extensive experiments on five open-sourced datasets are conducted to prove the effectiveness of our method and the conjecture mentioned above.

Computer Vision and Pattern Recognition
Learning Incrementally to Segment Multiple Organs in a CT Image

Pengbo Liu,Xia Wang,Mengsi Fan,Hongli Pan,Minmin Yin,Xiaohong Zhu,Dandan Du,Xiaoying Zhao,Li Xiao,Lian Ding,Xingwang Wu,S. Kevin Zhou

DOI: https://doi.org/10.48550/arXiv.2203.02100

2022-03-04

Abstract:There exists a large number of datasets for organ segmentation, which are partially annotated and sequentially constructed. A typical dataset is constructed at a certain time by curating medical images and annotating the organs of interest. In other words, new datasets with annotations of new organ categories are built over time. To unleash the potential behind these partially labeled, sequentially-constructed datasets, we propose to incrementally learn a multi-organ segmentation model. In each incremental learning (IL) stage, we lose the access to previous data and annotations, whose knowledge is assumingly captured by the current model, and gain the access to a new dataset with annotations of new organ categories, from which we learn to update the organ segmentation model to include the new organs. While IL is notorious for its `catastrophic forgetting' weakness in the context of natural image analysis, we experimentally discover that such a weakness mostly disappears for CT multi-organ segmentation. To further stabilize the model performance across the IL stages, we introduce a light memory module and some loss functions to restrain the representation of different categories in feature space, aggregating feature representation of the same class and separating feature representation of different classes. Extensive experiments on five open-sourced datasets are conducted to illustrate the effectiveness of our method.

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning
Transfer learning from a sparsely annotated dataset of 3D medical images

Gabriel Efrain Humpire-Mamani,Colin Jacobs,Mathias Prokop,Bram van Ginneken,Nikolas Lessmann

2023-11-09

Abstract:Transfer learning leverages pre-trained model features from a large dataset to save time and resources when training new models for various tasks, potentially enhancing performance. Due to the lack of large datasets in the medical imaging domain, transfer learning from one medical imaging model to other medical imaging models has not been widely explored. This study explores the use of transfer learning to improve the performance of deep convolutional neural networks for organ segmentation in medical imaging. A base segmentation model (3D U-Net) was trained on a large and sparsely annotated dataset; its weights were used for transfer learning on four new down-stream segmentation tasks for which a fully annotated dataset was available. We analyzed the training set size's influence to simulate scarce data. The results showed that transfer learning from the base model was beneficial when small datasets were available, providing significant performance improvements; where fine-tuning the base model is more beneficial than updating all the network weights with vanilla transfer learning. Transfer learning with fine-tuning increased the performance by up to 0.129 (+28\%) Dice score than experiments trained from scratch, and on average 23 experiments increased the performance by 0.029 Dice score in the new segmentation tasks. The study also showed that cross-modality transfer learning using CT scans was beneficial. The findings of this study demonstrate the potential of transfer learning to improve the efficiency of annotation and increase the accessibility of accurate organ segmentation in medical imaging, ultimately leading to improved patient care. We made the network definition and weights publicly available to benefit other users and researchers.

Image and Video Processing,Artificial Intelligence,Computer Vision and Pattern Recognition,Machine Learning
Combined Transfer Learning and Test-Time Augmentation Improves Convolutional Neural Network-Based Semantic Segmentation of Prostate Cancer from Multi-Parametric MR Images

David Hoar,Peter Q Lee,Alessandro Guida,Steven Patterson,Chris V Bowen,Jennifer Merrimen,Cheng Wang,Ricardo Rendon,Steven D Beyea,Sharon E Clarke

DOI: https://doi.org/10.1016/j.cmpb.2021.106375

Abstract:Purpose: Multiparametric MRI (mp-MRI) is a widely used tool for diagnosing and staging prostate cancer. The purpose of this study was to evaluate whether transfer learning, unsupervised pre-training and test-time augmentation significantly improved the performance of a convolutional neural network (CNN) for pixel-by-pixel prediction of cancer vs. non-cancer using mp-MRI datasets. Methods: 154 subjects undergoing mp-MRI were prospectively recruited, 16 of whom subsequently underwent radical prostatectomy. Logistic regression, random forest and CNN models were trained on mp-MRI data using histopathology as the gold standard. Transfer learning, unsupervised pre-training and test-time augmentation were used to boost CNN performance. Models were evaluated using Dice score and area under the receiver operating curve (AUROC) with leave-one-subject-out cross validation. Permutation feature importance testing was performed to evaluate the relative value of each MR contrast to CNN model performance. Statistical significance (p<0.05) was determined using the paired Wilcoxon signed rank test with Benjamini-Hochberg correction for multiple comparisons. Results: Baseline CNN outperformed logistic regression and random forest models. Transfer learning and unsupervised pre-training did not significantly improve CNN performance over baseline; however, test-time augmentation resulted in significantly higher Dice scores over both baseline CNN and CNN plus either of transfer learning or unsupervised pre-training. The best performing model was CNN with transfer learning and test-time augmentation (Dice score of 0.59 and AUROC of 0.93). The most important contrast was apparent diffusion coefficient (ADC), followed by Ktrans and T2, although each contributed significantly to classifier performance. Conclusions: The addition of transfer learning and test-time augmentation resulted in significant improvement in CNN segmentation performance in a small set of prostate cancer mp-MRI data. Results suggest that these techniques may be more broadly useful for the optimization of deep learning algorithms applied to the problem of semantic segmentation in biomedical image datasets. However, further work is needed to improve the generalizability of the specific model presented herein.
An Effective MR-Guided CT Network Training for Segmenting Prostate in CT Images

Wanqi Yang,Yinghuan Shi,Sang Hyun Park,Ming Yang,Yang Gao,Dinggang Shen

DOI: https://doi.org/10.1109/jbhi.2019.2960153

IF: 7.7

2020-08-01

IEEE Journal of Biomedical and Health Informatics

Abstract:Segmentation of prostate in medical imaging data (e.g., CT, MRI, TRUS) is often considered as a critical yet challenging task for radiotherapy treatment. It is relatively easier to segment prostate from MR images than from CT images, due to better soft tissue contrast of the MR images. For segmenting prostate from CT images, most previous methods mainly used CT alone, and thus their performances are often limited by low tissue contrast in the CT images. In this article, we explore the possibility of using indirect guidance from MR images for improving prostate segmentation in the CT images. In particular, we propose a novel deep transfer learning approach, i.e., MR-guided CT network training (namely MICS-NET), which can employ MR images to help better learning of features in CT images for prostate segmentation. In MICS-NET, the guidance from MRI consists of two steps: (1) learning informative and transferable features from MRI and then transferring them to CT images in a cascade manner, and (2) adaptively transferring the prostate likelihood of MRI model (i.e., well-trained convnet by purely using MR images) with a view consistency constraint. To illustrate the effectiveness of our approach, we evaluate MICS-NET on a real CT prostate image set, with the manual delineations available as the ground truth for evaluation. Our methods generate promising segmentation results which achieve (1) six percentages higher Dice Ratio than the CT model purely using CT images and (2) comparable performance with the MRI model purely using MR images.

computer science, interdisciplinary applications,mathematical & computational biology,medical informatics, information systems
Generalizing Deep Whole Brain Segmentation for Pediatric and Post-Contrast MRI with Augmented Transfer Learning

Camilo Bermudez,Justin Blaber,Samuel W. Remedios,Jess E. Reynolds,Catherine Lebel,Maureen McHugo,Stephan Heckers,Yuankai Huo,Bennett A. Landman

DOI: https://doi.org/10.48550/arXiv.1908.04702

2019-08-13

Abstract:Generalizability is an important problem in deep neural networks, especially in the context of the variability of data acquisition in clinical magnetic resonance imaging (MRI). Recently, the Spatially Localized Atlas Network Tiles (SLANT) approach has been shown to effectively segment whole brain non-contrast T1w MRI with 132 volumetric labels. Enhancing generalizability of SLANT would enable broader application of volumetric assessment in multi-site studies. Transfer learning (TL) is commonly used to update the neural network weights for local factors; yet, it is commonly recognized to risk degradation of performance on the original validation/test cohorts. Here, we explore TL by data augmentation to address these concerns in the context of adapting SLANT to anatomical variation and scanning protocol. We consider two datasets: First, we optimize for age with 30 T1w MRI of young children with manually corrected volumetric labels, and accuracy of automated segmentation defined relative to the manually provided truth. Second, we optimize for acquisition with 36 paired datasets of pre- and post-contrast clinically acquired T1w MRI, and accuracy of the post-contrast segmentations assessed relative to the pre-contrast automated assessment. For both studies, we augment the original TL step of SLANT with either only the new data or with both original and new data. Over baseline SLANT, both approaches yielded significantly improved performance (signed rank tests; pediatric: 0.89 vs. 0.82 DSC, p<0.001; contrast: 0.80 vs 0.76, p<0.001). The performance on the original test set decreased with the new-data only transfer learning approach, so data augmentation was superior to strict transfer learning.

Image and Video Processing,Computer Vision and Pattern Recognition
MS-Net: Multi-Site Network for Improving Prostate Segmentation with Heterogeneous MRI Data

Quande Liu,Qi Dou,Lequan Yu,Pheng Ann Heng

DOI: https://doi.org/10.1109/TMI.2020.2974574

2020-02-19

Abstract:Automated prostate segmentation in MRI is highly demanded for computer-assisted diagnosis. Recently, a variety of deep learning methods have achieved remarkable progress in this task, usually relying on large amounts of training data. Due to the nature of scarcity for medical images, it is important to effectively aggregate data from multiple sites for robust model training, to alleviate the insufficiency of single-site samples. However, the prostate MRIs from different sites present heterogeneity due to the differences in scanners and imaging protocols, raising challenges for effective ways of aggregating multi-site data for network training. In this paper, we propose a novel multi-site network (MS-Net) for improving prostate segmentation by learning robust representations, leveraging multiple sources of data. To compensate for the inter-site heterogeneity of different MRI datasets, we develop Domain-Specific Batch Normalization layers in the network backbone, enabling the network to estimate statistics and perform feature normalization for each site separately. Considering the difficulty of capturing the shared knowledge from multiple datasets, a novel learning paradigm, i.e., Multi-site-guided Knowledge Transfer, is proposed to enhance the kernels to extract more generic representations from multi-site data. Extensive experiments on three heterogeneous prostate MRI datasets demonstrate that our MS-Net improves the performance across all datasets consistently, and outperforms state-of-the-art methods for multi-site learning.

Image and Video Processing,Computer Vision and Pattern Recognition
Supervised Transfer Learning at Scale for Medical Imaging

Basil Mustafa,Aaron Loh,Jan Freyberg,Patricia MacWilliams,Megan Wilson,Scott Mayer McKinney,Marcin Sieniek,Jim Winkens,Yuan Liu,Peggy Bui,Shruthi Prabhakara,Umesh Telang,Alan Karthikesalingam,Neil Houlsby,Vivek Natarajan

DOI: https://doi.org/10.48550/arXiv.2101.05913

2021-01-14

Computer Vision and Pattern Recognition

Abstract:Transfer learning is a standard technique to improve performance on tasks with limited data. However, for medical imaging, the value of transfer learning is less clear. This is likely due to the large domain mismatch between the usual natural-image pre-training (e.g. ImageNet) and medical images. However, recent advances in transfer learning have shown substantial improvements from scale. We investigate whether modern methods can change the fortune of transfer learning for medical imaging. For this, we study the class of large-scale pre-trained networks presented by Kolesnikov et al. on three diverse imaging tasks: chest radiography, mammography, and dermatology. We study both transfer performance and critical properties for the deployment in the medical domain, including: out-of-distribution generalization, data-efficiency, sub-group fairness, and uncertainty estimation. Interestingly, we find that for some of these properties transfer from natural to medical images is indeed extremely effective, but only when performed at sufficient scale.
Transfer Learning for KiTS21 Challenge

Xi Yang,Jianpeng Zhang,Jing Zhang,Yong Xia

DOI: https://doi.org/10.1007/978-3-030-98385-7_21

2022-01-01

Abstract:Transfer learning has witnessed a recent surge of interest after proving successful in multiple applications. However, it highly relies on the quantity of annotated data. Constrained by the labor cost and expertise, it is hard to annotate sufficient organs and tumors at the voxel level for medical image segmentation. Consequently, most bench-mark datasets were collected for the segmentation of only one type of organ and/or tumor, and all task-irrelevant organs and tumors were annotated as the background. We aim to make use of these partially but plentifully labeled datasets to boost the segmentation performance of the annotation-limited KiTS21 segmentation task. To this end, we first construct a general medical image segmentation model that learns to segment these partially labeled organs or tumors. Then we transfer its pretrained weights to a specific downstream task, i.e., KiTS21. The primary experiments demonstrate the effectiveness of the proposed transfer learning strategy. Our method achieves 0.890 Dice score, 0.805 SurfaceDice, and 0.822 Tumor Dice in the KiTS21 challenge.
SegBook: A Simple Baseline and Cookbook for Volumetric Medical Image Segmentation

Jin Ye,Ying Chen,Yanjun Li,Haoyu Wang,Zhongying Deng,Ziyan Huang,Yanzhou Su,Chenglong Ma,Yuanfeng Ji,Junjun He

2024-11-22

Abstract:Computed Tomography (CT) is one of the most popular modalities for medical imaging. By far, CT images have contributed to the largest publicly available datasets for volumetric medical segmentation tasks, covering full-body anatomical structures. Large amounts of full-body CT images provide the opportunity to pre-train powerful models, e.g., STU-Net pre-trained in a supervised fashion, to segment numerous anatomical structures. However, it remains unclear in which conditions these pre-trained models can be transferred to various downstream medical segmentation tasks, particularly segmenting the other modalities and diverse targets. To address this problem, a large-scale benchmark for comprehensive evaluation is crucial for finding these conditions. Thus, we collected 87 public datasets varying in modality, target, and sample size to evaluate the transfer ability of full-body CT pre-trained models. We then employed a representative model, STU-Net with multiple model scales, to conduct transfer learning across modalities and targets. Our experimental results show that (1) there may be a bottleneck effect concerning the dataset size in fine-tuning, with more improvement on both small- and large-scale datasets than medium-size ones. (2) Models pre-trained on full-body CT demonstrate effective modality transfer, adapting well to other modalities such as MRI. (3) Pre-training on the full-body CT not only supports strong performance in structure detection but also shows efficacy in lesion detection, showcasing adaptability across target tasks. We hope that this large-scale open evaluation of transfer learning can direct future research in volumetric medical image segmentation.

Image and Video Processing,Computer Vision and Pattern Recognition
Selecting the Best Sequential Transfer Path for Medical Image Segmentation with Limited Labeled Data

Jingyun Yang,Jingge Wang,Guoqing Zhang,Yang Li

2024-10-09

Abstract:The medical image processing field often encounters the critical issue of scarce annotated data. Transfer learning has emerged as a solution, yet how to select an adequate source task and effectively transfer the knowledge to the target task remains challenging. To address this, we propose a novel sequential transfer scheme with a task affinity metric tailored for medical images. Considering the characteristics of medical image segmentation tasks, we analyze the image and label similarity between tasks and compute the task affinity scores, which assess the relatedness among tasks. Based on this, we select appropriate source tasks and develop an effective sequential transfer strategy by incorporating intermediate source tasks to gradually narrow the domain discrepancy and minimize the transfer cost. Thereby we identify the best sequential transfer path for the given target task. Extensive experiments on three MRI medical datasets, FeTS 2022, iSeg-2019, and WMH, demonstrate the efficacy of our method in finding the best source sequence. Compared with directly transferring from a single source task, the sequential transfer results underline a significant improvement in target task performance, achieving an average of 2.58% gain in terms of segmentation Dice score, notably, 6.00% for FeTS 2022. Code is available at the git repository.

Image and Video Processing,Computer Vision and Pattern Recognition
TC-Net: A joint learning framework based on CNN and vision transformer for multi-lesion medical images segmentation

Zhongxiang Zhang,Guangmin Sun,Kun Zheng,Jin-Kui Yang,Yu Li,Xiao-rong Zhu

DOI: https://doi.org/10.1016/j.compbiomed.2023.106967

IF: 7.7

2023-05-18

Computers in Biology and Medicine

Abstract:Background With the rapid advancement of medical imaging technology, the demand for accurate segmentation of medical images is increasing. However, most existing methods are unable to capture locality and long-range dependency information in integrated ways for medical images. Method In this paper, we propose an elegant segmentation framework for medical images named TC-Net, which can utilize both the locality-aware and long-range dependencies in the medical images. As for the locality-aware perspective, we employ a CNN-based encoder and decoder structure. The CNN branch uses the locality of convolution operations to dig out local information in medical images. As for the long-range dependencies, we construct a Transform branch to focus on the global context. Additionally, we proposed a locality-aware and long-range dependency concatenation strategy (LLCS) to aggregate the feature maps obtained from the two subbranches. Finally, we present a dynamic cyclical focal loss (DCFL) to address the class imbalance problem in multi-lesion segmentation. Results Comprehensive experiments were conducted on lesion segmentation tasks using two fundus image databases and a skin image database. The TC-Net achieves scores of 0.6985 and 0.5171 in the metric of pixel mean accuracy on the IDRiD and DDR databases, respectively. Moreover, on the skin image database, the TC-Net reached mean pixel accuracy of 0.8886. The experiment results demonstrate that the proposed method achieves better performance than other deep learning segmentation schemes. Furthermore, the proposed DCFL achieves higher performance than other loss functions in multi-lesion segmentation. Significance The proposed TC-Net is a promising new framework for multi-lesion medical image segmentation and many other challenging image segmentation tasks. © 2001 Elsevier Science. All rights reserved.

engineering, biomedical,computer science, interdisciplinary applications,mathematical & computational biology,biology
DIGEST: Deeply supervIsed knowledGE tranSfer neTwork learning for brain tumor segmentation with incomplete multi-modal MRI scans

Haoran Li,Cheng Li,Weijian Huang,Xiawu Zheng,Yan Xi,Shanshan Wang

DOI: https://doi.org/10.48550/arXiv.2211.07993

2022-11-15

Abstract:Brain tumor segmentation based on multi-modal magnetic resonance imaging (MRI) plays a pivotal role in assisting brain cancer diagnosis, treatment, and postoperative evaluations. Despite the achieved inspiring performance by existing automatic segmentation methods, multi-modal MRI data are still unavailable in real-world clinical applications due to quite a few uncontrollable factors (e.g. different imaging protocols, data corruption, and patient condition limitations), which lead to a large performance drop during practical applications. In this work, we propose a Deeply supervIsed knowledGE tranSfer neTwork (DIGEST), which achieves accurate brain tumor segmentation under different modality-missing scenarios. Specifically, a knowledge transfer learning frame is constructed, enabling a student model to learn modality-shared semantic information from a teacher model pretrained with the complete multi-modal MRI data. To simulate all the possible modality-missing conditions under the given multi-modal data, we generate incomplete multi-modal MRI samples based on Bernoulli sampling. Finally, a deeply supervised knowledge transfer loss is designed to ensure the consistency of the teacher-student structure at different decoding stages, which helps the extraction of inherent and effective modality representations. Experiments on the BraTS 2020 dataset demonstrate that our method achieves promising results for the incomplete multi-modal MR image segmentation task.

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning
A continuous learning approach to brain tumor segmentation: integrating multi-scale spatial distillation and pseudo-labeling strategies

Ruipeng Li,Jianming Ye,Yueqi Huang,Wei Jin,Peng Xu,Lilin Guo

DOI: https://doi.org/10.3389/fonc.2023.1247603

IF: 4.7

2024-01-08

Frontiers in Oncology

Abstract:Introduction This study presents a novel continuous learning framework tailored for brain tumour segmentation, addressing a critical step in both diagnosis and treatment planning. This framework addresses common challenges in brain tumour segmentation, such as computational complexity, limited generalisability, and the extensive need for manual annotation. Methods Our approach uniquely combines multi-scale spatial distillation with pseudo-labelling strategies, exploiting the coordinated capabilities of the ResNet18 and DeepLabV3+ network architectures. This integration enhances feature extraction and efficiently manages model size, promoting accurate and fast segmentation. To mitigate the problem of catastrophic forgetting during model training, our methodology incorporates a multi-scale spatial distillation scheme. This scheme is essential for maintaining model diversity and preserving knowledge from previous training phases. In addition, a confidence-based pseudo-labelling technique is employed, allowing the model to self-improve based on its predictions and ensuring a balanced treatment of data categories. Results The effectiveness of our framework has been evaluated on three publicly available datasets (BraTS2019, BraTS2020, BraTS2021) and one proprietary dataset (BraTS_FAHZU) using performance metrics such as Dice coefficient, sensitivity, specificity and Hausdorff95 distance. The results consistently show competitive performance against other state-of-the-art segmentation techniques, demonstrating improved accuracy and efficiency. Discussion This advance has significant implications for the field of medical image segmentation. Our code is freely available at https://github.com/smallboy-code/A-brain-tumor-segmentation-frameworkusing-continual-learning .

oncology
Pick the Best Pre-trained Model: Towards Transferability Estimation for Medical Image Segmentation

Yuncheng Yang,Meng Wei,Junjun He,Jie Yang,Jin Ye,Yun Gu

2023-07-22

Abstract:Transfer learning is a critical technique in training deep neural networks for the challenging medical image segmentation task that requires enormous resources. With the abundance of medical image data, many research institutions release models trained on various datasets that can form a huge pool of candidate source models to choose from. Hence, it's vital to estimate the source models' transferability (i.e., the ability to generalize across different downstream tasks) for proper and efficient model reuse. To make up for its deficiency when applying transfer learning to medical image segmentation, in this paper, we therefore propose a new Transferability Estimation (TE) method. We first analyze the drawbacks of using the existing TE algorithms for medical image segmentation and then design a source-free TE framework that considers both class consistency and feature variety for better estimation. Extensive experiments show that our method surpasses all current algorithms for transferability estimation in medical image segmentation. Code is available at <a class="link-external link-https" href="https://github.com/EndoluminalSurgicalVision-IMR/CCFV" rel="external noopener nofollow">this https URL</a>

Computer Vision and Pattern Recognition
DG-TTA: Out-of-domain medical image segmentation through Domain Generalization and Test-Time Adaptation

Christian Weihsbach,Christian N. Kruse,Alexander Bigalke,Mattias P. Heinrich

2024-04-10

Abstract:Applying pre-trained medical segmentation models on out-of-domain images often yields predictions of insufficient quality. Several strategies have been proposed to maintain model performance, such as finetuning or unsupervised- and source-free domain adaptation. These strategies set restrictive requirements for data availability. In this study, we propose to combine domain generalization and test-time adaptation to create a highly effective approach for reusing pre-trained models in unseen target domains. Domain-generalized pre-training on source data is used to obtain the best initial performance in the target domain. We introduce the MIND descriptor previously used in image registration tasks as a further technique to achieve generalization and present superior performance for small-scale datasets compared to existing approaches. At test-time, high-quality segmentation for every single unseen scan is ensured by optimizing the model weights for consistency given different image augmentations. That way, our method enables separate use of source and target data and thus removes current data availability barriers. Moreover, the presented method is highly modular as it does not require specific model architectures or prior knowledge of involved domains and labels. We demonstrate this by integrating it into the nnUNet, which is currently the most popular and accurate framework for medical image segmentation. We employ multiple datasets covering abdominal, cardiac, and lumbar spine scans and compose several out-of-domain scenarios in this study. We demonstrate that our method, combined with pre-trained whole-body CT models, can effectively segment MR images with high accuracy in all of the aforementioned scenarios. Open-source code can be found here:

Machine Learning
A Survey of Incremental Transfer Learning: Combining Peer-to-Peer Federated Learning and Domain Incremental Learning for Multicenter Collaboration

Yixing Huang,Christoph Bert,Ahmed Gomaa,Rainer Fietkau,Andreas Maier,Florian Putz

DOI: https://doi.org/10.48550/arXiv.2309.17192

2023-09-29

Abstract:Due to data privacy constraints, data sharing among multiple clinical centers is restricted, which impedes the development of high performance deep learning models from multicenter collaboration. Naive weight transfer methods share intermediate model weights without raw data and hence can bypass data privacy restrictions. However, performance drops are typically observed when the model is transferred from one center to the next because of the forgetting problem. Incremental transfer learning, which combines peer-to-peer federated learning and domain incremental learning, can overcome the data privacy issue and meanwhile preserve model performance by using continual learning techniques. In this work, a conventional domain/task incremental learning framework is adapted for incremental transfer learning. A comprehensive survey on the efficacy of different regularization-based continual learning methods for multicenter collaboration is performed. The influences of data heterogeneity, classifier head setting, network optimizer, model initialization, center order, and weight transfer type have been investigated thoroughly. Our framework is publicly accessible to the research community for further development.

Machine Learning,Computer Vision and Pattern Recognition
A dual meta-learning framework based on idle data for enhancing segmentation of pancreatic cancer

Jun Li,Liang Qi,Qingzhong Chen,Yu-Dong Zhang,Xiaohua Qian

DOI: https://doi.org/10.1016/j.media.2021.102342

IF: 10.9

2022-05-01

Medical Image Analysis

Abstract:Automated segmentation of pancreatic cancer is vital for clinical diagnosis and treatment. However, the small size and inconspicuous boundaries limit the segmentation performance, which is further exacerbated for deep learning techniques with the few training samples due to the high threshold of image acquisition and annotation. To alleviate this issue caused by the small-scale dataset, we collect idle multi-parametric MRIs of pancreatic cancer from different studies to construct a relatively large dataset for enhancing the CT pancreatic cancer segmentation. Therefore, we propose a deep learning segmentation model with the dual meta-learning framework for pancreatic cancer. It can integrate the common knowledge of tumors obtained from idle MRIs and salient knowledge from CT images, making high-level features more discriminative. Specifically, the random intermediate modalities between MRIs and CT are first generated to smoothly fill in the gaps in visual appearance and provide rich intermediate representations for ensuing meta-learning scheme. Subsequently, we employ intermediate modalities-based model-agnostic meta-learning to capture and transfer commonalities. At last, a meta-optimizer is utilized to adaptively learn the salient features within CT data, thus alleviating the interference due to internal differences. Comprehensive experimental results demonstrated our method achieved the promising segmentation performance, with a max Dice score of 64.94% on our private dataset, and outperformed state-of-the-art methods on a public pancreatic cancer CT dataset. The proposed method is an effective pancreatic cancer segmentation framework, which can be easily integrated into other segmentation networks and thus promises to be a potential paradigm for alleviating data scarcity challenges using idle data.

engineering, biomedical,computer science, interdisciplinary applications, artificial intelligence,radiology, nuclear medicine & medical imaging

In Reply to Gopal and Prochazka and to Youngclaus.

Incremental Learning for Heterogeneous Structure Segmentation in Brain Tumor MRI

A Classifier-Free Incremental Learning Framework for Scalable Medical Image Segmentation

Incremental Learning for Multi-organ Segmentation with Partially Labeled Datasets

Learning Incrementally to Segment Multiple Organs in a CT Image

Transfer learning from a sparsely annotated dataset of 3D medical images

Combined Transfer Learning and Test-Time Augmentation Improves Convolutional Neural Network-Based Semantic Segmentation of Prostate Cancer from Multi-Parametric MR Images

An Effective MR-Guided CT Network Training for Segmenting Prostate in CT Images

Generalizing Deep Whole Brain Segmentation for Pediatric and Post-Contrast MRI with Augmented Transfer Learning

MS-Net: Multi-Site Network for Improving Prostate Segmentation with Heterogeneous MRI Data

Supervised Transfer Learning at Scale for Medical Imaging

Transfer Learning for KiTS21 Challenge

SegBook: A Simple Baseline and Cookbook for Volumetric Medical Image Segmentation

Selecting the Best Sequential Transfer Path for Medical Image Segmentation with Limited Labeled Data

TC-Net: A joint learning framework based on CNN and vision transformer for multi-lesion medical images segmentation

DIGEST: Deeply supervIsed knowledGE tranSfer neTwork learning for brain tumor segmentation with incomplete multi-modal MRI scans

A continuous learning approach to brain tumor segmentation: integrating multi-scale spatial distillation and pseudo-labeling strategies

Pick the Best Pre-trained Model: Towards Transferability Estimation for Medical Image Segmentation

DG-TTA: Out-of-domain medical image segmentation through Domain Generalization and Test-Time Adaptation

A Survey of Incremental Transfer Learning: Combining Peer-to-Peer Federated Learning and Domain Incremental Learning for Multicenter Collaboration

A dual meta-learning framework based on idle data for enhancing segmentation of pancreatic cancer