Abstract:In medical image segmentation tasks, the scarcity of labeled training data poses a significant challenge when training deep neural networks. When using U-Net-style architectures, it is common practice to address this problem by pretraining the encoder part on a large general-purpose dataset like ImageNet. However, these methods are resource-intensive and do not guarantee improved performance on the downstream task. In this paper we investigate a variety of training setups on medical image segmentation datasets, using ImageNet-pretrained models. By examining over 300 combinations of models, datasets, and training methods, we find that shorter pretraining often leads to better results on the downstream task, providing additional proof to the well-known fact that the accuracy of the model on ImageNet is a poor indicator for downstream performance. As our main contribution, we introduce a novel transferability metric, based on contrastive learning, that measures how robustly a pretrained model is able to represent the target data. In contrast to other transferability scores, our method is applicable to the case of transferring from ImageNet classification to medical image segmentation. We apply our robustness score by measuring it throughout the pretraining phase to indicate when the model weights are optimal for downstream transfer. This reduces pretraining time and improves results on the target task.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve the problem of pre - training efficiency in medical image segmentation tasks, especially how to optimize the transfer learning process from large - scale general datasets (such as ImageNet) to specific medical image segmentation tasks. Specifically, the paper focuses on the following aspects: 1. **Scarcity of labeled data**: In medical image segmentation tasks, high - quality labeled data is very limited, which poses a challenge to the training of deep neural networks. 2. **Effectiveness of pre - training**: The common practice is to pre - train the encoder part of the U - Net - style architecture on large - general datasets (such as ImageNet) to improve the performance of downstream tasks. However, this method is resource - intensive and does not necessarily guarantee the improvement of downstream task performance. 3. **Risk of over - pre - training**: Long - term pre - training may cause the model to learn the features specific to the source task, thereby impairing its generalization ability on downstream tasks. To solve these problems, the authors propose the following research objectives: - **Finding the optimal pre - training duration**: Verify the impact of different pre - training durations on the performance of downstream tasks through experiments and find the best pre - training stop point. - **Introducing a new transferability metric**: Based on contrastive learning, a new transferability metric (robustness score) is proposed to evaluate the representational ability of the pre - trained model on the target dataset. - **Reducing pre - training time**: Use the proposed transferability metric as an early - stopping condition by an online method to reduce unnecessary pre - training steps, thereby saving computing resources. ### Specific contributions 1. **Application of transferability metric**: This study is the first to apply contrastive learning to measure the transferability from ImageNet classification tasks to medical image segmentation tasks, providing a method suitable for such task pairs. 2. **Experimental verification**: Through extensive experiments on more than 300 combinations of models, datasets, and training methods, it is proved that shorter pre - training often leads to better downstream task performance and verifies the effectiveness of the new transferability metric. 3. **Practical application value**: The new method not only improves the performance of downstream tasks but also significantly reduces the time and resources required for pre - training, which has important practical application value. ### Conclusion By introducing a transferability metric based on contrastive learning, the authors have successfully solved the problem of low pre - training efficiency in medical image segmentation tasks, providing new ideas and tools for future research.

Enhancing pretraining efficiency for medical image segmentation via transferability metrics

Pick the Best Pre-trained Model: Towards Transferability Estimation for Medical Image Segmentation

Transfer learning from a sparsely annotated dataset of 3D medical images

Rethinking Pre-Training on Medical Imaging.

Transfusion: Understanding Transfer Learning for Medical Imaging

How Transferable Are Self-supervised Features in Medical Image Classification Tasks?

A Systematic Benchmarking Analysis of Transfer Learning for Medical Image Analysis

Why does my medical AI look at pictures of birds? Exploring the efficacy of transfer learning across domain boundaries

Enhanced Medical Image Segmentation using Transfer Learning with Res101_UNet: Experimental Insights and Comparative Performance Analysis

STU-Net: Scalable and Transferable Medical Image Segmentation Models Empowered by Large-Scale Supervised Pre-training

Efficient Prediction of Model Transferability in Semantic Segmentation Tasks

UC-Hybrid: Uncertainty-based contrastive learning on hybrid network for medical image segmentation

Multi-Task Multi-Scale Contrastive Knowledge Distillation for Efficient Medical Image Segmentation

Enhanced Transfer Learning Through Medical Imaging and Patient Demographic Data Fusion

Cross-dimensional transfer learning in medical image segmentation with deep learning

LHU-Net: A Light Hybrid U-Net for Cost-Efficient, High-Performance Volumetric Medical Image Segmentation

Self-Supervised Pretraining for 2D Medical Image Segmentation

What Makes Transfer Learning Work For Medical Images: Feature Reuse & Other Factors

BCU-Net: Bridging ConvNeXt and U-Net for medical image segmentation

Prompt tuning for parameter-efficient medical image segmentation

Efficient Transferability Assessment for Selection of Pre-trained Detectors