Abstract:Pretraining CNN models (i.e., UNet) through self-supervision has become a powerful approach to facilitate medical image segmentation under low annotation regimes. Recent contrastive learning methods encourage similar global representations when the same image undergoes different transformations, or enforce invariance across different image/patch features that are intrinsically correlated. However, CNN-extracted global and local features are limited in capturing long-range spatial dependencies that are essential in biological anatomy. To this end, we present a keypoint-augmented fusion layer that extracts representations preserving both short- and long-range self-attention. In particular, we augment the CNN feature map at multiple scales by incorporating an additional input that learns long-range spatial self-attention among localized keypoint features. Further, we introduce both global and local self-supervised pretraining for the framework. At the global scale, we obtain global representations from both the bottleneck of the UNet, and by aggregating multiscale keypoint features. These global features are subsequently regularized through image-level contrastive objectives. At the local scale, we define a distance-based criterion to first establish correspondences among keypoints and encourage similarity between their features. Through extensive experiments on both MRI and CT segmentation tasks, we demonstrate the architectural advantages of our proposed method in comparison to both CNN and Transformer-based UNets, when all architectures are trained with randomly initialized weights. With our proposed pretraining strategy, our method further outperforms existing SSL methods by producing more robust self-attention and achieving state-of-the-art segmentation results. The code is available at <a class="link-external link-https" href="https://github.com/zshyang/kaf.git" rel="external noopener nofollow">this https URL</a>.

Contrastive learning of global and local features for medical image segmentation with limited annotations

Localized Region Contrast for Enhancing Self-Supervised Learning in Medical Image Segmentation

Self-supervised Contrastive Learning with Random Walks for Medical Image Segmentation with limited Annotations

Contrastive Semi-Supervised Learning for Domain Adaptive Segmentation Across Similar Anatomical Structures

Self-Ensembling Contrastive Learning for Semi-Supervised Medical Image Segmentation

Leveraging Unlabeled Data for 3D Medical Image Segmentation through Self-Supervised Contrastive Learning

IDEAL: Improved DEnse locAL Contrastive Learning for Semi-Supervised Medical Image Segmentation

Keypoint-Augmented Self-Supervised Learning for Medical Image Segmentation with Limited Annotation

Contrastive self-supervised learning from 100 million medical images with optional supervision

GLRP: Global and Local Contrastive Learning Based on Relative Position for Medical Image Segmentation on Cardiac MRI

Overcoming Dimensional Collapse in Self-supervised Contrastive Learning for Medical Image Segmentation

Cross-level Contrastive Learning and Consistency Constraint for Semi-supervised Medical Image Segmentation

Learning to segment with limited annotations: Self-supervised pretraining with regression and contrastive loss in MRI

Unsupervised Local Discrimination for Medical Images

Multi-level Asymmetric Contrastive Learning for Medical Image Segmentation Pre-training

Multiview Long-Short Spatial Contrastive Learning For 3D Medical Image Analysis

Multi-Level Contrastive Student-Teacher Structure for Semi-Supervised Medical Image Segmentation

Contrastive and Selective Hidden Embeddings for Medical Image Segmentation

Semi-supervised Task-driven Data Augmentation for Medical Image Segmentation

Self-Supervised Alignment Learning for Medical Image Segmentation