Abstract:Purpose: Building accurate and robust artificial intelligence systems for medical image assessment requires the creation of large sets of annotated training examples. However, constructing such datasets is very costly due to the complex nature of annotation tasks, which often require expert knowledge (e.g., a radiologist). To counter this limitation, we propose a method to learn from medical images at scale in a self-supervised way. Approach: Our approach, based on contrastive learning and online feature clustering, leverages training datasets of over 100,000,000 medical images of various modalities, including radiography, computed tomography (CT), magnetic resonance (MR) imaging, and ultrasonography (US). We propose to use the learned features to guide model training in supervised and hybrid self-supervised/supervised regime on various downstream tasks. Results: We highlight a number of advantages of this strategy on challenging image assessment problems in radiography, CT, and MR: (1) significant increase in accuracy compared to the state-of-the-art (e.g., area under the curve boost of 3% to 7% for detection of abnormalities from chest radiography scans and hemorrhage detection on brain CT); (2) acceleration of model convergence during training by up to 85% compared with using no pretraining (e.g., 83% when training a model for detection of brain metastases in MR scans); and (3) increase in robustness to various image augmentations, such as intensity variations, rotations or scaling reflective of data variation seen in the field. Conclusions: The proposed approach enables large gains in accuracy and robustness on challenging image assessment problems. The improvement is significant compared with other state-of-the-art approaches trained on medical or vision images (e.g., ImageNet).

ContraSurv: Enhancing Prognostic Assessment of Medical Images via Data-Efficient Weakly Supervised Contrastive Learning

Semi-Supervised Convolutional Vision Transformer with Bi-Level Uncertainty Estimation for Medical Image Segmentation

CD-Surv: a contrastive-based model for dynamic survival analysis

Contrastive Learning of Medical Visual Representations from Paired Images and Text

Self-Contrastive Weakly Supervised Learning Framework for Prognostic Prediction Using Whole Slide Images

A Multimodal Object-level Contrast Learning Method for Cancer Survival Risk Prediction

Supervised Contrastive Vision Transformer for Breast Histopathological Image Classification

Merging-Diverging Hybrid Transformer Networks for Survival Prediction in Head and Neck Cancer

CONSULT: Contrastive Self-Supervised Learning for Few-shot Tumor Detection

ConPro: Learning Severity Representation for Medical Images using Contrastive Learning and Preference Optimization

SurvivalCNN: A deep learning-based method for gastric cancer survival prediction using radiological imaging data and clinicopathological variables

Adaptive segmentation-to-survival learning for survival prediction from multi-modality medical images

Boosting Medical Image-based Cancer Detection via Text-guided Supervision from Reports

HVTSurv: Hierarchical Vision Transformer for Patient-Level Survival Prediction from Whole Slide Image

Contrastive self-supervised learning from 100 million medical images with optional supervision

SurvConvMixer: robust and interpretable cancer survival prediction based on ConvMixer using pathway-level gene expression images

A2HTL: An Automated Hybrid Transformer Based Learning for Predicting Survival of Esophageal Cancer Using CT Images

Uncertainty-Guided Voxel-Level Supervised Contrastive Learning for Semi-Supervised Medical Image Segmentation

Improving Medical Vision-Language Contrastive Pretraining with Semantics-aware Triage

Robust image representations with counterfactual contrastive learning

Enhancing Weakly Supervised 3D Medical Image Segmentation through Probabilistic-aware Learning