Abstract:A large labeled dataset is a key to the success of supervised deep learning, but for medical image segmentation, it is highly challenging to obtain sufficient annotated images for model training. In many scenarios, unannotated images are abundant and easy to acquire. Self-supervised learning (SSL) has shown great potentials in exploiting raw data information and representation learning. In this paper, we propose Hierarchical Self-Supervised Learning (HSSL), a new self-supervised framework that boosts medical image segmentation by making good use of unannotated data. Unlike the current literature on task-specific self-supervised pretraining followed by supervised fine-tuning, we utilize SSL to learn task-agnostic knowledge from heterogeneous data for various medical image segmentation tasks. Specifically, we first aggregate a dataset from several medical challenges, then pre-train the network in a self-supervised manner, and finally fine-tune on labeled data. We develop a new loss function by combining contrastive loss and classification loss, and pre-train an encoder-decoder architecture for segmentation tasks. Our extensive experiments show that multi-domain joint pre-training benefits downstream segmentation tasks and outperforms single-domain pre-training significantly. Compared to learning from scratch, our method yields better performance on various tasks (e.g., +0.69%documentclass[12pt]{minimal}usepackage{amsmath}usepackage{wasysym}usepackage{amsfonts}usepackage{amssymb}usepackage{amsbsy}usepackage{mathrsfs}usepackage{upgreek}setlength{oddsidemargin}{-69pt}egin{document}$$+0.69\%$$end{document} to +18.60%documentclass[12pt]{minimal}usepackage{amsmath}usepackage{wasysym}usepackage{amsfonts}usepackage{amssymb}usepackage{amsbsy}usepackage{mathrsfs}usepackage{upgreek}setlength{oddsidemargin}{-69pt}egin{document}$$+18.60\%$$end{document} in Dice with 5%documentclass[12pt]{minimal}usepackage{amsmath}usepackage{wasysym}usepackage{amsfonts}usepackage{amssymb}usepackage{amsbsy}usepackage{mathrsfs}usepackage{upgreek}setlength{oddsidemargin}{-69pt}egin{document}$$5\%$$end{document} of annotated data). With limited amounts of training data, our method can substantially bridge the performance gap with respect to denser annotations (e.g., 10%documentclass[12pt]{minimal}usepackage{amsmath}usepackage{wasysym}usepackage{amsfonts}usepackage{amssymb}usepackage{amsbsy}usepackage{mathrsfs}usepackage{upgreek}setlength{oddsidemargin}{-69pt}egin{document}$$10\%$$end{document} vs. 100%documentclass[12pt]{minimal}usepackage{amsmath}usepackage{wasysym}usepackage{amsfonts}usepackage{amssymb}usepackage{amsbsy}usepackage{mathrsfs}usepackage{upgreek}setlength{oddsidemargin}{-69pt}egin{document}$$100\%$$end{document} annotations).

Self-supervised multi-task learning for medical image analysis

Deep Semisupervised Multitask Learning Model and Its Interpretability for Survival Analysis.

Partly Supervised Multitask Learning

Big Self-Supervised Models Advance Medical Image Classification

Self-supervised learning framework application for medical image analysis: a review and summary

Self-supervised learning and semi-supervised learning for multi-sequence medical image classification

Deep neural models for automated multi-task diagnostic scan management—quality enhancement, view classification and report generation

Contrastive self-supervised learning from 100 million medical images with optional supervision

Self-Supervised Learning for Building Robust Pediatric Chest X-ray Classification Models

Continual Self-supervised Learning: Towards Universal Multi-modal Medical Data Representation Learning

MUSCLE: Multi-task Self-supervised Continual Learning to Pre-train Deep Models for X-ray Images of Multiple Body Parts

Multimodal Self-Supervised Learning for Medical Image Analysis

Semi-supervised Multi-task Learning with Chest X-Ray Images

Self-Supervised Multi-Modal Alignment for Whole Body Medical Imaging

Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model

Self-supervised Learning for Chest CT - Training Strategies and Effect on Downstream Applications

Self-supervised learning for medical image analysis using image context restoration

An Efficient Semi-Supervised Framework with Multi-Task and Curriculum Learning for Medical Image Segmentation

Two-Stage Multi-task Self-Supervised Learning for Medical Image Segmentation

Hierarchical Self-supervised Learning for Medical Image Segmentation Based on Multi-domain Data Aggregation

3D Self-Supervised Methods for Medical Imaging