Abstract:Medical image segmentation task has been a hot research topic in the field of computer vision and natural field for many years. With the rapid development and application of convolutional neural networks, more and more medical segmentation models based on deep learning have been proposed, and more successful results and applications have been achieved in many disease segmentation tasks.The effect of medical image segmentation tasks is getting better. However, the computational cost of the medical image segmentation model has not been improved well. In this work, we propose a medical image segmentation model compression scheme that is theoretically applicable to all convolutional neural networks. First, we choose to construct a search space based on the number of convolution kernels at each location where convolution is used in the model, and then use neural network search to find a sub-network in this search space with less computation and high segmentation accuracy. Aiming at the encoding-decoding structure of the segmentation network, we propose Symmetrical-NAS to ensure that the encoding structure and decoding structure of any sub-network in our search space are symmetrical. Since it is too expensive to traverse the entire search space for training to find the most suitable architecture, we use weight sharing for training. During training, a sub-network in the search space is randomly selected for activation each time. Second, we use the method of knowledge distillation for training. We use the basic model as the teacher model and the searched sub-network as the student model to realize the knowledge transfer between the teacher model and the student model. Third, we use separable convolution instead of convolutional layer. Our method can be applied to various medicial image segmentation models regardless of model architectures and learning algorithms. Our method can reduce the computation of medical image segmentation models by 90× regarding FLOPs with little loss of segmentation effect. The code and demo will be publicly available.

Model Compression with NAS and Knowledge Distillation for Medical Image Segmentation.

MS-NAS: Multi-scale Neural Architecture Search for Medical Image Segmentation

Attention-Fused CNN Model Compression with Knowledge Distillation for Brain Tumor Segmentation

Efficient Medical Image Segmentation Based on Knowledge Distillation

MSKD: Structured knowledge distillation for efficient medical image segmentation

Efficient knowledge distillation for liver CT segmentation using growing assistant network

A Novel Multi-Dimensional Joint Search Method for the Compression of Medical Image Segmentation Models

Post-Training Network Compression for 3D Medical Image Segmentation: Reducing Computational Efforts via Tucker Decomposition

Knowledge distillation with ensembles of convolutional neural networks for medical image segmentation

MH-Net: Model-data-driven hybrid-fusion network for medical image segmentation

NG-NAS: Node growth neural architecture search for 3D medical image segmentation

Image Complexity Guided Network Compression for Biomedical Image Segmentation

A Medical Image Segmentation Method Combining Knowledge Distillation and Contrastive Learning

Graph Flow: Cross-layer Graph Flow Distillation for Dual Efficient Medical Image Segmentation

CM-SegNet: A Deep Learning-Based Automatic Segmentation Approach for Medical Images by Combining Convolution and Multilayer Perceptron

A Novel Deep Learning Model for Medical Image Segmentation with Convolutional Neural Network and Transformer

Multi-Task Multi-Scale Contrastive Knowledge Distillation for Efficient Medical Image Segmentation

DCACNet: Dual context aggregation and attention-guided cross deconvolution network for medical image segmentation

Integrating prior knowledge into a bibranch pyramid network for medical image segmentation

Network-Agnostic Knowledge Transfer for Medical Image Segmentation

CC-NET: Image Complexity Guided Network Compression for Biomedical Image Segmentation