Abstract:The semantic image segmentation task consists of classifying each pixel of an image into an instance, where each instance corresponds to a class. This task is a part of the concept of scene understanding or better explaining the global context of an image. In the medical image analysis domain, image segmentation can be used for image-guided interventions, radiotherapy, or improved radiological diagnostics. In this review, we categorize the leading deep learning-based medical and non-medical image segmentation solutions into six main groups of deep architectural, data synthesis-based, loss function-based, sequenced models, weakly supervised, and multi-task methods and provide a comprehensive review of the contributions in each of these groups. Further, for each group, we analyze each variant of these groups and discuss the limitations of the current approaches and present potential future research directions for semantic image segmentation.

What problem does this paper attempt to address?

This paper primarily reviews the tasks of semantic segmentation for natural and medical images, systematically categorizing and discussing deep learning-based methods. The main objectives of the paper are: 1. **Comprehensive Coverage of Research Contributions**: It provides a comprehensive overview of research work in the field of semantic segmentation for natural and medical images. Special attention is given to medical imaging modalities, including 2D (RGB and grayscale) and 3D volumetric medical images. 2. **Method Classification**: The literature is divided into six categories: architecture improvements, optimization function improvements, data synthesis improvements, weakly supervised models, sequential models, and multi-task models, with detailed analysis of each category. 3. **Study of Loss Function Behavior**: It analyzes the performance of various popular loss functions in handling different levels of false positive and false negative predictions. 4. **Future Research Directions**: Based on the comprehensive review, it identifies important future research directions within each category. Specifically, the paper first introduces the basic definition and importance of the semantic segmentation task, followed by a detailed discussion of the progress in various methods. For example, in the **network architecture improvements** section, the paper reviews the development from fully convolutional networks (FCNs) to encoder-decoder models (such as U-Net and V-Net), and then to more complex structures (such as DeepLabV3+). Additionally, it explores how techniques like attention mechanisms and adversarial training can further enhance model performance. In the **medical images** section, the paper emphasizes the role of model compression, attention mechanisms, and adversarial training in improving the accuracy of medical image segmentation. These methods are particularly important when dealing with high-resolution or three-dimensional images, as they help achieve real-time processing and reduce memory consumption. Overall, this review paper aims to provide readers with a comprehensive perspective on the application of deep learning in the field of semantic segmentation for natural and medical images, and to offer guidance for future related research.

Deep Semantic Segmentation of Natural and Medical Images: A Review

A Review on Progress in Semantic Image Segmentation and Its Application to Medical Images

Development of an Interactive Semantic Medical Image Segmentation System.

Advances in Medical Image Segmentation: A Comprehensive Review of Traditional, Deep Learning and Hybrid Approaches

Deep Neural Networks for Medical Image Segmentation

Medical Image Segmentation Using Deep Learning: Review

Deep-Learning-Based Approaches for Semantic Segmentation of Natural Scene Images: A Review

Fully Convolutional Network for the Semantic Segmentation of Medical Images: A Survey

Deep learning approaches to biomedical image segmentation

A Review on Deep Learning Techniques Applied to Semantic Segmentation

A review of semantic segmentation using deep neural networks

Deep semi-supervised learning for medical image segmentation: A review

A Review of Deep-Learning-Based Medical Image Segmentation Methods

Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges

Deep Interactive Segmentation of Medical Images: A Systematic Review and Taxonomy

Image Segmentation Using Deep Learning: A Survey

A Survey on Medical Image Segmentation Based on Deep Learning Techniques

A Review Of Deep Learning Based Methods For Medical Image Multi-Organ Segmentation

Review the state-of-the-art technologies of semantic segmentation based on deep learning

Medical Image Semantic Segmentation Based on Deep Learning

A Survey on Image Semantic Segmentation Using Deep Learning Techniques