Abstract:Aiming at the problem of insignificant target morphological features, inaccurate detection and unclear boundary of small-target regions, and multitarget boundary overlap in multitarget complex image segmentation, combining the image segmentation mechanism of generative adversarial network with the feature enhancement method of nonlocal attention, a generative adversarial network fused with attention mechanism (AM-GAN) is proposed. The generative network in the model is composed of residual network and nonlocal attention module, which use the feature extraction and multiscale fusion mechanism of residual network, as well as feature enhancement and global information fusion ability of nonlocal spatial-channel dual attention to enhance the target features in the detection area and improve the continuity and clarity of the segmentation boundary. The adversarial network is composed of fully convolutional networks, which penalizes the loss of information in small-target regions by judging the authenticity of prediction and label segmentation and improves the detection ability of the generative adversarial model for small targets and the accuracy of multitarget segmentation. AM-GAN can use the GAN’s inherent mechanism that reconstruct and repair high-resolution image, as well as the ability of nonlocal attention global receptive field to strengthen detail features, automatically learn to focus on target structures of different shapes and sizes, highlight salient features useful for specific tasks, reduce the loss of image detail features, improve the accuracy of small-target detection, and optimize the segmentation boundary of multitargets. Taking medical MRI abdominal image segmentation as a verification experiment, multitargets such as liver, left/right kidney, and spleen are selected for segmentation and abnormal tissue detection. In the case of small and unbalanced sample datasets, the class pixels’ accuracy reaches 87.37%, the intersection over union is 92.42%, and the average Dice coefficient is 93%. Compared with other methods in the experiment, the segmentation precision and accuracy are greatly improved. It shows that the proposed method has good applicability for solving typical multitarget image segmentation problems such as small-target feature detection, boundary overlap, and offset deformation.

Conditional generative adversarial network based multi-task learning framework for breast lesion segmentation of ultrasound images

Lesion Segmentation in Ultrasound Using Semi-pixel-wise Cycle Generative Adversarial Nets

Conditional Generative Adversarial and Convolutional Networks for X-ray Breast Mass Segmentation and Shape Classification

Breast Tumor Segmentation and Shape Classification in Mammograms using Generative Adversarial and Convolutional Neural Network

Medical image synthesis via conditional GANs: Application to segmenting brain tumours

Breast Ultrasound Images Augmentation and Segmentation Using GAN with Identity Block and Modified U-Net 3

Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images

Applying Conditional Generative Adversarial Networks for Imaging Diagnosis

Multi-Task Generative Adversarial Network for Handling Imbalanced Clinical Data

Multi-task Learning for Segmentation and Classification of Tumors in 3D Automated Breast Ultrasound Images

A Generative Adversarial Network Fused with Dual-Attention Mechanism and Its Application in Multitarget Image Fine Segmentation

Lesion Conditional Image Generation for Improved Segmentation of Intracranial Hemorrhage from CT Images

Bidirectional cross-modality unsupervised domain adaptation using generative adversarial networks for cardiac image segmentation

SAP‐cGAN: Adversarial learning for breast mass segmentation in digital mammogram based on superpixel average pooling

Unified Attentional Generative Adversarial Network for Brain Tumor Segmentation from Multimodal Unpaired Images

U-shaped GAN for Semi-Supervised Learning and Unsupervised Domain Adaptation in High Resolution Chest Radiograph Segmentation

A bidirectional multilayer contrastive adaptation network with anatomical structure preservation for unpaired cross-modality medical image segmentation

A Multi-Task Learning Framework for Automated Segmentation and Classification of Breast Tumors From Ultrasound Images

Multi-Model Medical Image Segmentation Using Multi-Stage Generative Adversarial Network

Learning with limited target data to detect cells in cross-modality images

Generative Adversarial Networks based Skin Lesion Segmentation