Abstract:Background and ObjectiveDeep learning approaches are common in image processing, but often rely on supervised learning, which requires a large volume of training images, usually accompanied by hand-crafted labels. As labelled data are often not available, it would be desirable to develop methods that allow such data to be compiled automatically. In this study, we used a Generative Adversarial Network (GAN) to generate realistic B-mode musculoskeletal ultrasound images, and tested the suitability of two automated labelling approaches.MethodsWe used a model including two GANs each trained to transfer an image from one domain to another. The two inputs were a set of 100 longitudinal images of the gastrocnemius medialis muscle, and a set of 100 synthetic segmented masks that featured two aponeuroses and a random number of 'fascicles'. The model output a set of synthetic ultrasound images and an automated segmentation of each real input image. This automated segmentation process was one of the two approaches we assessed. The second approach involved synthesising ultrasound images and then feeding these images into an ImageJ/Fiji-based automated algorithm, to determine whether it could detect the aponeuroses and muscle fascicles.ResultsHistogram distributions were similar between real and synthetic images, but synthetic images displayed less variation between samples and a narrower range. Mean entropy values were statistically similar (real: 6.97, synthetic: 7.03; p = 0.218), but the range was much narrower for synthetic images (6.91 – 7.11 versus 6.30 – 7.62). When comparing GAN-derived and manually labelled segmentations, intersection-over-union values- denoting the degree of overlap between aponeurosis labels- varied between 0.0280 – 0.612 (mean ± SD: 0.312 ± 0.159), and pennation angles were higher for the GAN-derived segmentations (25.1° vs. 19.3 °; p < 0.001). For the second segmentation approach, the algorithm generally performed equally well on synthetic and real images, yielding pennation angles within the physiological range (13.8-20°).ConclusionsWe used a GAN to generate realistic B-mode ultrasound images, and extracted muscle architectural parameters from these images automatically. This approach could enable generation of large labelled datasets for image segmentation tasks, and may also be useful for data sharing. Automatic generation and labelling of ultrasound images minimises user input and overcomes several limitations associated with manual analysis.

Ultrasound Image Classification using ACGAN with Small Training Dataset

GSDA: Generative Adversarial Network-based Semi-Supervised Data Augmentation for Ultrasound Image Classification

GANs for generation of synthetic ultrasound images from small datasets

Breast Cancer Histopathological Image Classification with Adversarial Image Synthesis

Empirical Analysis of Deep Convolutional Generative Adversarial Network for Ultrasound Image Synthesis

Analyzing Data Augmentation for Medical Images: A Case Study in Ultrasound Images

Voice-assisted Image Labelling for Endoscopic Ultrasound Classification using Neural Networks

Echocardiography Segmentation Based on Cross-modal Data Augmentation Method

Reliable Breast Cancer Diagnosis with Deep Learning: DCGAN-Driven Mammogram Synthesis and Validity Assessment

Using deep learning to generate synthetic B-mode musculoskeletal ultrasound images

Data Augmentation of Thyroid Ultrasound Images Using Generative Adversarial Network

Automatic Detection of Breast Lesions in Automated 3D Breast Ultrasound with Cross-Organ Transfer Learning

Deep Learning for Ovarian Tumor Classification with Ultrasound Images.

Deep Learning Based Data Augmentation and Classification for Limited Medical Data Learning

An experimental study on breast lesion detection and classification from ultrasound images using deep learning architectures

Breast Ultrasound Images Augmentation and Segmentation Using GAN with Identity Block and Modified U-Net 3

Semantic Consistency Generative Adversarial Network for Cross-Modality Domain Adaptation in Ultrasound Thyroid Nodule Classification

Knowledge-guided Synthetic Medical Image Adversarial Augmentation for Ultrasonography Thyroid Nodule Classification

Dganet: a dual global attention neural network for breast lesion detection in ultrasound images

Generating synthetic medical images with limited data using auxiliary classifier generative adversarial network: a study on thyroid ultrasound images

Endoscopic Image Analysis Using Deep Convolutional GAN and Traditional Data Augmentation