De-speckling of Optical Coherence Tomography Images Using a Generative Adversarial Network

Zhao Dong,Guangming Ni,Guoyan Liu,Jason Jerwick,Chao Zhou

DOI: https://doi.org/10.1364/oct.2020.ow2e.6

2020-01-01

Abstract:We developed a custom generative adversarial network (GAN) that can effectively reduce speckle noise in OCT images and videos without increase system complexity while maintaining spatial and temporal resolutions.

What problem does this paper attempt to address?

Probabilistic volumetric speckle suppression in OCT using deep learning

Bhaskara Rao Chintada,Sebastián Ruiz-Lopera,René Restrepo,Brett E. Bouma,Martin Villiger,Néstor Uribe-Patarroyo

DOI: https://doi.org/10.1364/BOE.523716

2023-12-08

Abstract:We present a deep learning framework for volumetric speckle reduction in optical coherence tomography (OCT) based on a conditional generative adversarial network (cGAN) that leverages the volumetric nature of OCT data. In order to utilize the volumetric nature of OCT data, our network takes partial OCT volumes as input, resulting in artifact-free despeckled volumes that exhibit excellent speckle reduction and resolution preservation in all three dimensions. Furthermore, we address the ongoing challenge of generating ground truth data for supervised speckle suppression deep learning frameworks by using volumetric non-local means despeckling-TNode to generate training data. We show that, while TNode processing is computationally demanding, it serves as a convenient, accessible gold-standard source for training data; our cGAN replicates efficient suppression of speckle while preserving tissue structures with dimensions approaching the system resolution of non-local means despeckling while being two orders of magnitude faster than TNode. We demonstrate fast, effective, and high-quality despeckling of the proposed network in different tissue types acquired with three different OCT systems compared to existing deep learning methods. The open-source nature of our work facilitates re-training and deployment in any OCT system with an all-software implementation, working around the challenge of generating high-quality, speckle-free training data.

Image and Video Processing,Medical Physics,Optics
Improved Sparse Representation Algorithm for Optical Coherence Tomography Images

Hanyuan Miao,Xiaohong Zhou,Wei Wang,Weiliang Jiang,Tao Jin

DOI: https://doi.org/10.1002/ima.22711

IF: 2.177

2022-01-01

International Journal of Imaging Systems and Technology

Abstract:An improved sparse representation algorithm is proposed for reducing speckle noise in optical coherence tomography (OCT) images. To reduce the dictionary training time, the orthogonal matching pursuit algorithm (OMP) is replaced by a piecewise orthogonal matching pursuit algorithm (ST-OMP), and a threshold standard is set to train multiple atoms simultaneously to improve the tracking efficiency. Normal, diabetic macular edema, and fundus neovascularization membrane retina OCT images were tested using the proposed algorithm and other denoising algorithms. The results show that the proposed algorithm exhibits an excellent denoising ability. The details and edge information in the OCT images were well preserved by the proposed algorithm. Meanwhile, the image processing time was reduced by half with the proposed algorithm compared with the traditional K-SVD algorithm.
Resolution enhancement and realistic speckle recovery with generative adversarial modeling of micro-optical coherence tomography

Kaicheng Liang,Xinyu Liu,Si Chen,Jun Xie,Wei Qing Lee,Linbo Liu,Hwee Kuan Lee

DOI: https://doi.org/10.1364/boe.402847

IF: 3.4

2020-11-19

Biomedical Optics Express

Abstract:A resolution enhancement technique for optical coherence tomography (OCT), based on Generative Adversarial Networks (GANs), was developed and investigated. GANs have been previously used for resolution enhancement of photography and optical microscopy images. We have adapted and improved this technique for OCT image generation. Conditional GANs (cGANs) were trained on a novel set of ultrahigh resolution spectral domain OCT volumes, termed micro-OCT, as the high-resolution ground truth (∼1 <i>μ</i>m isotropic resolution). The ground truth was paired with a low-resolution image obtained by synthetically degrading resolution 4x in one of (1-D) or both axial and lateral axes (2-D). Cross-sectional image (B-scan) volumes obtained from <i>in vivo</i> imaging of human labial (lip) tissue and mouse skin were used in separate feasibility experiments. Accuracy of resolution enhancement compared to ground truth was quantified with human perceptual accuracy tests performed by an OCT expert. The GAN loss in the optimization objective, noise injection in both the generator and discriminator models, and multi-scale discrimination were found to be important for achieving realistic speckle appearance in the generated OCT images. The utility of high-resolution speckle recovery was illustrated by an example of micro-OCT imaging of blood vessels in lip tissue. Qualitative examples applying the models to image data from outside of the training data distribution, namely human retina and mouse bladder, were also demonstrated, suggesting potential for cross-domain transferability. This preliminary study suggests that deep learning generative models trained on OCT images from high-performance prototype systems may have potential in enhancing lower resolution data from mainstream/commercial systems, thereby bringing cutting-edge technology to the masses at low cost.

radiology, nuclear medicine & medical imaging,biochemical research methods,optics
Noise-Powered Disentangled Representation for Unsupervised Speckle Reduction of Optical Coherence Tomography Images

Yongqiang Huang,Wenjun Xia,Zexin Lu,Yan Liu,Hu Chen,Jiliu Zhou,Leyuan Fang,Yi Zhang

DOI: https://doi.org/10.1109/tmi.2020.3045207

IF: 10.6

2021-10-01

IEEE Transactions on Medical Imaging

Abstract:Due to its noninvasive character, optical coherence tomography (OCT) has become a popular diagnostic method in clinical settings. However, the low-coherence interferometric imaging procedure is inevitably contaminated by heavy speckle noise, which impairs both visual quality and diagnosis of various ocular diseases. Although deep learning has been applied for image denoising and achieved promising results, the lack of well-registered clean and noisy image pairs makes it impractical for supervised learning-based approaches to achieve satisfactory OCT image denoising results. In this paper, we propose an unsupervised OCT image speckle reduction algorithm that does not rely on well-registered image pairs. Specifically, by employing the ideas of disentangled representation and generative adversarial network, the proposed method first disentangles the noisy image into content and noise spaces by corresponding encoders. Then, the generator is used to predict the denoised OCT image with the extracted content features. In addition, the noise patches cropped from the noisy image are utilized to facilitate more accurate disentanglement. Extensive experiments have been conducted, and the results suggest that our proposed method is superior to the classic methods and demonstrates competitive performance to several recently proposed learning-based approaches in both quantitative and qualitative aspects. Code is available at: https://github.com/tsmotlp/DRGAN-OCT.

engineering, biomedical,imaging science & photographic technology, electrical & electronic,computer science, interdisciplinary applications,radiology, nuclear medicine & medical imaging
Speckle reduction in laser illuminated endoscopy using adversarial deep learning

Taylor L. Bobrow,Faisal Mahmood,Miguel Inserni,Nicholas J. Durr

DOI: https://doi.org/10.1117/12.2510587

2019-03-04

Abstract:Endoscope size is a major design constraint that must be managed with the clinical demand for high-quality illumination and imaging. Existing commercial endoscopes most often use an arc lamp to produce bright, incoherent white light, requiring large-area fiber bundles to deliver sufficient illumination power to the sample. Moreover, the power instability of these light sources creates challenges for computer vision applications. We demonstrate an alternative illumination technique using red-green-blue laser light and a data-driven approach to combat the speckle noise that is a byproduct of coherent illumination. We frame the speckle artifact problem as an image-to-image translation task solved using conditional Generative Adversarial Networks (cGANs). To train the network, we acquire images illuminated with a coherent laser diode, with a laser diode source made partially- coherent using a laser speckle reducer, and with an incoherent LED light source as the target domain. We train networks using laser-illuminated endoscopic images of ex-vivo, porcine gastrointestinal tissues, augmented by images of laser-illuminated household and laboratory objects. The network is then benchmarked against state of-the-art optical and image processing speckle reduction methods, achieving an increased peak signal-to-noise ratio (PSNR) of 4.1 db, compared to 0.7 dB using optical speckle reduction, 0.6 dB using median filtering, and 0.5 dB using non-local means. This approach not only allows for endoscopes with smaller, more efficient light sources with extremely short triggering times, but it also enables imaging modalities that require both coherent and incoherent sources, such as combined widefield and speckle ow contrast imaging in a single image frame.
Hybrid-structure Network and Network Comparative Study for Deep-Learning-based Speckle-Modulating Optical Coherence Tomography

Guangming Ni,Renxiong Wu,Junming Zhong,Ying Chen,Ling Wan,Yao Xie,Jie Mei,Yong Liu

DOI: https://doi.org/10.1364/oe.454504

IF: 3.8

2022-01-01

Optics Express

Abstract:Optical coherence tomography (OCT), a promising noninvasive bioimaging technique, can resolve sample three-dimensional microstructures. However, speckle noise imposes obvious limitations on OCT resolving capabilities. Here we proposed a deep-learning-based speckle-modulating OCT based on a hybrid-structure network, residual-dense-block U-Net generative adversarial network (RDBU-Net GAN), and further conducted a comprehensively comparative study to explore multi-type deep-learning architectures' abilities to extract speckle pattern characteristics and remove speckle, and resolve microstructures. This is the first time that network comparative study has been performed on a customized dataset containing mass more-general speckle patterns obtained from a custom-built speckle-modulating OCT, but not on retinal OCT datasets with limited speckle patterns. Results demonstrated that the proposed RDBU-Net GAN has a more excellent ability to extract speckle pattern characteristics and remove speckle, and resolve microstructures. This work will be useful for future studies on OCT speckle removing and deep-learning-based speckle-modulating OCT.
Noise Reduction in Optical Coherence Tomography Images Using a Deep Neural Network with Perceptually-Sensitive Loss Function.

Bin Qiu,Zhiyu Huang,Xi Liu,Xiangxi Meng,Yunfei You,Gangjun Liu,Kun Yang,Andreas Maier,Qiushi Ren,Yanye Lu

DOI: https://doi.org/10.1364/boe.379551

IF: 3.4

2020-01-01

Biomedical Optics Express

Abstract:Optical coherence tomography (OCT) is susceptible to the coherent noise, which is the speckle noise that deteriorates contrast and the detail structural information of OCT images, thus imposing significant limitations on the diagnostic capability of OCT. In this paper, we propose a novel OCT image denoising method by using an end-to-end deep learning network with a perceptually-sensitive loss function. The method has been validated on OCT images acquired from healthy volunteers' eyes. The label images for training and evaluating OCT denoising deep learning models are images generated by averaging 50 frames of respective registered B-scans acquired from a region with scans occurring in one direction. The results showed that the new approach can outperform other related denoising methods on the aspects of preserving detail structure information of retinal layers and improving the perceptual metrics in the human visual perception.
Sm-Net OCT: a Deep-Learning-based Speckle-Modulating Optical Coherence Tomography.

Guangming Ni,Ying Chen,Renxiong Wu,Xiaoshan Wang,Ming Zeng,Yong Liu

DOI: https://doi.org/10.1364/oe.431475

IF: 3.8

2021-01-01

Optics Express

Abstract:Speckle imposes obvious limitations on resolving capabilities of optical coherence tomography (OCT), while speckle-modulating OCT can efficiently reduce speckle arbitrarily. However, speckle-modulating OCT seriously reduces the imaging sensitivity and temporal resolution of the OCT system when reducing speckle. Here, we proposed a deep-learning-based speckle-modulating OCT, termed Sm-Net OCT, by deeply integrating conventional OCT setup and generative adversarial network trained with a customized large speckle-modulating OCT dataset containing massive speckle patterns. The customized large speckle-modulating OCT dataset was obtained from the aforementioned conventional OCT setup rebuilt into a speckle-modulating OCT and performed imaging using different scanning parameters. Experimental results demonstrated that the proposed Sm-Net OCT can effectively obtain high-quality OCT images without the electronic noise and speckle, and conquer the limitations of reducing the imaging sensitivity and temporal resolution which conventional speckle-modulating OCT has. The proposed Sm-Net OCT can significantly improve the adaptability and practicality capabilities of OCT imaging, and expand its application fields.
Self-supervised Speckle Noise Reduction of Optical Coherence Tomography Without Clean Data.

Yangxi Li,Yingwei Fan,Hongen Liao

DOI: https://doi.org/10.1364/boe.471497

IF: 3.4

2022-01-01

Biomedical Optics Express

Abstract:Optical coherence tomography (OCT) is widely used in clinical diagnosis due to its non-invasive, real-time, and high-resolution characteristics. However, the inherent speckle noise seriously degrades the image quality, which might damage the fine structures in OCT, thus affecting the diagnosis results. In recent years, supervised deep learning-based denoising methods have shown excellent denoising ability. To train a deep denoiser, a large number of paired noisy-clean images are required, which is difficult to achieve in clinical practice, since acquiring a speckle-free OCT image requires dozens of repeated scans and image registration. In this research, we propose a self-supervised strategy that helps build a despeckling model by training it to map neighboring pixels in a single noisy OCT image. Adjacent pixel patches are randomly selected from the original OCT image to generate two similar undersampled images, which are respectively used as the input and target images for training a deep neural network. To ensure both the despeckling and the structure-preserving effects, a multi-scale pixel patch sampler and corresponding loss functions are adopted in our practice. Through quantitative evaluation and qualitative visual comparison, we found that the proposed method performs better than state-of-the-art methods regarding despeckling effects and structure preservation. Besides, the proposed method is much easier to train and deploy without the need for clean OCT images, which has great significance in clinical practice.
Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss

Qingsong Yang,Pingkun Yan,Yanbo Zhang,Hengyong Yu,Yongyi Shi,Xuanqin Mou,Mannudeep K. Kalra,Yi Zhang,Ling Sun,Ge Wang

DOI: https://doi.org/10.1109/tmi.2018.2827462

IF: 10.6

2018-06-01

IEEE Transactions on Medical Imaging

Abstract:The continuous development and extensive use of computed tomography (CT) in medical practice has raised a public concern over the associated radiation dose to the patient. Reducing the radiation dose may lead to increased noise and artifacts, which can adversely affect the radiologists' judgment and confidence. Hence, advanced image reconstruction from low-dose CT data is needed to improve the diagnostic performance, which is a challenging problem due to its ill-posed nature. Over the past years, various low-dose CT methods have produced impressive results. However, most of the algorithms developed for this application, including the recently popularized deep learning techniques, aim for minimizing the mean-squared error (MSE) between a denoised CT image and the ground truth under generic penalties. Although the peak signal-to-noise ratio is improved, MSE- or weighted-MSE-based methods can compromise the visibility of important structural details after aggressive denoising. This paper introduces a new CT image denoising method based on the generative adversarial network (GAN) with Wasserstein distance and perceptual similarity. The Wasserstein distance is a key concept of the optimal transport theory and promises to improve the performance of GAN. The perceptual loss suppresses noise by comparing the perceptual features of a denoised output against those of the ground truth in an established feature space, while the GAN focuses more on migrating the data noise distribution from strong to weak statistically. Therefore, our proposed method transfers our knowledge of visual perception to the image denoising task and is capable of not only reducing the image noise level but also trying to keep the critical information at the same time. Promising results have been obtained in our experiments with clinical CT images.

engineering, biomedical,imaging science & photographic technology, electrical & electronic,computer science, interdisciplinary applications,radiology, nuclear medicine & medical imaging
Joint Generative Image Deblurring Aided by Edge Attention Prior and Dynamic Kernel Selection

Zhichao Zhang,Hui Chen,Xiaoqing Yin,Jinsheng Deng

DOI: https://doi.org/10.1155/2021/1391801

2021-01-01

Wireless Communications and Mobile Computing

Abstract:Image deblurring is a classic and important problem in industrial fields, such as aviation photo restoration, object recognition in robotics, and autonomous vehicles. Blurry images in real-world scenarios consist of mixed blurring types, such as a natural motion blurring owing to shaking of the camera. Fast deblurring does not deblur the entire image because it is not the best option. Considering the computational costs, it is also better to have an alternative kernel to deblur different objects at a high-semantic level. To achieve better image restoration quality, it is also beneficial to combine the blurring category location and important structural information in terms of specific artifacts and degree of blurring. The goal of blind image deblurring is to restore sharpness from the unknown blurring kernel of an image. Recent deblurring methods tend to reconstruct prior knowledge, neglecting the influence of blur estimation and visual fidelity on image details and structure. Generative adversarial networks(GANs) have recently been attracting considerable attention from both academia and industry because GAN can perfectly generate new data with the same statistics as the training set. Therefore, this study proposes a generative neural architecture and an edge attention algorithm developed to restore vivid multimedia patches. Joint edge generation and image restoration techniques are designed to solve the low-level multimedia retrieval. This multipath refinement fusion network (MRFNet) can not only perform deblurring of images directly but also individual the frames separately from videos. Ablation experiments validate that our generative adversarial network MRFNet performs better in joint training than in multimodel. Compared to other GAN methods, our two-phase method exhibited state-of-the-art performance in terms of speed and accuracy as well as has a significant visual improvement.
Ultrasound image denoising using generative adversarial networks with residual dense connectivity and weighted joint loss

Lun Zhang,Junhua Zhang

DOI: https://doi.org/10.7717/peerj-cs.873

2022-02-16

PeerJ Computer Science

Abstract:Background Ultrasound imaging has been recognized as a powerful tool in clinical diagnosis. Nonetheless, the presence of speckle noise degrades the signal-to-noise of ultrasound images. Various denoising algorithms cannot fully reduce speckle noise and retain image features well for ultrasound imaging. The application of deep learning in ultrasound image denoising has attracted more and more attention in recent years. Methods In the article, we propose a generative adversarial network with residual dense connectivity and weighted joint loss (GAN-RW) to avoid the limitations of traditional image denoising algorithms and surpass the most advanced performance of ultrasound image denoising. The denoising network is based on U-Net architecture which includes four encoder and four decoder modules. Each of the encoder and decoder modules is replaced with residual dense connectivity and BN to remove speckle noise. The discriminator network applies a series of convolutional layers to identify differences between the translated images and the desired modality. In the training processes, we introduce a joint loss function consisting of a weighted sum of the L1 loss function, binary cross-entropy with a logit loss function and perceptual loss function. Results We split the experiments into two parts. First, experiments were performed on Berkeley segmentation (BSD68) datasets corrupted by a simulated speckle. Compared with the eight existing denoising algorithms, the GAN-RW achieved the most advanced despeckling performance in terms of the peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and subjective visual effect. When the noise level was 15, the average value of the GAN-RW increased by approximately 3.58% and 1.23% for PSNR and SSIM, respectively. When the noise level was 25, the average value of the GAN-RW increased by approximately 3.08% and 1.84% for PSNR and SSIM, respectively. When the noise level was 50, the average value of the GAN-RW increased by approximately 1.32% and 1.98% for PSNR and SSIM, respectively. Secondly, experiments were performed on the ultrasound images of lymph nodes, the foetal head, and the brachial plexus. The proposed method shows higher subjective visual effect when verifying on the ultrasound images. In the end, through statistical analysis, the GAN-RW achieved the highest mean rank in the Friedman test.

computer science, information systems, artificial intelligence, theory & methods
SNR-Net OCT: Brighten and Denoise Low-Light Optical Coherence Tomography Images Via Deep Learning.

Shaoyan Huang,Rong Wang,Renxiong Wu,Junming Zhong,Xin Ge,Yong Liu,Guangming Ni

DOI: https://doi.org/10.1364/oe.491391

IF: 3.8

2023-01-01

Optics Express

Abstract:Low-light optical coherence tomography (OCT) images generated when using low input power, low-quantum-efficiency detection units, low exposure time, or facing high-reflective surfaces, have low bright and signal-to-noise rates (SNR), and restrict OCT technique and clinical applications. While low input power, low quantum efficiency, and low exposure time can help reduce the hardware requirements and accelerate imaging speed; high-reflective surfaces are unavoidable sometimes. Here we propose a deep-learning-based technique to brighten and denoise low-light OCT images, termed SNR-Net OCT. The proposed SNR-Net OCT deeply integrated a conventional OCT setup and a residual-dense-block U-Net generative adversarial network with channel-wise attention connections trained using a customized large speckle-free SNR-enhanced brighter OCT dataset. Results demonstrated that the proposed SNR-Net OCT can brighten low-light OCT images and remove the speckle noise effectively, with enhancing SNR and maintaining the tissue microstructures well. Moreover, compared to the hardware-based techniques, the proposed SNR-Net OCT can be of lower cost and better performance.
Laser Speckle Denoising with Deep Convolutional Network

Namulun,Hong Zhao,Tao Yang,Xiang Zhou

DOI: https://doi.org/10.1117/12.2572967

2020-01-01

Abstract:Laser speckle is a long-standing issue which will corrupt image formation and interpretation. Over the years, various techniques have been developed to mitigate the speckle noise. Although machine learning approaches have been widely applied for image denoising tasks, there are very few of them are specifically designed for speckle reduction. In this work, we present a network specialized for reducing speckle noise in images. The training set consists of nearly 3,000 coherent- and incoherent-illuminated image pairs of a variety objects. The network is trained to learn a transformation from speckled to speckle-free images. We compare against the traditional image processing methods, and show that our learning-based approach outperforms these methods regarding both high PSNR, SSIM results and maintaining high-frequency edge features. Our data-driven approach dramatically reduces laser speckle noise by 11.71 dB, compared to a 0.17 dB reduction from non-local means filtering, a 0.1 dB reduction from median filtering and a 0.12 dB reduction from Gaussian filtering. Moreover, conventional image processing approaches reduce both laser speckle and high-frequency image features, which will result in the blurring effect. In contrast with optical speckle-denoising approaches, our method reduces cost and computational complexity. So, applications that require small and bright illumination sources with high-quality imaging can benefit from our work.
Applying a Pix2Pix Generative Adversarial Network to a Fourier-Domain Optical Coherence Tomography System for Artifact Elimination

Chun-Ming Huang,Eddy Wijanto,Hsu-Chih Cheng

DOI: https://doi.org/10.1109/access.2021.3098865

IF: 3.9

2021-01-01

IEEE Access

Abstract:The presence of artifacts, including conjugate, DC, and auto-correlation artifacts, is a critical limitation of Fourier-domain optical coherence tomography (FD-OCT). Many methods have been proposed to resolve this problem to obtain high-quality images. Furthermore, the development of deep learning has resulted in many prospective advancements in the medical field; image-to-image translation by using generative adversarial networks (GANs) is one such advancement. In this study, we propose applying the Pix2Pix GAN to eliminate artifacts from FD-OCT images. The first experiment results showed that the proposed framework could translate conventional FD-OCT depth profiles into artifact-free FD-OCT depth profiles. In addition, the FD-OCT depth profile and optical distance of translated images matched those of ground truth images. Second experiment verified that the proposed GAN-based FD-OCT can be applied to generate artifact-free FD-OCT image with different parameters of sample refractive index, the front surface of the sample toward the zero-delay position, and the physical thickness of the sample. Third experiment proved that the proposed model could translated the conventional FD-OCT depth profiles with additional Gaussian noises source image into artifacts-free FD-OCT and successfully relieved the noise.

computer science, information systems,telecommunications,engineering, electrical & electronic
Generative Low-Dose Ct Image Denoising

Qingsong Yang,Pingkun Yan,Yanbo Zhang,Hengyong Yu,Yongyi Shi,Xuanqin Mou,Mannudeep K. Kalra,Yi Zhang,Ling Sun,Ge Wang

DOI: https://doi.org/10.1007/978-3-030-13969-8_14

2019-01-01

Abstract:The continuous development and extensive use of CT in medical practice have raised a public concern over the associated radiation dose to patients. Reducing the radiation dose may lead to increased noise and artifacts, which can adversely affect radiologists' judgment and confidence. Hence, advanced image reconstruction from low-dose CT data is needed to improve the diagnostic performance, which is a challenging problem due to its ill-posed nature. Over the past years, various low-dose CT methods have produced impressive results. However, most of the algorithms developed for this application, including the recently popularized deep learning techniques, aim for minimizing the mean squared error (MSE) between a denoised CT image and the ground truth under generic penalties. Although the peak signal-to-noise ratio (PSNR) is improved, MSE- or weighted-MSE-based methods can compromise the visibility of important structural details after aggressive denoising. This paper introduces a new CT image denoising method based on the generative adversarial network (GAN) with Wasserstein distance and perceptual similarity. The Wasserstein distance is a key concept of the optimal transport theory, and promises to improve the performance of GAN. The perceptual loss suppresses noise by comparing the perceptual features of a denoised output against those of the ground truth in an established feature space, while the GAN focuses more on migrating the data noise distribution from strong to weak statistically. Therefore, our proposed method transfers our knowledge of visual perception to the image denoising task and is capable of not only reducing the image noise level but also trying to keep the critical information at the same time. Promising results have been obtained in our experiments with clinical CT images.
SpeckleGAN: a generative adversarial network with an adaptive speckle layer to augment limited training data for ultrasound image processing

Lennart Bargsten,Alexander Schlaefer

DOI: https://doi.org/10.1007/s11548-020-02203-1

2020-06-18

International Journal of Computer Assisted Radiology and Surgery

Abstract:Abstract Purpose In the field of medical image analysis, deep learning methods gained huge attention over the last years. This can be explained by their often improved performance compared to classic explicit algorithms. In order to work well, they need large amounts of annotated data for supervised learning, but these are often not available in the case of medical image data. One way to overcome this limitation is to generate synthetic training data, e.g., by performing simulations to artificially augment the dataset. However, simulations require domain knowledge and are limited by the complexity of the underlying physical model. Another method to perform data augmentation is the generation of images by means of neural networks. Methods We developed a new algorithm for generation of synthetic medical images exhibiting speckle noise via generative adversarial networks (GANs). Key ingredient is a speckle layer, which can be incorporated into a neural network in order to add realistic and domain-dependent speckle. We call the resulting GAN architecture SpeckleGAN. Results We compared our new approach to an equivalent GAN without speckle layer. SpeckleGAN was able to generate ultrasound images with very crisp speckle patterns in contrast to the baseline GAN, even for small datasets of 50 images. SpeckleGAN outperformed the baseline GAN by up to 165 % with respect to the Fréchet Inception distance. For artery layer and lumen segmentation, a performance improvement of up to 4 % was obtained for small datasets, when these were augmented with images by SpeckleGAN. Conclusion SpeckleGAN facilitates the generation of realistic synthetic ultrasound images to augment small training sets for deep learning based image processing. Its application is not restricted to ultrasound images but could be used for every imaging methodology that produces images with speckle such as optical coherence tomography or radar.
Speckle Noise Reduction for Medical Ultrasound Images Based on Cycle-Consistent Generative Adversarial Network

Jieyi Liu,Changchun Li,Liping Liu,Haobo Chen,Hong Han,Bo Zhang,Qi Zhang

DOI: https://doi.org/10.1016/j.bspc.2023.105150

IF: 5.1

2023-01-01

Biomedical Signal Processing and Control

Abstract:Medical ultrasound (US) images are corrupted by speckle noise, which can adversely affect the disease diagnosis and treatment. Recently, the cycle-consistent adversarial network (CycleGAN) has been used in the style transfer for both natural and medical images. In this study, we aim to develop a US despeckling method based on the CycleGAN by the style transfer between the speckled noisy data domain and noise-free data domain by forming a bi-directional universal mapping. The inputs of noisy and noise-free images are designed in the CycleGAN model. For simulation work, we use both the multiplicative model and the spatial impulse response model to obtain noisy images from noise-free images. However, noise-free US images are not clinically available. Hence, for the real US despeckling scenario, the clinical images of hearts, lymph nodes, and breast tumors are used as noisy images; and the high-quality images that are derived from the clinical images by despeckling with the Gaborbased anisotropic diffusion (GAD) and selected with a new metric named the edge-to-noise ratio, are used as the noise-free images. We compare our CycleGAN based denoising method with nine existing denoising methods, i.e., the speckle reduction anisotropic diffusion, GAD, non-local means, wavelet transform, unbiased non-local means, statistical nearest-neighbors, TVHTVM, improved non-local self-similarity measures, and generative adversarial network. Our method outperforms other methods by visual assessment and quantitative comparison, which demonstrates its superiority for noise reduction and detail preservation.
Comparative Study of Deep Neural Networks with Unsupervised Noise2Noise Strategy for Noise Reduction of Optical Coherence Tomography Images.

Bin Qiu,Shuang Zeng,Xiangxi Meng,Zhe Jiang,Yunfei You,Mufeng Geng,Ziyuan Li,Yicheng Hu,Zhiyu Huang,Chuanqing Zhou,Qiushi Ren,Yanye Lu

DOI: https://doi.org/10.1002/jbio.202100151

IF: 3.3899

2021-01-01

Journal of Biophotonics

Abstract:As a powerful diagnostic tool, optical coherence tomography (OCT) has been widely used in various clinical setting. However, OCT images are susceptible to inherent speckle noise that may contaminate subtle structure information, due to low-coherence interferometric imaging procedure. Many supervised learning-based models have achieved impressive performance in reducing speckle noise of OCT images trained with a large number of noisy-clean paired OCT images, which are not commonly feasible in clinical practice. In this article, we conducted a comparative study to investigate the denoising performance of OCT images over different deep neural networks through an unsupervised Noise2Noise (N2N) strategy, which only trained with noisy OCT samples. Four representative network architectures including U-shaped model, multi-information stream model, straight-information stream model and GAN-based model were investigated on an OCT image dataset acquired from healthy human eyes. The results demonstrated all four unsupervised N2N models offered denoised OCT images with a performance comparable with that of supervised learning models, illustrating the effectiveness of unsupervised N2N models in denoising OCT images. Furthermore, U-shaped models and GAN-based models using UNet network as generator are two preferred and suitable architectures for reducing speckle noise of OCT images and preserving fine structure information of retinal layers under unsupervised N2N circumstances.
Creating Realistic Anterior Segment Optical Coherence Tomography Images using Generative Adversarial Networks

Jad F. Assaf,Anthony Abou Mrad,Dan Z. Reinstein,Guillermo Amescua,Cyril Zakka,Timothy Archer,Jeffrey Yammine,Elsa Lamah,Michèle Haykal,Shady T. Awwad

DOI: https://doi.org/10.1136/bjo-2023-324633

2023-06-25

Abstract:This paper presents the development and validation of a Generative Adversarial Network (GAN) purposed to create high-resolution, realistic Anterior Segment Optical Coherence Tomography (AS-OCT) images. We trained the Style and WAvelet based GAN (SWAGAN) on 142,628 AS-OCT B-scans. Three experienced refractive surgeons performed a blinded assessment to evaluate the realism of the generated images; their results were not significantly better than chance in distinguishing between real and synthetic images, thus demonstrating a high degree of image realism. To gauge their suitability for machine learning tasks, a convolutional neural network (CNN) classifier was trained with a dataset containing both real and GAN-generated images. The CNN demonstrated an accuracy rate of 78% trained on real images alone, but this accuracy rose to 100% when training included the generated images. This underscores the utility of synthetic images for machine learning applications. We further improved the resolution of the generated images by up-sampling them twice (2x) using an Enhanced Super Resolution GAN (ESRGAN), which outperformed traditional up-sampling techniques. In conclusion, GANs can effectively generate high-definition, realistic AS-OCT images, proving highly beneficial for machine learning and image analysis tasks.

Image and Video Processing,Computer Vision and Pattern Recognition

De-speckling of Optical Coherence Tomography Images Using a Generative Adversarial Network

Probabilistic volumetric speckle suppression in OCT using deep learning

Improved Sparse Representation Algorithm for Optical Coherence Tomography Images

Resolution enhancement and realistic speckle recovery with generative adversarial modeling of micro-optical coherence tomography

Noise-Powered Disentangled Representation for Unsupervised Speckle Reduction of Optical Coherence Tomography Images

Speckle reduction in laser illuminated endoscopy using adversarial deep learning

Hybrid-structure Network and Network Comparative Study for Deep-Learning-based Speckle-Modulating Optical Coherence Tomography

Noise Reduction in Optical Coherence Tomography Images Using a Deep Neural Network with Perceptually-Sensitive Loss Function.

Sm-Net OCT: a Deep-Learning-based Speckle-Modulating Optical Coherence Tomography.

Self-supervised Speckle Noise Reduction of Optical Coherence Tomography Without Clean Data.

Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss

Joint Generative Image Deblurring Aided by Edge Attention Prior and Dynamic Kernel Selection

Ultrasound image denoising using generative adversarial networks with residual dense connectivity and weighted joint loss

SNR-Net OCT: Brighten and Denoise Low-Light Optical Coherence Tomography Images Via Deep Learning.

Laser Speckle Denoising with Deep Convolutional Network

Applying a Pix2Pix Generative Adversarial Network to a Fourier-Domain Optical Coherence Tomography System for Artifact Elimination

Generative Low-Dose Ct Image Denoising

SpeckleGAN: a generative adversarial network with an adaptive speckle layer to augment limited training data for ultrasound image processing

Speckle Noise Reduction for Medical Ultrasound Images Based on Cycle-Consistent Generative Adversarial Network

Comparative Study of Deep Neural Networks with Unsupervised Noise2Noise Strategy for Noise Reduction of Optical Coherence Tomography Images.

Creating Realistic Anterior Segment Optical Coherence Tomography Images using Generative Adversarial Networks