[A case of non-endemic Burkitt's lymphoma of the colon].

M. Scaglione,L. Romano,F. Pinto,F. Flagiello,T. Valente,A. Forner,L. Vicenzo

1997-04-01

La radiologia medica

Abstract:

What problem does this paper attempt to address?

OA-GAN: Organ-Aware Generative Adversarial Network for Synthesizing Contrast-Enhanced Medical Images

Yulin Yang,Jing Liu,Gan Zhan,Qingqing Chen,Fang Wang,Yinhao Li,Rahul Kumar Jain,Lanfen Lin,Hongjie Hu,Yen-Wei Chen

DOI: https://doi.org/10.1088/2057-1976/ad31fa

2024-01-01

Biomedical Physics & Engineering Express

Abstract:Contrast-enhanced computed tomography (CE-CT) images are vital for clinical diagnosis of focal liver lesions (FLLs). However, the use of CE-CT images imposes a significant burden on patients due to the injection of contrast agents and extended shooting. Deep learning-based image synthesis models offer a promising solution that synthesizes CE-CT images from non-contrasted CT (NC-CT) images. Unlike natural images, medical image synthesis requires a specific focus on certain organs or localized regions to ensure accurate diagnosis. Determining how to effectively emphasize target organs poses a challenging issue in medical image synthesis. To solve this challenge, we present a novel CE-CT image synthesis model called, Organ-Aware Generative Adversarial Network (OA-GAN). The OA-GAN comprises an organ-aware (OA) network and a dual decoder-based generator. First, the OA network learns the most discriminative spatial features about the target organ (i.e. liver) by utilizing the ground truth organ mask as localization cues. Subsequently, NC-CT image and captured feature are fed into the dual decoder-based generator, which employs a local and global decoder network to simultaneously synthesize the organ and entire CECT image. Moreover, the semantic information extracted from the local decoder is transferred to the global decoder to facilitate better reconstruction of the organ in entire CE-CT image. The qualitative and quantitative evaluation on a CE-CT dataset demonstrates that the OA-GAN outperforms state-of-the-art approaches for synthesizing two types of CE-CT images such as arterial phase and portal venous phase. Additionally, subjective evaluations by expert radiologists and a deep learning-based FLLs classification also affirm that CE-CT images synthesized from the OA-GAN exhibit a remarkable resemblance to real CE-CT images.
Shape-Consistent Generative Adversarial Networks for Multi-Modal Medical Segmentation Maps

Leo Segre,Or Hirschorn,Dvir Ginzburg,Dan Raviv

DOI: https://doi.org/10.1109/isbi52829.2022.9761478

2022-03-28

Abstract:Image translation across domains for unpaired datasets has gained interest and great improvement lately. In medical imaging, there are multiple imaging modalities, with very different characteristics. Our goal is to use cross-modality adaptation between CT and MRI whole cardiac scans for semantic segmentation. We present a segmentation network using synthesised cardiac volumes for extremely limited datasets. Our solution is based on a 3D cross-modality generative adversarial network to share information between modalities and generate synthesized data using unpaired datasets. Our network utilizes semantic segmentation to improve generator shape consistency, thus creating more realistic synthesised volumes to be used when re-training the segmentation network. We show that improved segmentation can be achieved on small datasets when using spatial augmentations to improve a generative adversarial network. These augmentations improve the generator capabilities, thus enhancing the performance of the Segmentor. Using only 16 CT and 16 MRI cardiovascular volumes, improved results are shown over other segmentation methods while using the suggested architecture. Our code is publicly available1.
Cross-Modal Attention-Guided Convolutional Network for Multi-modal Cardiac Segmentation.

Ziqi Zhou,Xinna Guo,Wanqi Yang,Yinghuan Shi,Luping Zhou,Lei Wang,Ming Yang

DOI: https://doi.org/10.1007/978-3-030-32692-0_69

2019-01-01

Abstract:To leverage the correlated information between modalities to benefit the cross-modal segmentation, we propose a novel cross-modal attention-guided convolutional network for multi-modal cardiac segmentation. In particular, we first employed the cycle-consistency generative adversarial networks to complete the bidirectional image generation (i.e., MR to CT, CT to MR) to help reduce the modal-level inconsistency. Then, with the generated and original MR and CT images, a novel convolutional network is proposed where (1) two encoders learn individual features separately and (2) a common decoder learns shareable features between modalities for a final consistent segmentation. Also, we propose a cross-modal attention module between the encoders and decoder in order to leverage the correlated information between modalities. Our model can be trained in an end-to-end manner. With extensive evaluation on the unpaired CT and MR cardiac images, our method outperforms the baselines in terms of the segmentation performance.
SA-GAN: Structure-Aware GAN for Organ-Preserving Synthetic CT Generation

Hajar Emami,Ming Dong,Siamak Nejad-Davarani,Carri Glide-Hurst

DOI: https://doi.org/10.48550/arXiv.2105.07044

2021-05-14

Image and Video Processing

Abstract:In medical image synthesis, model training could be challenging due to the inconsistencies between images of different modalities even with the same patient, typically caused by internal status/tissue changes as different modalities are usually obtained at a different time. This paper proposes a novel deep learning method, Structure-aware Generative Adversarial Network (SA-GAN), that preserves the shapes and locations of in-consistent structures when generating medical images. SA-GAN is employed to generate synthetic computed tomography (synCT) images from magnetic resonance imaging (MRI) with two parallel streams: the global stream translates the input from the MRI to the CT domain while the local stream automatically segments the inconsistent organs, maintains their locations and shapes in MRI, and translates the organ intensities to CT. Through extensive experiments on a pelvic dataset, we demonstrate that SA-GAN provides clinically acceptable accuracy on both synCTs and organ segmentation and supports MR-only treatment planning in disease sites with internal organ status changes.
SynSeg-Net: Synthetic Segmentation Without Target Modality Ground Truth

Yuankai Huo,Zhoubing Xu,Hyeonsoo Moon,Shunxing Bao,Albert Assad,Tamara K. Moyo,Michael R. Savona,Richard G. Abramson,Bennett A. Landman

DOI: https://doi.org/10.1109/TMI.2018.2876633

2019-09-28

Abstract:A key limitation of deep convolutional neural networks (DCNN) based image segmentation methods is the lack of generalizability. Manually traced training images are typically required when segmenting organs in a new imaging modality or from distinct disease cohort. The manual efforts can be alleviated if the manually traced images in one imaging modality (e.g., MRI) are able to train a segmentation network for another imaging modality (e.g., CT). In this paper, we propose an end-to-end synthetic segmentation network (SynSeg-Net) to train a segmentation network for a target imaging modality without having manual labels. SynSeg-Net is trained by using (1) unpaired intensity images from source and target modalities, and (2) manual labels only from source modality. SynSeg-Net is enabled by the recent advances of cycle generative adversarial networks (CycleGAN) and DCNN. We evaluate the performance of the SynSeg-Net on two experiments: (1) MRI to CT splenomegaly synthetic segmentation for abdominal images, and (2) CT to MRI total intracranial volume synthetic segmentation (TICV) for brain images. The proposed end-to-end approach achieved superior performance to two stage methods. Moreover, the SynSeg-Net achieved comparable performance to the traditional segmentation network using target modality labels in certain scenarios. The source code of SynSeg-Net is publicly available (<a class="link-external link-https" href="https://github.com/MASILab/SynSeg-Net" rel="external noopener nofollow">this https URL</a>).

Computer Vision and Pattern Recognition
Label-informed cardiac magnetic resonance image synthesis through conditional generative adversarial networks

Sina Amirrajab,Yasmina Al Khalil,Cristian Lorenz,Jürgen Weese,Josien Pluim,Marcel Breeuwer

DOI: https://doi.org/10.1016/j.compmedimag.2022.102123

Abstract:Synthesis of a large set of high-quality medical images with variability in anatomical representation and image appearance has the potential to provide solutions for tackling the scarcity of properly annotated data in medical image analysis research. In this paper, we propose a novel framework consisting of image segmentation and synthesis based on mask-conditional GANs for generating high-fidelity and diverse Cardiac Magnetic Resonance (CMR) images. The framework consists of two modules: i) a segmentation module trained using a physics-based simulated database of CMR images to provide multi-tissue labels on real CMR images, and ii) a synthesis module trained using pairs of real CMR images and corresponding multi-tissue labels, to translate input segmentation masks to realistic-looking cardiac images. The anatomy of synthesized images is based on labels, whereas the appearance is learned from the training images. We investigate the effects of the number of tissue labels, quantity of training data, and multi-vendor data on the quality of the synthesized images. Furthermore, we evaluate the effectiveness and usability of the synthetic data for a downstream task of training a deep-learning model for cardiac cavity segmentation in the scenarios of data replacement and augmentation. The results of the replacement study indicate that segmentation models trained with only synthetic data can achieve comparable performance to the baseline model trained with real data, indicating that the synthetic data captures the essential characteristics of its real counterpart. Furthermore, we demonstrate that augmenting real with synthetic data during training can significantly improve both the Dice score (maximum increase of 4%) and Hausdorff Distance (maximum reduction of 40%) for cavity segmentation, suggesting a good potential to aid in tackling medical data scarcity.
Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonization

Mengwei Ren,Neel Dey,James Fishbaugh,Guido Gerig

DOI: https://doi.org/10.1109/tmi.2021.3059726

IF: 10.6

2021-06-01

IEEE Transactions on Medical Imaging

Abstract:Deep networks are now ubiquitous in large-scale multi-center imaging studies. However, the direct aggregation of images across sites is contraindicated for downstream statistical and deep learning-based image analysis due to inconsistent contrast, resolution, and noise. To this end, in the absence of paired data, variations of Cycle-consistent Generative Adversarial Networks have been used to harmonize image sets between a source and target domain. Importantly, these methods are prone to instability, contrast inversion, intractable manipulation of pathology, and steganographic mappings which limit their reliable adoption in real-world medical imaging. In this work, based on an underlying assumption that morphological shape is consistent across imaging sites, we propose a segmentation-renormalized image translation framework to reduce inter-scanner heterogeneity while preserving anatomical layout. We replace the affine transformations used in the normalization layers within generative networks with trainable scale and shift parameters conditioned on jointly learned anatomical segmentation embeddings to modulate features at every level of translation. We evaluate our methodologies against recent baselines across several imaging modalities (T1w MRI, FLAIR MRI, and OCT) on datasets with and without lesions. Segmentation-renormalization for translation GANs yields superior image harmonization as quantified by Inception distances, demonstrates improved downstream utility via post-hoc segmentation accuracy, and improved robustness to translation perturbation and self-adversarial attacks.
Multi-Constraint Transferable Generative Adversarial Networks for Cross-Modal Brain Image Synthesis

Yawen Huang,Hao Zheng,Yuexiang Li,Feng Zheng,Xiantong Zhen,GuoJun Qi,Ling Shao,Yefeng Zheng

DOI: https://doi.org/10.1007/s11263-024-02109-4

IF: 13.369

2024-05-30

International Journal of Computer Vision

Abstract:Recent progress in generative models has led to the drastic growth of research in image generation. Existing approaches show visually compelling results by learning multi-modal distributions, but they still lack realism, especially in certain scenarios like medical image synthesis. In this paper, we propose a novel Brain Generative Adversarial Network (BrainGAN) that explores GANs with multi-constraint and transferable property for cross-modal brain image synthesis. We formulate BrainGAN by introducing a unified framework with new constraints that can enhance modal matching, texture details and anatomical structure, simultaneously. We show how BrainGAN can learn meaningful tissue representations with rich variability of brain images. In addition to generating 3D volumes that are visually indistinguishable from real ones, we model adversarial discriminators and segmentors jointly, along with the proposed cost functions, which forces our networks to synthesize brain MRIs with realistic textures conditioned on anatomical structures. BrainGAN is evaluated on three public datasets, where it consistently outperforms the other state-of-the-art approaches by a large margin, advancing cross-modal synthesis of brain images both visually and practically.

computer science, artificial intelligence
Segmentation Guided Crossing Dual Decoding Generative Adversarial Network for Synthesizing Contrast-Enhanced Computed Tomography Images

Yulin Yang,Qingqing Chen,Yinhao Li,Fang Wang,Xian-Hua Han,Yutaro Iwamoto,Jing Liu,Lanfen Lin,Hongjie Hu,Yen-Wei Chen

DOI: https://doi.org/10.1109/JBHI.2024.3403199

Abstract:Although contrast-enhanced computed tomography (CE-CT) images significantly improve the accuracy of diagnosing focal liver lesions (FLLs), the administration of contrast agents imposes a considerable physical burden on patients. The utilization of generative models to synthesize CE-CT images from non-contrasted CT images offers a promising solution. However, existing image synthesis models tend to overlook the importance of critical regions, inevitably reducing their effectiveness in downstream tasks. To overcome this challenge, we propose an innovative CE-CT image synthesis model called Segmentation Guided Crossing Dual Decoding Generative Adversarial Network (SGCDD-GAN). Specifically, the SGCDD-GAN involves a crossing dual decoding generator including an attention decoder and an improved transformation decoder. The attention decoder is designed to highlight some critical regions within the abdominal cavity, while the improved transformation decoder is responsible for synthesizing CE-CT images. These two decoders are interconnected using a crossing technique to enhance each other's capabilities. Furthermore, we employ a multi-task learning strategy to guide the generator to focus more on the lesion area. To evaluate the performance of proposed SGCDD-GAN, we test it on an in-house CE-CT dataset. In both CE-CT image synthesis tasks-namely, synthesizing ART images and synthesizing PV images-the proposed SGCDD-GAN demonstrates superior performance metrics across the entire image and liver region, including SSIM, PSNR, MSE, and PCC scores. Furthermore, CE-CT images synthetized from our SGCDD-GAN achieve remarkable accuracy rates of 82.68%, 94.11%, and 94.11% in a deep learning-based FLLs classification task, along with a pilot assessment conducted by two radiologists.
SUSAN: Segment Unannotated image Structure using Adversarial Network

Fang Liu

DOI: https://doi.org/10.48550/arXiv.1812.00555

2018-12-03

Abstract:Segmentation of magnetic resonance (MR) images is a fundamental step in many medical imaging-based applications. The recent implementation of deep convolutional neural networks (CNNs) in image processing has been shown to have significant impacts on medical image segmentation. Network training of segmentation CNNs typically requires images and paired annotation data representing pixel-wise tissue labels referred to as masks. However, the supervised training of highly efficient CNNs with deeper structure and more network parameters requires a large number of training images and paired tissue masks. Thus, there is great need to develop a generalized CNN-based segmentation method which would be applicable for a wide variety of MR image datasets with different tissue contrasts. The purpose of this study was to develop and evaluate a generalized CNN-based method for fully-automated segmentation of different MR image datasets using a single set of annotated training data. A technique called cycle-consistent generative adversarial network (CycleGAN) is applied as the core of the proposed method to perform image-to-image translation between MR image datasets with different tissue contrasts. A joint segmentation network is incorporated into the adversarial network to obtain additional segmentation functionality. The proposed method was evaluated for segmenting bone and cartilage on two clinical knee MR image datasets acquired at our institution using only a single set of annotated data from a publicly available knee MR image dataset. The new technique may further improve the applicability and efficiency of CNN-based segmentation of medical images while eliminating the need for large amounts of annotated training data.

Computer Vision and Pattern Recognition,Artificial Intelligence
Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptation

Devavrat Tomar,Manana Lortkipanidze,Guillaume Vray,Behzad Bozorgtabar,Jean-Philippe Thiran

DOI: https://doi.org/10.1109/TMI.2021.3059265

2021-03-06

Abstract:Despite the successes of deep neural networks on many challenging vision tasks, they often fail to generalize to new test domains that are not distributed identically to the training data. The domain adaptation becomes more challenging for cross-modality medical data with a notable domain shift. Given that specific annotated imaging modalities may not be accessible nor complete. Our proposed solution is based on the cross-modality synthesis of medical images to reduce the costly annotation burden by radiologists and bridge the domain gap in radiological images. We present a novel approach for image-to-image translation in medical images, capable of supervised or unsupervised (unpaired image data) setups. Built upon adversarial training, we propose a learnable self-attentive spatial normalization of the deep convolutional generator network's intermediate activations. Unlike previous attention-based image-to-image translation approaches, which are either domain-specific or require distortion of the source domain's structures, we unearth the importance of the auxiliary semantic information to handle the geometric changes and preserve anatomical structures during image translation. We achieve superior results for cross-modality segmentation between unpaired MRI and CT data for multi-modality whole heart and multi-modal brain tumor MRI (T1/T2) datasets compared to the state-of-the-art methods. We also observe encouraging results in cross-modality conversion for paired MRI and CT images on a brain dataset. Furthermore, a detailed analysis of the cross-modality image translation, thorough ablation studies confirm our proposed method's efficacy.

Computer Vision and Pattern Recognition,Image and Video Processing
Generalizable Cross-modality Medical Image Segmentation Via Style Augmentation and Dual Normalization

Ziqi Zhou,Lei Qi,Xin Yang,Dong Ni,Yinghuan Shi

DOI: https://doi.org/10.1109/cvpr52688.2022.02019

2022-01-01

Abstract:For medical image segmentation, imagine if a model was only trained using MR images in source domain, how about its performance to directly segment CT images in target domain? This setting, namely generalizable cross-modality segmentation, owning its clinical potential, is much more challenging than other related settings, e.g., domain adaptation. To achieve this goal, we in this paper propose a novel dual-normalization model by leveraging the augmented source-similar and source-dissimilar images during our generalizable segmentation. To be specific, given a single source domain, aiming to simulate the possible appearance change in unseen target domains, we first utilize a nonlinear transformation to augment source-similar and source-dissimilar images. Then, to sufficiently exploit these two types of augmentations, our proposed dualnormalization based model employs a shared backbone yet independent batch normalization layer for separate normalization. Afterward, we put forward a style-based selection scheme to automatically choose the appropriate path in the test stage. Extensive experiments on three publicly available datasets, i.e., BraTS, Cross-Modality Cardiac, and Abdominal Multi-Organ datasets, have demonstrated that our method outperforms other state-of-the-art domain generalization methods. Code is available at https://github.com/zzzqzhou/Dual-Normalization.
ReeGAN: MRI image edge-preserving synthesis based on GANs trained with misaligned data

Xiangjiang Lu,Xiaoshuang Liang,Wenjing Liu,Xiuxia Miao,Xianglong Guan

DOI: https://doi.org/10.1007/s11517-024-03035-w

Abstract:As a crucial medical examination technique, different modalities of magnetic resonance imaging (MRI) complement each other, offering multi-angle and multi-dimensional insights into the body's internal information. Therefore, research on MRI cross-modality conversion is of great significance, and many innovative techniques have been explored. However, most methods are trained on well-aligned data, and the impact of misaligned data has not received sufficient attention. Additionally, many methods focus on transforming the entire image and ignore crucial edge information. To address these challenges, we propose a generative adversarial network based on multi-feature fusion, which effectively preserves edge information while training on noisy data. Notably, we consider images with limited range random transformations as noisy labels and use an additional small auxiliary registration network to help the generator adapt to the noise distribution. Moreover, we inject auxiliary edge information to improve the quality of synthesized target modality images. Our goal is to find the best solution for cross-modality conversion. Comprehensive experiments and ablation studies demonstrate the effectiveness of the proposed method.
CycleSGAN: A cycle-consistent and semantics-preserving generative adversarial network for unpaired MR-to-CT image synthesis

Runze Wang,Alexander F Heimann,Moritz Tannast,Guoyan Zheng

DOI: https://doi.org/10.1016/j.compmedimag.2024.102431

2024-09-04

Abstract:CycleGAN has been leveraged to synthesize a CT image from an available MR image after trained on unpaired data. Due to the lack of direct constraints between the synthetic and the input images, CycleGAN cannot guarantee structural consistency and often generates inaccurate mappings that shift the anatomy, which is highly undesirable for downstream clinical applications such as MRI-guided radiotherapy treatment planning and PET/MRI attenuation correction. In this paper, we propose a cycle-consistent and semantics-preserving generative adversarial network, referred as CycleSGAN, for unpaired MR-to-CT image synthesis. Our design features a novel and generic way to incorporate semantic information into CycleGAN. This is done by designing a pair of three-player games within the CycleGAN framework where each three-player game consists of one generator and two discriminators to formulate two distinct types of adversarial learning: appearance adversarial learning and structure adversarial learning. These two types of adversarial learning are alternately trained to ensure both realistic image synthesis and semantic structure preservation. Results on unpaired hip MR-to-CT image synthesis show that our method produces better synthetic CT images in both accuracy and visual quality as compared to other state-of-the-art (SOTA) unpaired MR-to-CT image synthesis methods.
Generalizable Cross-modality Medical Image Segmentation via Style Augmentation and Dual Normalization

Zhou Zhang,Qi Liu,Xin Yang,Dong Ni,Yinghuan Shi

DOI: https://doi.org/10.48550/arxiv.2112.11177

2021-01-01

Abstract:For medical image segmentation, imagine if a model was only trained using MR images in source domain, how about its performance to directly segment CT images in target domain? This setting, namely generalizable cross-modality segmentation, owning its clinical potential, is much more challenging than other related settings, e.g., domain adaptation. To achieve this goal, we in this paper propose a novel dual-normalization model by leveraging the augmented source-similar and source-dissimilar images during our generalizable segmentation. To be specific, given a single source domain, aiming to simulate the possible appearance change in unseen target domains, we first utilize a nonlinear transformation to augment source-similar and source-dissimilar images. Then, to sufficiently exploit these two types of augmentations, our proposed dual-normalization based model employs a shared backbone yet independent batch normalization layer for separate normalization. Afterward, we put forward a style-based selection scheme to automatically choose the appropriate path in the test stage. Extensive experiments on three publicly available datasets, i.e., BraTS, Cross-Modality Cardiac, and Abdominal Multi-Organ datasets, have demonstrated that our method outperforms other state-of-the-art domain generalization methods. Code is available at https://github.com/zzzqzhou/Dual-Normalization.
Application and prospect for generative adversarial networks in cross-modality reconstruction of medical images.

Jie Sun,Shichen Jin,Rong Shi,Chuantao Zuo,Jiehui Jiang

DOI: https://doi.org/10.11817/j.issn.1672-7347.2022.220189

2022-01-01

Abstract:Cross-modality reconstruction of medical images refers to predicting the image from one modality to another so as to achieve more accurate personalized medicine. Generative adversarial networks is the most commonly used deep learning technique in cross-modality reconstruction. It can generate realistic images by learning features from implicit distributions that follow the distributions of real data and then reconstruct the image of another modality rapidly. With the sharp increase in clinical demand for multi-modality medical image, this technology has been widely used in the task of cross modal reconstruction between different medical image modalities, such as magnetic resonance imaging, computed tomography and positron emission computed tomography. It can achieve accurate and efficient cross-modality image reconstruction in different parts of the body, such as the brain, heart, etc. In addition, although GAN has achieved some success in cross-modality reconstruction, its stability, generalization ability, and accuracy still need further research and improvement.
Multimodal MR Image Synthesis Using Gradient Prior and Adversarial Learning

Xiaoming Liu,Aihui Yu,Xiangkai Wei,Zhifang Pan,Jinshan Tang

DOI: https://doi.org/10.1109/jstsp.2020.3013418

IF: 7.695

2020-10-01

IEEE Journal of Selected Topics in Signal Processing

Abstract:In magnetic resonance imaging (MRI), several images can be obtained using different imaging settings (e.g. T1, T2, DWI, and Flair). These images have similar anatomical structures but are with different contrasts, which provide a wealth of information for diagnosis. However, the images under specific imaging settings may not be available due to the limitation of scanning time or corruption caused by noises. It is attractive to derive missing images with some settings from the available MR images. In this paper, we propose a novel end-to-end multisetting MR image synthesis method. The proposed method is based on generative adversarial networks (GANs) - a deep learning model. In the proposed method, different MR images obtained by different settings are used as the inputs of a GANs and each image is encoded by an encoder. Each encoder includes a refinement structure which is used to extract a multiscale feature map from an input image. The multiscale feature maps from different input images are then fused to generate several desired target images under specific settings. Because the resultant images obtained with GANs have blurred edges, we fuse gradient prior information in the model to protect high frequency information such as important tissue textures of medical images. In the proposed model, the multiscale information is also adopted in the adversarial learning (not just in the generator or discriminator) so that we can produce high quality synthesized images. We evaluated the proposed method on two public datasets: BRATS and ISLES. Experimental results demonstrate that the proposed approach is superior to current state-of-the-art methods.

engineering, electrical & electronic
Unsupervised learning for cross-domain medical image synthesis using deformation invariant cycle consistency networks

Chengjia Wang,Gillian Macnaught,Giorgos Papanastasiou,Tom MacGillivray,David Newby

DOI: https://doi.org/10.48550/arXiv.1808.03944

2018-08-12

Computer Vision and Pattern Recognition

Abstract:Recently, the cycle-consistent generative adversarial networks (CycleGAN) has been widely used for synthesis of multi-domain medical images. The domain-specific nonlinear deformations captured by CycleGAN make the synthesized images difficult to be used for some applications, for example, generating pseudo-CT for PET-MR attenuation correction. This paper presents a deformation-invariant CycleGAN (DicycleGAN) method using deformable convolutional layers and new cycle-consistency losses. Its robustness dealing with data that suffer from domain-specific nonlinear deformations has been evaluated through comparison experiments performed on a multi-sequence brain MR dataset and a multi-modality abdominal dataset. Our method has displayed its ability to generate synthesized data that is aligned with the source while maintaining a proper quality of signal compared to CycleGAN-generated data. The proposed model also obtained comparable performance with CycleGAN when data from the source and target domains are alignable through simple affine transformations.
MSGAN: Multi-Stage Generative Adversarial Networks for Cross-Modality Domain Adaptation.

Yan Wang,Yixin Chen,Wenjun Wang,Haogang Zhu

DOI: https://doi.org/10.1109/embc48229.2022.9871048

2022-01-01

Abstract:Domain adaptation has become an important topic because the trained neural networks from the source domain generally perform poorly in the target domain due to domain shifts, especially for cross-modality medical images. In this work, we present a new unsupervised domain adaptation approach called Multi-Stage GAN (MSGAN) to tackle the problem of domain shift for CT and MRI segmentation tasks. We adopt the multi-stage strategy in parallel to avoid information loss and transfer rough styles on low-resolution feature maps to the detailed textures on high-resolution feature maps. In detail, the style layers map the learnt style codes from the Gaussian noise to the input features in order to synthesize images with different styles. We validate the proposed method for cross-modality medical image segmentation tasks on two public datasets, and the results demonstrate the effectiveness of our method. Clinical relevance— This technique paves the way to translate cross-modality images (MRI and CT) and it can also mitigate the performance degradation when applying deep neural networks in a cross-domain scenario.
XCAT-GAN for Synthesizing 3D Consistent Labeled Cardiac MR Images on Anatomically Variable XCAT Phantoms

Sina Amirrajab,Samaneh Abbasi-Sureshjani,Yasmina Al Khalil,Cristian Lorenz,Juergen Weese,Josien Pluim,Marcel Breeuwer

DOI: https://doi.org/10.48550/arXiv.2007.13408

2020-07-31

Abstract:Generative adversarial networks (GANs) have provided promising data enrichment solutions by synthesizing high-fidelity images. However, generating large sets of labeled images with new anatomical variations remains unexplored. We propose a novel method for synthesizing cardiac magnetic resonance (CMR) images on a population of virtual subjects with a large anatomical variation, introduced using the 4D eXtended Cardiac and Torso (XCAT) computerized human phantom. We investigate two conditional image synthesis approaches grounded on a semantically-consistent mask-guided image generation technique: 4-class and 8-class XCAT-GANs. The 4-class technique relies on only the annotations of the heart; while the 8-class technique employs a predicted multi-tissue label map of the heart-surrounding organs and provides better guidance for our conditional image synthesis. For both techniques, we train our conditional XCAT-GAN with real images paired with corresponding labels and subsequently at the inference time, we substitute the labels with the XCAT derived ones. Therefore, the trained network accurately transfers the tissue-specific textures to the new label maps. By creating 33 virtual subjects of synthetic CMR images at the end-diastolic and end-systolic phases, we evaluate the usefulness of such data in the downstream cardiac cavity segmentation task under different augmentation strategies. Results demonstrate that even with only 20% of real images (40 volumes) seen during training, segmentation performance is retained with the addition of synthetic CMR images. Moreover, the improvement in utilizing synthetic images for augmenting the real data is evident through the reduction of Hausdorff distance up to 28% and an increase in the Dice score up to 5%, indicating a higher similarity to the ground truth in all dimensions.

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning

[A case of non-endemic Burkitt's lymphoma of the colon].

OA-GAN: Organ-Aware Generative Adversarial Network for Synthesizing Contrast-Enhanced Medical Images

Shape-Consistent Generative Adversarial Networks for Multi-Modal Medical Segmentation Maps

Cross-Modal Attention-Guided Convolutional Network for Multi-modal Cardiac Segmentation.

SA-GAN: Structure-Aware GAN for Organ-Preserving Synthetic CT Generation

SynSeg-Net: Synthetic Segmentation Without Target Modality Ground Truth

Label-informed cardiac magnetic resonance image synthesis through conditional generative adversarial networks

Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonization

Multi-Constraint Transferable Generative Adversarial Networks for Cross-Modal Brain Image Synthesis

Segmentation Guided Crossing Dual Decoding Generative Adversarial Network for Synthesizing Contrast-Enhanced Computed Tomography Images

SUSAN: Segment Unannotated image Structure using Adversarial Network

Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptation

Generalizable Cross-modality Medical Image Segmentation Via Style Augmentation and Dual Normalization

ReeGAN: MRI image edge-preserving synthesis based on GANs trained with misaligned data

CycleSGAN: A cycle-consistent and semantics-preserving generative adversarial network for unpaired MR-to-CT image synthesis

Generalizable Cross-modality Medical Image Segmentation via Style Augmentation and Dual Normalization

Application and prospect for generative adversarial networks in cross-modality reconstruction of medical images.

Multimodal MR Image Synthesis Using Gradient Prior and Adversarial Learning

Unsupervised learning for cross-domain medical image synthesis using deformation invariant cycle consistency networks

MSGAN: Multi-Stage Generative Adversarial Networks for Cross-Modality Domain Adaptation.

XCAT-GAN for Synthesizing 3D Consistent Labeled Cardiac MR Images on Anatomically Variable XCAT Phantoms