Abstract:Training deep learning models for image registration or segmentation of dynamic contrast enhanced (DCE)-MRI data is challenging. This is mainly due to the wide variations in contrast enhancement within and between patients. To train a model effectively, a large dataset is needed, but acquiring it is expensive and time consuming. Instead, style transfer can be used to generate new images from existing images. In this study, our objective is to develop a style transfer method that incorporates spatio-temporal information to either add or remove contrast enhancement from an existing image. We propose a Temporal Image-to-Image Style Transfer Network (TIST-Net), consisting of an auto-encoder combined with convolutional long short-term memory (LSTM) networks. This enables disentanglement of the content and style latent spaces of the time series data, using spatio-temporal information to learn and predict key structures . To generate new images , we use deformable and adaptive convolutions which allow fine grained control over the combination of the content and style latent spaces. We evaluate our method, using popular metrics and a previously proposed contrast weighted structural similarity index measure (CW-SSIM). We also perform a clinical evaluation, where experts are asked to rank images generated by multiple methods. Our model achieves state-of-the-art performance on three datasets (kidney, prostate and uterus) achieving an SSIM of 0.91±0.03, 0.73±0.04, 0.88±0.04 respectively when performing style transfer between a non-enhanced image and a contrast-enhanced image. Similarly, SSIM results for style transfer from a contrast-enhanced image to a non-enhanced image were 0.89±0.03, 0.82±0.03, 0.87±0.03. In the clinical evaluation, our method was ranked consistently higher than other approaches. TIST-Net can be used to generate new DCE-MRI data from existing images. In future, this may improve models for tasks such as image registration or segmentation by allowing small training datasets to be expanded.

Style Transfer and Self-Supervised Learning Powered Myocardium Infarction Super-Resolution Segmentation

A Fully Automated Two-Stage Segmentation Approach for Late Gadolinium-Enhanced Cardiac Magnetic Resonance Images in Personalized Cardiac Modeling

Super-Resolution Based Patch-Free 3D Medical Image Segmentation with Self-Supervised Guidance

Neural Style Transfer Improves 3D Cardiovascular MR Image Segmentation on Inconsistent Data

Style-invariant Cardiac Image Segmentation with Test-time Augmentation

Deep Learning Assessment of Myocardial Infarction From MR Image Sequences.

Semi-supervised Cardiac Image Segmentation via Label Propagation and Style Transfer

Self-Supervised Generative Style Transfer for One-Shot Medical Image Segmentation

Transforming Heart Chamber Imaging: Self-Supervised Learning for Whole Heart Reconstruction and Segmentation

Image Super-Resolution by Supervised Adaption of Patchwise Self-similarity from High-Resolution Image.

Style Curriculum Learning for Robust Medical Image Segmentation

Improving style transfer in dynamic contrast enhanced MRI using a spatio-temporal approach

MSMHSA-DeepLab V3+: An Effective Multi-Scale, Multi-Head Self-Attention Network for Dual-Modality Cardiac Medical Image Segmentation

Pseudo-3D Network for Multi-sequence Cardiac MR Segmentation

Deep Learning for Breast MRI Style Transfer with Limited Training Data

Fully automated deep learning based segmentation of normal, infarcted and edema regions from multiple cardiac MRI sequences

Style Consistency Unsupervised Domain Adaptation Medical Image Segmentation

Multi-Modality Cardiac Image Analysis with Deep Learning

AWSnet: An Auto-weighted Supervision Attention Network for Myocardial Scar and Edema Segmentation in Multi-sequence Cardiac Magnetic Resonance Images

Style Transfer Enabled Sim2Real Framework for Efficient Learning of Robotic Ultrasound Image Analysis Using Simulated Data

TIST-Net: style transfer in dynamic contrast enhanced MRI using spatial and temporal information