Abstract:Purpose Magnetic resonance imaging (MRI) serves as an important medical imaging modality for a variety of clinical applications. However, the problem of long imaging time limited its wide usage. In addition, prolonged scan time will cause discomfort to the patient, leading to severe image artifacts. On the other hand, manually lesion segmentation is time consuming. Algorithm-based automatic lesion segmentation is still challenging, especially for accelerated imaging with low quality. Methods In this paper, we proposed a multitask learning-based method to perform image reconstruction and lesion segmentation simultaneously, called "RecSeg". Our hypothesis is that both tasks can benefit from the usage of the proposed combined model. In the experiment, we validated the proposed multitask model on MR k-space data with different acceleration factors (2x, 4x, and 6x). Two connected U-nets were used for the tasks of liver and renal image reconstruction and segmentation. A total of 50 healthy subjects and 100 patients with hepatocellular carcinoma were included for training and testing. For the segmentation part, we use healthy subjects to verify organ segmentation, and hepatocellular carcinoma patients to verify lesion segmentation. The organs and lesions were manually contoured by an experienced radiologist. Results Experimental results show that the proposed RecSeg yielded the highest PSNR (RecSeg: 32.39 +/- 1.64 vs. KSVD: 29.53 +/- 2.74 and single U-net: 31.18 +/- 1.68, respectively, p < 0.05) and highest structural similarity index measure (SSIM) (RecSeg: 0.93 +/- 0.01 vs. KSVD: 0.88 +/- 0.02 and single U-net: 0.90 +/- 0.01, respectively, p < 0.05) under 6x acceleration. Moreover, in the task of lesion segmentation, it is proposed that RecSeg produced the highest Dice score (RecSeg: 0.86 +/- 0.01 vs. KSVD: 0.82 +/- 0.01 and single U-net: 0.84 +/- 0.01, respectively, p < 0.05). Conclusions This study focused on the simultaneous reconstruction of medical images and the segmentation of organs and lesions. It is observed that the multitask learning-based method can improve performances of both image reconstruction and lesion segmentation.

Cooperative-Net: An End-to-End Multi-Task Interaction Network for Unified Reconstruction and Segmentation of MR Image

Mutual Information-Based Graph Co-Attention Networks for Multimodal Prior-Guided Magnetic Resonance Imaging Segmentation

Parallel Imaging and Convolutional Neural Network Combined Fast MR Image Reconstruction: Applications in Low-Latency Accelerated Real-Time Imaging.

Simultaneous Image Reconstruction and Lesion Segmentation in Accelerated MRI Using Multitasking Learning.

Simultaneous reconstruction and segmentation of MRI image by manifold learning

Multi-Modal Multi-Slice Cooperative Dual-Domain Cascaded De-Aliasing Network for MR Imaging Reconstruction

MLMFNet: A Multi-Level Modality Fusion Network for Multi-Modal Accelerated MRI Reconstruction

Joint Under-Sampling Pattern and Dual-Domain Reconstruction for Accelerating Multi-Contrast MRI

Deep unregistered multi-contrast MRI reconstruction

Multi-Modality MR Image Synthesis via Confidence-Guided Aggregation and Cross-Modality Refinement

Learning Unified Hyper-Network for Multi-Modal MR Image Synthesis and Tumor Segmentation With Missing Modalities

Joint MR Image Reconstruction and Super-Resolution via Mutual Co-Attention Network

Enhancing CT Image synthesis from multi-modal MRI data based on a multi-task neural network framework

Cross-contrast mutual fusion network for joint MRI reconstruction and super-resolution

Task Transformer Network for Joint MRI Reconstruction and Super-Resolution

CAMP-Net: Consistency-Aware Multi-Prior Network for Accelerated MRI Reconstruction

Joint Calibrationless Reconstruction and Segmentation of Parallel MRI

SelfCoLearn: Self-supervised Collaborative Learning for Accelerating Dynamic MR Imaging.

$Σ$-net: Systematic Evaluation of Iterative Deep Neural Networks for Fast Parallel MR Image Reconstruction

Accelerated Multi-Contrast MRI Reconstruction via Frequency and Spatial Mutual Learning

SK-Unet: an Improved U-net Model with Selective Kernel for the Segmentation of Multi-sequence Cardiac MR