Low-dose CT Denoising with Language-engaged Dual-space Alignment

Zhihao Chen,Tao Chen,Chenhui Wang,Chuang Niu,Ge Wang,Hongming Shan

2024-03-10

Abstract:While various deep learning methods were proposed for low-dose computed tomography (CT) denoising, they often suffer from over-smoothing, blurring, and lack of explainability. To alleviate these issues, we propose a plug-and-play Language-Engaged Dual-space Alignment loss (LEDA) to optimize low-dose CT denoising models. Our idea is to leverage large language models (LLMs) to align denoised CT and normal dose CT images in both the continuous perceptual space and discrete semantic space, which is the first LLM-based scheme for low-dose CT denoising. LEDA involves two steps: the first is to pretrain an LLM-guided CT autoencoder, which can encode a CT image into continuous high-level features and quantize them into a token space to produce semantic tokens derived from the LLM's vocabulary; and the second is to minimize the discrepancy between the denoised CT images and normal dose CT in terms of both encoded high-level features and quantized token embeddings derived by the LLM-guided CT autoencoder. Extensive experimental results on two public LDCT denoising datasets demonstrate that our LEDA can enhance existing denoising models in terms of quantitative metrics and qualitative evaluation, and also provide explainability through language-level image understanding. Source code is available at

Image and Video Processing,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper aims to address the issues of excessive smoothing, blurring, and lack of interpretability in the denoising process of Low-Dose Computed Tomography (LDCT) images. Specifically:

1. **Current Issues**: Although various existing deep learning methods can be used for LDCT image denoising, they often result in excessive smoothing, loss of details, and lack of interpretability of the denoising results.

2. **Solution**: To address these issues, the authors propose a new plug-and-play loss function—Language-Engaged Dual-space Alignment Loss (LEDA). This method leverages Large Language Models (LLMs) to align the denoised CT images with normal-dose CT images in both continuous perceptual space and discrete semantic space, thereby enhancing the denoising effect.

3. **Specific Steps**:
- First, pre-train a CT autoencoder guided by an LLM, which can encode CT images into continuous high-level features and quantize them into text tokens based on the LLM vocabulary.
- Then, during the training process of the denoising model, minimize the differences between the denoised images and the normal-dose CT images, optimizing not only in continuous features but also in discrete text embeddings.

4. **Experimental Validation**: Extensive experiments on two public low-dose CT denoising datasets demonstrate that LEDA can significantly improve the performance of existing denoising models in terms of quantitative metrics and qualitative evaluation, providing language-level interpretability.

In summary, the goal of this paper is to improve the quality of low-dose CT image denoising and enhance its interpretability by introducing LLM.