LighTDiff: Surgical Endoscopic Image Low-Light Enhancement with T-Diffusion

Tong Chen,Qingcheng Lyu,Long Bai,Erjian Guo,Huxin Gao,Xiaoxiao Yang,Hongliang Ren,Luping Zhou
2024-05-17
Abstract:Advances in endoscopy use in surgeries face challenges like inadequate lighting. Deep learning, notably the Denoising Diffusion Probabilistic Model (DDPM), holds promise for low-light image enhancement in the medical field. However, DDPMs are computationally demanding and slow, limiting their practical medical applications. To bridge this gap, we propose a lightweight DDPM, dubbed LighTDiff. It adopts a T-shape model architecture to capture global structural information using low-resolution images and gradually recover the details in subsequent denoising steps. We further prone the model to significantly reduce the model size while retaining performance. While discarding certain downsampling operations to save parameters leads to instability and low efficiency in convergence during the training, we introduce a Temporal Light Unit (TLU), a plug-and-play module, for more stable training and better performance. TLU associates time steps with denoised image features, establishing temporal dependencies of the denoising steps and improving denoising outcomes. Moreover, while recovering images using the diffusion model, potential spectral shifts were noted. We further introduce a Chroma Balancer (CB) to mitigate this issue. Our LighTDiff outperforms many competitive LLIE methods with exceptional computational efficiency.
Image and Video Processing,Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
### Problems Addressed by the Paper The paper aims to address the issue of poor imaging quality of endoscopic images under low-light conditions in minimally invasive surgery (MIS). Specifically, low-light environments lead to insufficient image brightness and contrast, affecting detail recognition, making it difficult for surgeons to observe tissue structures or pathological areas, thereby increasing the difficulty and risk of errors in surgery. To tackle this challenge, the authors propose a lightweight denoising diffusion probabilistic model (DDPM) called **LighTDiff**. The main features of LighTDiff include: 1. **T-shaped Architecture**: Captures global structural information using low-resolution images and gradually restores details. 2. **Temporal Lighting Unit (TLU)**: Introduces a plugin module to stabilize the training process and improve denoising effects. 3. **Chroma Balancer (CB)**: Mitigates chromatic shift issues that may arise during the diffusion process. Experimental results show that LighTDiff performs excellently on two public datasets (EndoVis17 and EndoVis18) as well as a real-world dataset, not only improving image quality and efficiency but also significantly outperforming other existing methods. Additionally, LighTDiff demonstrates outstanding computational efficiency, making it suitable for consumer-grade hardware.