TOGS: Gaussian Splatting with Temporal Opacity Offset for Real-Time 4D DSA Rendering

Shuai Zhang,Huangxuan Zhao,Zhenghong Zhou,Guanjun Wu,Chuansheng Zheng,Xinggang Wang,Wenyu Liu

2024-03-29

Abstract:Four-dimensional Digital Subtraction Angiography (4D DSA) is a medical imaging technique that provides a series of 2D images captured at different stages and angles during the process of contrast agent filling blood vessels. It plays a significant role in the diagnosis of cerebrovascular diseases. Improving the rendering quality and speed under sparse sampling is important for observing the status and location of lesions. The current methods exhibit inadequate rendering quality in sparse views and suffer from slow rendering speed. To overcome these limitations, we propose TOGS, a Gaussian splatting method with opacity offset over time, which can effectively improve the rendering quality and speed of 4D DSA. We introduce an opacity offset table for each Gaussian to model the temporal variations in the radiance of the contrast agent. By interpolating the opacity offset table, the opacity variation of the Gaussian at different time points can be determined. This enables us to render the 2D DSA image at that specific moment. Additionally, we introduced a Smooth loss term in the loss function to mitigate overfitting issues that may arise in the model when dealing with sparse view scenarios. During the training phase, we randomly prune Gaussians, thereby reducing the storage overhead of the model. The experimental results demonstrate that compared to previous methods, this model achieves state-of-the-art reconstruction quality under the same number of training views. Additionally, it enables real-time rendering while maintaining low storage overhead. The code will be publicly available.

Computer Vision and Pattern Recognition,Graphics

What problem does this paper attempt to address?

The paper primarily addresses the issues present in four-dimensional Digital Subtraction Angiography (4D DSA) and proposes solutions. Specifically, the paper tackles the following problems:

1. **Rendering quality and speed**: Existing 4D DSA methods suffer from insufficient rendering quality under sparse views and slow rendering speed.

2. **Need to reduce radiation exposure**: The commonly used Feldkamp-Davis-Kress (FDK) reconstruction algorithm in clinical practice requires a large number of 2D DSA images to obtain high-quality 4D DSA results, which leads to significant radiation exposure for patients.

3. **Real-time rendering and low storage overhead**: Doctors need to quickly view the lesion area during diagnosis, so real-time rendering and low storage overhead are particularly important for 4D DSA.

To address the above issues, the paper proposes TOGS (Temporal Opacity Offset Gaussian Splatting), a method based on Gaussian splatting. This method introduces an opacity offset table in the time dimension, which can effectively simulate the diffusion process of the contrast agent in the blood vessels, thereby improving the rendering quality and speed of 4D DSA. Additionally, the paper introduces a smooth loss term to mitigate overfitting issues and further reduces the model's storage overhead by randomly pruning Gaussian functions.

Experimental results show that compared to previous methods, TOGS achieves state-of-the-art reconstruction quality with the same number of training views and can achieve real-time rendering while maintaining low storage overhead.