Event-based Camera Simulation using Monte Carlo Path Tracing with Adaptive Denoising

Yuta Tsuji,Tatsuya Yatagawa,Hiroyuki Kubo,Shigeo Morishima

DOI: https://doi.org/10.1109/icip49359.2023.10222771

2023-08-22

Abstract:This paper presents an algorithm to obtain an event-based video from noisy frames given by physics-based Monte Carlo path tracing over a synthetic 3D scene. Given the nature of dynamic vision sensor (DVS), rendering event-based video can be viewed as a process of detecting the changes from noisy brightness values. We extend a denoising method based on a weighted local regression (WLR) to detect the brightness changes rather than applying denoising to every pixel. Specifically, we derive a threshold to determine the likelihood of event occurrence and reduce the number of times to perform the regression. Our method is robust to noisy video frames obtained from a few path-traced samples. Despite its efficiency, our method performs comparably to or even better than an approach that exhaustively denoises every frame.

Computer Vision and Pattern Recognition,Graphics

What problem does this paper attempt to address?

The paper attempts to address the problem of efficiently rendering event-based camera videos from noisy video frames generated by physics-based Monte Carlo Path Tracing (MCPT). Specifically, the goals of the paper are:

1. **Reduce computational complexity**: Traditional MCPT-based methods require a significant amount of computation time to generate noise-free video frames, making it challenging to achieve efficient event camera simulation in practical applications. This paper proposes a new algorithm that reduces the computational load through an adaptive denoising method, thereby significantly improving efficiency while maintaining high accuracy.

2. **Improve event detection accuracy**: Event cameras trigger events by detecting changes in pixel brightness. Since video frames generated by MCPT typically contain a lot of noise, directly detecting events from these noisy frames leads to many false detections. This paper proposes a method based on Weighted Local Regression (WLR) to determine the event occurrence threshold through theoretical derivation, thereby reducing false detections and improving event detection accuracy.

3. **Robustness**: The proposed algorithm is robust to noisy video frames and can still achieve good event detection results with a small number of path samples. Compared to existing methods, this method strikes a good balance between efficiency and accuracy.

In summary, the main goal of this paper is to achieve efficient and accurate rendering of event-based camera videos from noisy video frames through improved denoising and event detection methods.