Practical Deep Raw Image Denoising on Mobile Devices

Yuzhi Wang,Haibin Huang,Qin Xu,Jiaming Liu,Yiqun Liu,Jue Wang

2020-10-14

Abstract:Deep learning-based image denoising approaches have been extensively studied in recent years, prevailing in many public benchmark datasets. However, the stat-of-the-art networks are computationally too expensive to be directly applied on mobile devices. In this work, we propose a light-weight, efficient neural network-based raw image denoiser that runs smoothly on mainstream mobile devices, and produces high quality denoising results. Our key insights are twofold: (1) by measuring and estimating sensor noise level, a smaller network trained on synthetic sensor-specific data can out-perform larger ones trained on general data; (2) the large noise level variation under different ISO settings can be removed by a novel k-Sigma Transform, allowing a small network to efficiently handle a wide range of noise levels. We conduct extensive experiments to demonstrate the efficiency and accuracy of our approach. Our proposed mobile-friendly denoising model runs at ~70 milliseconds per megapixel on Qualcomm Snapdragon 855 chipset, and it is the basis of the night shot feature of several flagship smartphones released in 2019.

Image and Video Processing,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper primarily addresses the issue of efficient and high-quality raw image denoising on mobile devices.

Specifically, although deep learning-based image denoising methods have achieved significant results on many public datasets, these methods are usually computationally expensive and difficult to apply directly to resource-limited mobile devices. To solve this problem, the authors propose a lightweight and efficient neural network model that can run smoothly on mainstream mobile devices and produce high-quality denoising results.

The main contributions of the paper include:

1. **Systematic Sensor Noise Estimation Method**: By measuring and estimating the noise level of the sensor, a sensor-specific synthetic dataset is constructed to train the denoising neural network.
2. **k-Sigma Transformation**: This is a novel linear transformation that maps noisy images captured under different ISO settings to an ISO-independent signal-noise space, allowing a small network to effectively handle various noise levels in different scenarios.
3. **Mobile-Friendly Network Architecture**: A convolutional neural network architecture suitable for mobile devices is designed to achieve efficient image denoising.

In addition, the paper demonstrates the efficiency and accuracy of the method in practical applications, showing significant advantages especially in photos taken under low-light conditions. Experimental results show that the proposed model not only performs comparably to state-of-the-art methods but also significantly reduces the required computational resources, making it a practical denoising solution for mobile devices.