Image Enhancement for Remote Photoplethysmography in a Low-Light Environment

Lin Xi,Weihai Chen,Changchen Zhao,Xingming Wu,Jianhua Wang

DOI: https://doi.org/10.1109/FG47880.2020.00076

2023-03-16

Abstract:With the improvement of sensor technology and significant algorithmic advances, the accuracy of remote heart rate monitoring technology has been significantly improved. Despite of the significant algorithmic advances, the performance of rPPG algorithm can degrade in the long-term, high-intensity continuous work occurred in evenings or insufficient light environments. One of the main challenges is that the lost facial details and low contrast cause the failure of detection and tracking. Also, insufficient lighting in video capturing hurts the quality of physiological signal. In this paper, we collect a large-scale dataset that was designed for remote heart rate estimation recorded with various illumination variations to evaluate the performance of the rPPG algorithm (Green, ICA, and POS). We also propose a low-light enhancement solution (technical solution) for remote heart rate estimation under the low-light condition. Using collected dataset, we found 1) face detection algorithm cannot detect faces in video captured in low light conditions; 2) A decrease in the amplitude of the pulsatile signal will lead to the noise signal to be in the dominant position; and 3) the chrominance-based method suffers from the limitation in the assumption about skin-tone will not hold, and Green and ICA method receive less influence than POS in dark illuminance environment. The proposed solution for rPPG process is effective to detect and improve the signal-to-noise ratio and precision of the pulsatile signal.

Computer Vision and Pattern Recognition,Image and Video Processing

What problem does this paper attempt to address?

This paper attempts to solve the problems encountered in heart rate monitoring by remote photoplethysmography (rPPG) in low - light environments. Specifically, the paper focuses on:

1. **Failure of face detection and tracking**: In low - light conditions, due to the loss of facial details and reduced contrast, existing face detection algorithms cannot work properly.
2. **Weakening of pulsatile signals**: In low - light conditions, the amplitude of pulsatile signals decreases, and noise signals become dominant, affecting the accuracy of heart rate estimation.
3. **Invalidity of skin - color assumptions**: Existing pulse extraction models rely on skin - color assumptions, but in low - light conditions, these assumptions no longer hold, especially when the blue and red channel signals in the RGB space are almost zero.

To address these problems, the paper proposes a low - light enhancement solution aimed at improving the performance of heart rate monitoring in low - light conditions. This solution includes the following three main steps:

1. **Video enhancement**: Improve the visibility of the video through an image enhancement algorithm to make physiological signals clearer.
2. **Region of interest (ROI) detection and tracking**: Detect and track the ROI containing pulsatile signals from the enhanced video frames.
3. **Heart rate extraction**: Extract the heart rate from the original signal in the ROI.

The main contributions of the paper include:

1. Providing the first solution for robust rPPG measurement in low - light environments, including an image enhancement module and a pulse extraction module.
2. Systematically analyzing the influence of illumination on rPPG measurement and testing the accuracy of several rPPG algorithms under different illumination intensities, using mean absolute error (MAE), root mean square error (RMSE) and signal - to - noise ratio (SNR) as evaluation metrics.
3. Constructing a large - scale dataset covering the illumination range from 1 to 100 lux, which is currently the only publicly available dataset suitable for rPPG research in low - light environments.

Through these methods, the paper effectively improves the accuracy and reliability of heart rate monitoring in low - light conditions.