Abstract:Driver monitoring is a popular topic of investigation. Contactless vital-sign measurement via remote photoplethysmography (rPPG) is an easy and effective alternative to cumbersome wearable devices during driving. However, driving is a noise-heavy situation for remote vital-sign sensing, owing to the disturbances caused by jolted cars or varying illumination from time and weather variations. In this study, we propose deep learning models for rPPG construction and heart rate estimation that work for both red–green–blue (RGB) and monochrome near-infrared (NIR) images. The proposed method overcomes the adaptation issues commonly seen in deep-learning-based heart rate estimation approaches and achieves a significant improvement in robustness for heart rate measurement. To evaluate the performance thoroughly, in addition to a public dataset, we construct four testing driving sets for various practical conditions, including driving at night, on rainy days, and on long-term journeys. It is also worth noting that we conducted the assessment with a cross-dataset protocol to further verify the efficiency of our method. The experimental results on the five datasets show that our proposed method can perform better than the state-of-the-art method for both the RGB and NIR images, especially in some challenging cases. The root-mean-square error (RMSE) for the situation with additional head motion conditions can be improved by 28.6% with RGB images and by 21.91% with NIR images on the open dataset. Moreover, RMSE can be improved by up to 42.6% at night, 14.41% on rainy days, and 18.78% on long-term journeys.

What problem does this paper attempt to address?

This paper attempts to solve the problem of non - contact heart rate measurement using remote photoplethysmography (rPPG) in driving scenarios. Specifically, the paper focuses on how to overcome various interfering factors existing in the driving environment, such as head movements caused by vehicle jolts, light changes (caused by time and weather changes), low - light conditions at night, etc. All these factors will affect the accurate extraction of rPPG signals and the precise estimation of heart rate. To address the above challenges, the authors propose a deep - learning - based method that can process red - green - blue (RGB) images and monochrome near - infrared (NIR) images to achieve more robust heart rate measurement. By introducing optimized model structures and training strategies for different image types, this study aims to improve the accuracy and robustness of heart rate measurement under complex driving conditions. The main contributions of the paper include: 1. Proposing a deep - learning - based heart rate measurement process applicable to RGB and NIR inputs. This process first constructs rPPG signals from input images through averaging operations and then uses a convolutional neural network (CNN) - based model for heart rate estimation to maintain robustness under different conditions. 2. Proposing two different CNN model structures for different purposes: one is an end - to - end rPPG construction model for single - channel NIR images; the other is a heart rate estimation model that converts rPPG signals into predicted heart rate values. This is the first proposed model structure that can effectively extract vital signs in driving scenarios, whether RGB or single - channel NIR images are used as rPPG inputs. 3. In terms of evaluation, all experiments are carried out using a cross - dataset protocol, and several practical test datasets are used, including newly constructed datasets and datasets from previous work, covering various advanced rPPG construction scenarios, such as night - driving or rain - driving, and long - time driving situations. Through these methods and improvements, the paper shows that its proposed model can significantly improve the accuracy of heart rate measurement compared to existing technologies under a variety of challenging driving conditions.

Deep-Learning-Based Remote Photoplethysmography Measurement in Driving Scenarios With Color and Near-Infrared Images

Ivrr-PPG: An Illumination Variation Robust Remote-PPG Algorithm for Monitoring Heart Rate of Drivers

Benchmark of Physiological Model Based and Deep Learning Based Remote Photoplethysmography in Automotive Applications

AI-Driven rPPG Heart Rate Detection for In-Vehicle Monitoring

Internship Report: Benchmark of Deep Learning-based Imaging PPG in Automotive Domain

Heart Rate Estimation in Driver Monitoring System Using Quality-Guided Spectrum Peak Screening

Assessment of Deep Learning-based Heart Rate Estimation using Remote Photoplethysmography under Different Illuminations

A Compensation Network With Error Mapping for Robust Remote Photoplethysmography in Noise-Heavy Conditions

Heart Rate Extraction Based on Near-Infrared Camera: Towards Driver State Monitoring

An Improved Approach to the Performance of Remote Photoplethysmography

Remote Heart Rate Measurement From Near-Infrared Videos Based on Joint Blind Source Separation With Delay-Coordinate Transformation

Image Enhancement for Remote Photoplethysmography in a Low-Light Environment

Heart Rate Estimation from Facial Image Sequences of a Dual-Modality RGB-NIR Camera

Remote Heart Rate Measurement from Face Videos under Realistic Situations

RealSense = Real Heart Rate: Illumination Invariant Heart Rate Estimation from Videos

An image enhancement based method for improving rPPG extraction under low-light illumination

Deep learning-based remote-photoplethysmography measurement from short-time facial video

AutoHR: A Strong End-to-End Baseline for Remote Heart Rate Measurement With Neural Searching

Heart rate estimation by leveraging static and dynamic region weights

GRGB rPPG: An Efficient Low-Complexity Remote Photoplethysmography-Based Algorithm for Heart Rate Estimation

Remote photoplethysmography (rPPG) based learning fatigue detection