Abstract:Noncontact heart-rate (HR) measurement is a very important trend in clinical medicine. Recently, a variety of deep networks have been applied to estimate HRs from facial videos. However, due to limited data resources and poor parameter optimization, few existing models have achieved incredible performance in complicated scenarios, such as those with illumination changes, different skin tones, and facial motion. To address these challenges, this article proposes a novel multistage deep network (MSDN) that can decentralize the learnable parameters into different stages to reduce the difficulty of learning through multiple training steps. Specifically, the proposed network consists of three stages in an end-to-end way. In the first stage, an HR-aware feature extractor uses the next convolutional neural network (ConvNeXt) embedded with a newly designed bandpass filter as its backbone to extract spatial-temporal features for determining HR changes. Moreover, pseudolabels are generated to make the features compatible with illumination, motion, and color variance. In the second stage, various modules, including singular value decomposition (SVD) pooling and enhanced difference convolution (EDC) modules, are then designed and combined with a transformer encoder to construct a feature-compressed remote photoplethysmography (rPPG) generator. In the third stage, an HR estimator with an interbeat interval (IBI) analyzer and a 1-D filter is newly designed for HR estimation. Extensive experiments are performed on three publicly available databases (i.e., VIPL-HR, COHFACE, and PURE), and the results demonstrate the effectiveness of the proposed method through ablation studies and comparison experiments with state-of-the-art (SOTA) methods.

Deep-HR: Fast Heart Rate Estimation from Face Video under Realistic Conditions

Remote Heart Rate Measurement from Face Videos under Realistic Situations

Deep learning-based remote-photoplethysmography measurement from short-time facial video

MSDN: A Multistage Deep Network for Heart-Rate Estimation from Facial Videos

AutoHR: A Strong End-to-End Baseline for Remote Heart Rate Measurement With Neural Searching

Conventional and deep learning methods in heart rate estimation from RGB face videos

RhythmNet: End-to-end Heart Rate Estimation from Face via Spatial-temporal Representation

Visual Heart Rate Estimation from RGB Facial Video using Spectral Reflectance

EVM-CNN: Real-Time Contactless Heart Rate Estimation from Facial Video

Lightweight and interpretable convolutional neural network for real-time heart rate monitoring using low-cost video camera under realistic conditions

Heart rate estimation by leveraging static and dynamic region weights

Heart Rate Estimation From Facial Videos Using a Spatiotemporal Representation With Convolutional Neural Networks

Human Heart Rate Estimation Using Ordinary Cameras under Natural Movement.

Adaptive Heart Rate Estimation from Face Videos

Visual Heart Rate Estimation from Facial Video Based on CNN

Accurate Face Alignment and Adaptive Patch Selection for Heart Rate Estimation from Videos under Realistic Scenarios.

Information-enhanced Network for Noncontact Heart Rate Estimation from Facial Videos

Heart Rate Estimation Via Self-Adaptive Region Selection and Multiregion-Fusion 1D CNN

Multi-hierarchical Convolutional Network for Efficient Remote Photoplethysmograph Signal and Heart Rate Estimation from Face Video Clips.

VIPL-HR: A Multi-modal Database for Pulse Estimation from Less-constrained Face Video

RealSense = Real Heart Rate: Illumination Invariant Heart Rate Estimation from Videos