Abstract:Noncontact heart-rate (HR) measurement is a very important trend in clinical medicine. Recently, a variety of deep networks have been applied to estimate HRs from facial videos. However, due to limited data resources and poor parameter optimization, few existing models have achieved incredible performance in complicated scenarios, such as those with illumination changes, different skin tones, and facial motion. To address these challenges, this article proposes a novel multistage deep network (MSDN) that can decentralize the learnable parameters into different stages to reduce the difficulty of learning through multiple training steps. Specifically, the proposed network consists of three stages in an end-to-end way. In the first stage, an HR-aware feature extractor uses the next convolutional neural network (ConvNeXt) embedded with a newly designed bandpass filter as its backbone to extract spatial-temporal features for determining HR changes. Moreover, pseudolabels are generated to make the features compatible with illumination, motion, and color variance. In the second stage, various modules, including singular value decomposition (SVD) pooling and enhanced difference convolution (EDC) modules, are then designed and combined with a transformer encoder to construct a feature-compressed remote photoplethysmography (rPPG) generator. In the third stage, an HR estimator with an interbeat interval (IBI) analyzer and a 1-D filter is newly designed for HR estimation. Extensive experiments are performed on three publicly available databases (i.e., VIPL-HR, COHFACE, and PURE), and the results demonstrate the effectiveness of the proposed method through ablation studies and comparison experiments with state-of-the-art (SOTA) methods.

Heart Rate Estimation Via Self-Adaptive Region Selection and Multiregion-Fusion 1D CNN

Deep-HR: Fast Heart Rate Estimation from Face Video under Realistic Conditions

CoSTHR: A Heart Rate Estimating Network with Adaptive Color Space Transformation

Remote Heart Rate Estimation Based on Convolutional Neural Network and Regional Adaptive Weighting.

Heart rate estimation by leveraging static and dynamic region weights

EVM-CNN: Real-Time Contactless Heart Rate Estimation from Facial Video

Visual Heart Rate Estimation from Facial Video Based on CNN

Heart Rate Estimation From Facial Videos Using a Spatiotemporal Representation With Convolutional Neural Networks

Human Heart Rate Estimation Using Ordinary Cameras under Natural Movement.

Remote Heart Rate Measurement from Face Videos under Realistic Situations

MSDN: A Multistage Deep Network for Heart-Rate Estimation from Facial Videos

Robust Heart Rate Estimation with Spatial–Temporal Attention Network from Facial Videos

Lightweight and interpretable convolutional neural network for real-time heart rate monitoring using low-cost video camera under realistic conditions

Multi-hierarchical Convolutional Network for Efficient Remote Photoplethysmograph Signal and Heart Rate Estimation from Face Video Clips.

Accurate Face Alignment and Adaptive Patch Selection for Heart Rate Estimation from Videos under Realistic Scenarios.

Heart Rate Estimation from Facial Videos Based on Convolutional Neural Network

Visual Heart Rate Estimation from RGB Facial Video using Spectral Reflectance

Automated Heartbeat Classification Using 3-D Inputs Based on Convolutional Neural Network with Multi-Fields of View.

Non-contact PPG Signal and Heart Rate Estimation with Multi-Hierarchical Convolutional Network

A Feasibility Study of a Video-Based Heart Rate Estimation Method with Convolutional Neural Networks

RhythmNet: End-to-end Heart Rate Estimation from Face via Spatial-temporal Representation