Abstract:Remote photoplethysmography (rPPG), which measures human heart rate without physical contact with the skin, has become active research in recent years. Neural networks have been introduced into rPPG for accurate pulse measurement and have achieved overwhelming results. However, there is a lack of in-depth analysis of key components of neural networks exhibiting a crucial impact on pulse extraction from video. In this paper, we present a network with attention and spatio-temporal convolutional block (ASTNet), exploiting the impact of key factors including different spatio-temporal convolutions, attention mechanism, the number of convolutional layers, and receptive field sizes. The novel attention module named spatio-temporal-channel (STC) attention is designed to jointly learn weights in spatial, temporal, and channel dimensions in a more efficient way. Extensive experiments have been conducted on two uncompressed datasets and one compressed dataset. Results show that ASTNet outperforms state-of-the-art methods in accuracy and computational time. Specifically, networks with larger receptive field sizes and more spatio-temporal blocks generally achieve better performance. Networks with pseudo 3D convolution outperform those with convolutional 3D in static videos, and the opposite is true in motion videos. The results exhibit a similar tendency both on uncompressed and compressed datasets. The proposed method improves the performance of pulse signal compared to PhysNet (the second-best approach in the compared methods), with the signal-to-noise ratio increased by 7.03%, 10.19%, 4.79%, the mean absolute error decreased by 17.95%, 14.17%, 22.76%, and the root-mean-square error decreased by 21.43%, 2.73%, 25.43%, on the PURE, Self-rPPG, and COHFACE datasets, respectively.

AutoHR: A Strong End-to-End Baseline for Remote Heart Rate Measurement With Neural Searching

Deep-HR: Fast Heart Rate Estimation from Face Video under Realistic Conditions

CoSTHR: A Heart Rate Estimating Network with Adaptive Color Space Transformation

Deep learning-based remote-photoplethysmography measurement from short-time facial video

BYHE: A Simple Framework for Boosting End-to-end Video-based Heart Rate Measurement Network

HRUNet: Assessing Uncertainty in Heart Rates Measured from Facial Videos

RhythmNet: End-to-end Heart Rate Estimation from Face via Spatial-temporal Representation

Heart Rate Estimation From Facial Videos Using a Spatiotemporal Representation With Convolutional Neural Networks

Remote Heart Rate Measurement from Face Videos under Realistic Situations

Remote Heart Rate Measurement From Near-Infrared Videos Based on Joint Blind Source Separation With Delay-Coordinate Transformation

Deep Learning Methods for Remote Heart Rate Measurement: A Review and Future Research Agenda

MDAR: A Multiscale Features-Based Network for Remotely Measuring Human Heart Rate Utilizing Dual-Branch Architecture and Alternating Frame Shifts in Facial Videos

Convolutional neural network with spatio-temporal-channel attention for remote heart rate estimation

Efficient Spatiotemporal Attention Network for Remote Heart Rate Variability Analysis

Deep Super-Resolution Network for rPPG Information Recovery and Noncontact Heart Rate Estimation

A Feasibility Study of a Video-Based Heart Rate Estimation Method with Convolutional Neural Networks

Lightweight and interpretable convolutional neural network for real-time heart rate monitoring using low-cost video camera under realistic conditions

PulseNet: A multitask learning network for remote heart rate estimation

Video-Based Heart Rate Measurement: Recent Advances and Future Prospects.

Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks

rPPG-MAE: Self-supervised Pre-training with Masked Autoencoders for Remote Physiological Measurement