Abstract:Remote photoplethysmography (rPPG), which measures human heart rate without physical contact with the skin, has become active research in recent years. Neural networks have been introduced into rPPG for accurate pulse measurement and have achieved overwhelming results. However, there is a lack of in-depth analysis of key components of neural networks exhibiting a crucial impact on pulse extraction from video. In this paper, we present a network with attention and spatio-temporal convolutional block (ASTNet), exploiting the impact of key factors including different spatio-temporal convolutions, attention mechanism, the number of convolutional layers, and receptive field sizes. The novel attention module named spatio-temporal-channel (STC) attention is designed to jointly learn weights in spatial, temporal, and channel dimensions in a more efficient way. Extensive experiments have been conducted on two uncompressed datasets and one compressed dataset. Results show that ASTNet outperforms state-of-the-art methods in accuracy and computational time. Specifically, networks with larger receptive field sizes and more spatio-temporal blocks generally achieve better performance. Networks with pseudo 3D convolution outperform those with convolutional 3D in static videos, and the opposite is true in motion videos. The results exhibit a similar tendency both on uncompressed and compressed datasets. The proposed method improves the performance of pulse signal compared to PhysNet (the second-best approach in the compared methods), with the signal-to-noise ratio increased by 7.03%, 10.19%, 4.79%, the mean absolute error decreased by 17.95%, 14.17%, 22.76%, and the root-mean-square error decreased by 21.43%, 2.73%, 25.43%, on the PURE, Self-rPPG, and COHFACE datasets, respectively.

ACTNet: Attention based CNN and Transformer network for respiratory rate estimation

CoSTHR: A Heart Rate Estimating Network with Adaptive Color Space Transformation

TCNTransNet: A Semi-Supervised Temporal-Spatial Fusion Framework for Heart Rate Estimation from Camera Video

Robust Heart Rate Estimation with Spatial–Temporal Attention Network from Facial Videos

CDCCA-RPPGFormer: Transformer-Like Network Based on 3D-CDC-ST and ECA for Remote Heart Rate Measurement

ETA-rPPGNet: Effective Time-Domain Attention Network for Remote Heart Rate Measurement.

Convolutional neural network with spatio-temporal-channel attention for remote heart rate estimation

Dual Attention Network for Heart Rate and Respiratory Rate Estimation

Non-Contact Heart Rate Estimation from Photoplethysmography Using EEMD and Convolution-Transformer Network

Multi-hierarchical Convolutional Network for Efficient Remote Photoplethysmograph Signal and Heart Rate Estimation from Face Video Clips.

Heart Rate Estimation From Facial Videos Using a Spatiotemporal Representation With Convolutional Neural Networks

ECG Signal Reconstruction Based on Facial Videos Via Combined Explicit and Implicit Supervision.

Deep-HR: Fast Heart Rate Estimation from Face Video under Realistic Conditions

Video-Based Multiphysiological Disentanglement and Remote Robust Estimation for Respiration

Channel-Wise Interactive Learning for Remote Heart Rate Estimation From Facial Video

STFMRPPGFormer:Vision Transformer Network Based on MLP-3D and STFM for Remote Heart Rate Measurement

Efficient Spatiotemporal Attention Network for Remote Heart Rate Variability Analysis

Heart Rate Estimation Via Self-Adaptive Region Selection and Multiregion-Fusion 1D CNN

Rppg-Based Heart Rate Estimation Using Spatial-Temporal Attention Network

An effective cross-scenario remote heart rate estimation network based on global-local information and video transformer

Information-enhanced Network for Noncontact Heart Rate Estimation from Facial Videos