Abstract:Remote heart rate estimation aims to predict cardiac activity signals from facial videos without any physical contact, which has been showing promising results recently. However, existing estimation methods based on deep convolutional networks only focus on the rigid receptive field, while ignoring potential spatial correlations of different facial regions, which obviously cannot reduce the overfitting caused by various noise and motion interference unrelated to cardiac activity. To address these issues, this paper proposes PhysGCN, an end-to-end spatiotemporal graph convolutional network with the hyperbolic embedding, to coordinate the contributions of intra- and inter-frame features of facial videos for long-term heart rate estimation. Specifically, firstly, we convert the facial video captured by the vision system into a graph-structure spatiotemporal map, and use the link set of the graph to determine and lock the spatial relative positions of multiple skin sub-regions formed by intra-frame face segmentation and projection. Secondly, to purify the signal and prevent the interference from heart rate irrelevant features, we integrate and measure the similarity between sub-regions within the graph in a non-Euclidean space by a hyperbolic embedding module, which can characterize the correlation more distinctly compared to the plane space. Finally, we dynamically and elaborately orchestrate the inherent temporal and learned spatial features in a graph convolutional module to obtain reliable heart rate waveforms. We conduct extensive comparative experiments and ablation studies on multiple public datasets to verify the superiority and robustness of our method. Experiments show that our method can effectively estimate heart rate from facial videos, and its performance surpasses or matches the state-of-the-art methods.

Convolutional neural network with spatio-temporal-channel attention for remote heart rate estimation

Heart Rate Measurement Based on 3D Central Difference Convolution with Attention Mechanism

CoSTHR: A Heart Rate Estimating Network with Adaptive Color Space Transformation

Heart Rate Estimation From Facial Videos Using a Spatiotemporal Representation With Convolutional Neural Networks

Rppg-Based Heart Rate Estimation Using Spatial-Temporal Attention Network

Robust Heart Rate Estimation with Spatial–Temporal Attention Network from Facial Videos

Efficient Spatiotemporal Attention Network for Remote Heart Rate Variability Analysis

Multi-hierarchical Convolutional Network for Efficient Remote Photoplethysmograph Signal and Heart Rate Estimation from Face Video Clips.

ETA-rPPGNet: Effective Time-Domain Attention Network for Remote Heart Rate Measurement.

PulseNet: A multitask learning network for remote heart rate estimation

A Feasibility Study of a Video-Based Heart Rate Estimation Method with Convolutional Neural Networks

Non-contact PPG Signal and Heart Rate Estimation with Multi-Hierarchical Convolutional Network

STGNet: Spatio-temporal graph neural networks considering inherent properties of physiological signals for camera-based remote photoplethysmography

STFMRPPGFormer:Vision Transformer Network Based on MLP-3D and STFM for Remote Heart Rate Measurement

TCNTransNet: A Semi-Supervised Temporal-Spatial Fusion Framework for Heart Rate Estimation from Camera Video

Non-Contact Heart Rate Estimation from Photoplethysmography Using EEMD and Convolution-Transformer Network

CDCCA-RPPGFormer: Transformer-Like Network Based on 3D-CDC-ST and ECA for Remote Heart Rate Measurement

TranPhys: Spatiotemporal Masked Transformer Steered Remote Photoplethysmography Estimation

Learning Spatio-Temporal Pulse Representation With Global-Local Interaction and Supervision for Remote Prediction of Heart Rate

Hyperbolic Embedding Steered Spatiotemporal Graph Convolutional Network for Video-Based Remote Heart Rate Estimation

JAMSNet: A Remote Pulse Extraction Network Based on Joint Attention and Multi-Scale Fusion