Abstract:Remote heart rate estimation aims to predict cardiac activity signals from facial videos without any physical contact, which has been showing promising results recently. However, existing estimation methods based on deep convolutional networks only focus on the rigid receptive field, while ignoring potential spatial correlations of different facial regions, which obviously cannot reduce the overfitting caused by various noise and motion interference unrelated to cardiac activity. To address these issues, this paper proposes PhysGCN, an end-to-end spatiotemporal graph convolutional network with the hyperbolic embedding, to coordinate the contributions of intra- and inter-frame features of facial videos for long-term heart rate estimation. Specifically, firstly, we convert the facial video captured by the vision system into a graph-structure spatiotemporal map, and use the link set of the graph to determine and lock the spatial relative positions of multiple skin sub-regions formed by intra-frame face segmentation and projection. Secondly, to purify the signal and prevent the interference from heart rate irrelevant features, we integrate and measure the similarity between sub-regions within the graph in a non-Euclidean space by a hyperbolic embedding module, which can characterize the correlation more distinctly compared to the plane space. Finally, we dynamically and elaborately orchestrate the inherent temporal and learned spatial features in a graph convolutional module to obtain reliable heart rate waveforms. We conduct extensive comparative experiments and ablation studies on multiple public datasets to verify the superiority and robustness of our method. Experiments show that our method can effectively estimate heart rate from facial videos, and its performance surpasses or matches the state-of-the-art methods.

Channel-Wise Interactive Learning for Remote Heart Rate Estimation From Facial Video

Heart Rate Estimation From Facial Videos Using a Spatiotemporal Representation With Convolutional Neural Networks

Information-enhanced Network for Noncontact Heart Rate Estimation from Facial Videos

Multi-hierarchical Convolutional Network for Efficient Remote Photoplethysmograph Signal and Heart Rate Estimation from Face Video Clips.

Non-contact PPG Signal and Heart Rate Estimation with Multi-Hierarchical Convolutional Network

Towards Practical Facial Video-based Remote Heart Rate Estimation Via Cross Domain Rppg Adaptation.

Rppg-Based Heart Rate Estimation Using Spatial-Temporal Attention Network

Deep learning-based remote-photoplethysmography measurement from short-time facial video

Remote Heart Rate Measurement from Face Videos under Realistic Situations

TCNTransNet: A Semi-Supervised Temporal-Spatial Fusion Framework for Heart Rate Estimation from Camera Video

ETA-rPPGNet: Effective Time-Domain Attention Network for Remote Heart Rate Measurement.

Hyperbolic Embedding Steered Spatiotemporal Graph Convolutional Network for Video-Based Remote Heart Rate Estimation

Robust heart-rate estimation from facial videos using Project_ICA

Deep-HR: Fast Heart Rate Estimation from Face Video under Realistic Conditions

ECG Signal Reconstruction Based on Facial Videos Via Combined Explicit and Implicit Supervision.

Heart rate estimation by leveraging static and dynamic region weights

Heart Rate Measurement Based on 3D Central Difference Convolution with Attention Mechanism

Remote Heart Rate Measurement from Highly Compressed Facial Videos: an End-to-end Deep Learning Solution with Video Enhancement

A Feasibility Study of a Video-Based Heart Rate Estimation Method with Convolutional Neural Networks

Convolutional neural network with spatio-temporal-channel attention for remote heart rate estimation

Robust Heart Rate Estimation with Spatial–Temporal Attention Network from Facial Videos