Abstract:Contactless physiological signal measurement has great applications in various fields, such as affective computing and health monitoring. Physiological measurements based on remote photoplethysmography (rPPG) are realized by capturing the weak periodic color changes. The changes are caused by the variation in the light absorption of skin surface during systole and diastole stages of a functioning heart. This measurement mode has advantages of contactless measurement, simple operation, low cost, etc. In recent years, several deep learning-based rPPG measurement methods have been proposed. However, the features learned by deep learning models are vulnerable to motion and illumination artefacts, and are unable to fully exploit the intrinsic temporal characteristics of the rPPG. This paper presents an efficient spatiotemporal modeling-based rPPG recovery method for physiological signal measurements. First, two modules are utilized in the rPPG task: 1) 3D central difference convolution for temporal context modeling with enhanced representation and generalization capacity, and 2) Huber loss for robust intensity-level rPPG recovery. Second, a dual branch structure for both motion and appearance modeling and a soft attention mask are adapted to take full advantage of the central difference convolution. Third, a multi-task setting for joint cardiac and respiratory signals measurements is introduced to benefit from the internal relevance between two physiological signals. Last, extensive experiments performed on three public databases show that the proposed method outperforms prior state-of-the-art methods with the Pearson's correlation coefficient higher than 0.96 on all three datasets. The generalization ability of the proposed method is also evaluated by cross-database and video compression experiments. The effectiveness and necessity of each module are confirmed by ablation studies.

Learning Spatio-Temporal Pulse Representation With Global-Local Interaction and Supervision for Remote Prediction of Heart Rate

Convolutional neural network with spatio-temporal-channel attention for remote heart rate estimation

PulseNet: A multitask learning network for remote heart rate estimation

An effective cross-scenario remote heart rate estimation network based on global-local information and video transformer

Rppg-Based Heart Rate Estimation Using Spatial-Temporal Attention Network

Heart Rate Estimation From Facial Videos Using a Spatiotemporal Representation With Convolutional Neural Networks

Multi-hierarchical Convolutional Network for Efficient Remote Photoplethysmograph Signal and Heart Rate Estimation from Face Video Clips.

An End-to-end Efficient Framework for Remote Physiological Signal Sensing

STGNet: Spatio-temporal graph neural networks considering inherent properties of physiological signals for camera-based remote photoplethysmography

Remote physiological signal recovery with efficient spatio-temporal modeling

Efficient Spatiotemporal Attention Network for Remote Heart Rate Variability Analysis

JAMSNet: A Remote Pulse Extraction Network Based on Joint Attention and Multi-Scale Fusion

MDAR: A Multiscale Features-Based Network for Remotely Measuring Human Heart Rate Utilizing Dual-Branch Architecture and Alternating Frame Shifts in Facial Videos

AutoHR: A Strong End-to-End Baseline for Remote Heart Rate Measurement With Neural Searching

DSE-NN: Deeply Supervised Efficient Neural Network for Real-Time Remote Photoplethysmography

A Feasibility Study of a Video-Based Heart Rate Estimation Method with Convolutional Neural Networks

Robust Remote Photoplethysmography Estimation With Environmental Noise Disentanglement

X-iPPGNet: A novel one stage deep learning architecture based on depthwise separable convolutions for video-based pulse rate estimation

High-accuracy heart rate detection using multispectral IPPG technology combined with a deep learning algorithm

Remote Heart Rate Estimation by Signal Quality Attention Network

Rppg-Hiba:Hierarchical Balanced Framework for Remote Physiological Measurement