Abstract:Remote photoplethysmography (rPPG), which measures human heart rate without physical contact with the skin, has become active research in recent years. Neural networks have been introduced into rPPG for accurate pulse measurement and have achieved overwhelming results. However, there is a lack of in-depth analysis of key components of neural networks exhibiting a crucial impact on pulse extraction from video. In this paper, we present a network with attention and spatio-temporal convolutional block (ASTNet), exploiting the impact of key factors including different spatio-temporal convolutions, attention mechanism, the number of convolutional layers, and receptive field sizes. The novel attention module named spatio-temporal-channel (STC) attention is designed to jointly learn weights in spatial, temporal, and channel dimensions in a more efficient way. Extensive experiments have been conducted on two uncompressed datasets and one compressed dataset. Results show that ASTNet outperforms state-of-the-art methods in accuracy and computational time. Specifically, networks with larger receptive field sizes and more spatio-temporal blocks generally achieve better performance. Networks with pseudo 3D convolution outperform those with convolutional 3D in static videos, and the opposite is true in motion videos. The results exhibit a similar tendency both on uncompressed and compressed datasets. The proposed method improves the performance of pulse signal compared to PhysNet (the second-best approach in the compared methods), with the signal-to-noise ratio increased by 7.03%, 10.19%, 4.79%, the mean absolute error decreased by 17.95%, 14.17%, 22.76%, and the root-mean-square error decreased by 21.43%, 2.73%, 25.43%, on the PURE, Self-rPPG, and COHFACE datasets, respectively.

Robust Heart Rate Estimation with Spatial–Temporal Attention Network from Facial Videos

Heart Rate Estimation From Facial Videos Using a Spatiotemporal Representation With Convolutional Neural Networks

CoSTHR: A Heart Rate Estimating Network with Adaptive Color Space Transformation

Convolutional neural network with spatio-temporal-channel attention for remote heart rate estimation

Multi-hierarchical Convolutional Network for Efficient Remote Photoplethysmograph Signal and Heart Rate Estimation from Face Video Clips.

Deep-HR: Fast Heart Rate Estimation from Face Video under Realistic Conditions

Deep learning-based remote-photoplethysmography measurement from short-time facial video

Efficient Spatiotemporal Attention Network for Remote Heart Rate Variability Analysis

AutoHR: A Strong End-to-End Baseline for Remote Heart Rate Measurement With Neural Searching

RhythmNet: End-to-end Heart Rate Estimation from Face via Spatial-temporal Representation

A robust non-contact heart rate estimation from facial video based on a non-parametric signal extraction model

EVM-CNN: Real-Time Contactless Heart Rate Estimation from Facial Video

Remote Heart Rate Estimation by Signal Quality Attention Network

ACTNet: Attention based CNN and Transformer network for respiratory rate estimation

A Feasibility Study of a Video-Based Heart Rate Estimation Method with Convolutional Neural Networks

Remote Heart Rate Measurement from Face Videos under Realistic Situations

Adaptive Heart Rate Estimation from Face Videos

Lightweight and interpretable convolutional neural network for real-time heart rate monitoring using low-cost video camera under realistic conditions

Robust heart-rate estimation from facial videos using Project_ICA

Heart rate prediction from facial video with masks using eye location and corrected by convolutional neural networks

PulseNet: A multitask learning network for remote heart rate estimation