Abstract:Remote photoplethysmography (rPPG), which measures human heart rate without physical contact with the skin, has become active research in recent years. Neural networks have been introduced into rPPG for accurate pulse measurement and have achieved overwhelming results. However, there is a lack of in-depth analysis of key components of neural networks exhibiting a crucial impact on pulse extraction from video. In this paper, we present a network with attention and spatio-temporal convolutional block (ASTNet), exploiting the impact of key factors including different spatio-temporal convolutions, attention mechanism, the number of convolutional layers, and receptive field sizes. The novel attention module named spatio-temporal-channel (STC) attention is designed to jointly learn weights in spatial, temporal, and channel dimensions in a more efficient way. Extensive experiments have been conducted on two uncompressed datasets and one compressed dataset. Results show that ASTNet outperforms state-of-the-art methods in accuracy and computational time. Specifically, networks with larger receptive field sizes and more spatio-temporal blocks generally achieve better performance. Networks with pseudo 3D convolution outperform those with convolutional 3D in static videos, and the opposite is true in motion videos. The results exhibit a similar tendency both on uncompressed and compressed datasets. The proposed method improves the performance of pulse signal compared to PhysNet (the second-best approach in the compared methods), with the signal-to-noise ratio increased by 7.03%, 10.19%, 4.79%, the mean absolute error decreased by 17.95%, 14.17%, 22.76%, and the root-mean-square error decreased by 21.43%, 2.73%, 25.43%, on the PURE, Self-rPPG, and COHFACE datasets, respectively.

A Feasibility Study of a Video-Based Heart Rate Estimation Method with Convolutional Neural Networks

Heart Rate Estimation From Facial Videos Using a Spatiotemporal Representation With Convolutional Neural Networks

Lightweight and interpretable convolutional neural network for real-time heart rate monitoring using low-cost video camera under realistic conditions

Deep-HR: Fast Heart Rate Estimation from Face Video under Realistic Conditions

Deep learning-based remote-photoplethysmography measurement from short-time facial video

Convolutional neural network with spatio-temporal-channel attention for remote heart rate estimation

3D Convolutional Neural Networks for Remote Pulse Rate Measurement and Mapping from Facial Video

Multi-hierarchical Convolutional Network for Efficient Remote Photoplethysmograph Signal and Heart Rate Estimation from Face Video Clips.

Non-contact heart rate estimation based on singular spectrum component reconstruction using low-rank matrix and autocorrelation

A robust non-contact heart rate estimation from facial video based on a non-parametric signal extraction model

A machine learning-based approach for constructing remote photoplethysmogram signals from video cameras

An effective cross-scenario remote heart rate estimation network based on global-local information and video transformer

PulseNet: A multitask learning network for remote heart rate estimation

Remote Heart Rate Measurement from Face Videos under Realistic Situations

Robust Efficient Estimation of Heart Rate Pulse from Video.

A novel one-stage framework for visual pulse rate estimation using deep neural networks

GRGB rPPG: An Efficient Low-Complexity Remote Photoplethysmography-Based Algorithm for Heart Rate Estimation

EVM-CNN: Real-Time Contactless Heart Rate Estimation from Facial Video

RhythmNet: End-to-end Heart Rate Estimation from Face via Spatial-temporal Representation

Efficient Real-Time Camera Based Estimation of Heart Rate and Its Variability

Motion Resistant Facial Video based Heart Rate Estimation Method using Head-Mounted Camera