Abstract:Physical signs of the body that are too subtle to be observed by human eyes can reflect significant health indicators. Although many vision-based approaches have been devoted to recovery, they topically focus on recognizing explicit features such as colors, textures, and patches, and pay less attention to the entanglement and disentanglement among implicit biological characteristics. Meanwhile, existing deep networks for remote physiological detection are generally weak or deliberately neglectful in eliminating long-term time-varying interference and noise. To address these issues, we propose TranPulse, a novel remote estimation paradigm dedicated to video transformer-based polyphysiological disentanglement for robust heart rate (HR) prediction. Specifically, we improve existing single-stage transformer-based cardiac estimation backbones into a targeted two-stage architecture, which consists of a pair of asymmetric encoder and decoder. The encoder, which is the important module for heart waveform prediction, guides global attention to spatio-temporal enhancement and perception of periodic signals according to facial frame differences. The decoder, which is the core strategy pipeline for unwrapping time-varying disturbances, uses waveform gradient variations as the constraint for high-dimensional representations to separate spikes from multiple uncorrelated obstructions. Simultaneously, we integrate the synchronous computation of the encoder's fusion of appearance and frame difference to provide more detailed guidance for in-the-wild spatio-temporal modeling, and design a more reliable regression loss function to coordinate long-term temporal and frame-wise spatial supervision. We train, validate, and practice our proposed model on multiple publicly available datasets, where it achieves competitive performance in pulse estimation by extensive experimental results.

Demodulation Based Transformer for rPPG Generation and Heart Rate Estimation

STFMRPPGFormer:Vision Transformer Network Based on MLP-3D and STFM for Remote Heart Rate Measurement

TransPPG: Two-stream Transformer for Remote Heart Rate Estimate

TranPhys: Spatiotemporal Masked Transformer Steered Remote Photoplethysmography Estimation

TCNTransNet: A Semi-Supervised Temporal-Spatial Fusion Framework for Heart Rate Estimation from Camera Video

Non-Contact Heart Rate Estimation from Photoplethysmography Using EEMD and Convolution-Transformer Network

An effective cross-scenario remote heart rate estimation network based on global-local information and video transformer

CDCCA-RPPGFormer: Transformer-Like Network Based on 3D-CDC-ST and ECA for Remote Heart Rate Measurement

Information-enhanced Network for Noncontact Heart Rate Estimation from Facial Videos

Heart Rate Estimation From Facial Videos Using a Spatiotemporal Representation With Convolutional Neural Networks

Multi-hierarchical Convolutional Network for Efficient Remote Photoplethysmograph Signal and Heart Rate Estimation from Face Video Clips.

Remote Heart Rate Estimation Based on Transformer with Multi-Skip Connection Decoder: Method and Evaluation in the Wild

Non-contact PPG Signal and Heart Rate Estimation with Multi-Hierarchical Convolutional Network

TransPhys: Transformer-based unsupervised contrastive learning for remote heart rate measurement

TranPulse: Remote Photoplethysmography Estimation with Time-Varying Supervision to Disentangle Multi-Physiologically Interference

Heart rate estimation by leveraging static and dynamic region weights

Deep-HR: Fast Heart Rate Estimation from Face Video under Realistic Conditions

Heart rate estimation based on face video under unstable illumination

Rppg-Based Heart Rate Estimation Using Spatial-Temporal Attention Network

Deep learning-based remote-photoplethysmography measurement from short-time facial video

A Feasibility Study of a Video-Based Heart Rate Estimation Method with Convolutional Neural Networks