Abstract:Physical signs of the body that are too subtle to be observed by human eyes can reflect significant health indicators. Although many vision-based approaches have been devoted to recovery, they topically focus on recognizing explicit features such as colors, textures, and patches, and pay less attention to the entanglement and disentanglement among implicit biological characteristics. Meanwhile, existing deep networks for remote physiological detection are generally weak or deliberately neglectful in eliminating long-term time-varying interference and noise. To address these issues, we propose TranPulse, a novel remote estimation paradigm dedicated to video transformer-based polyphysiological disentanglement for robust heart rate (HR) prediction. Specifically, we improve existing single-stage transformer-based cardiac estimation backbones into a targeted two-stage architecture, which consists of a pair of asymmetric encoder and decoder. The encoder, which is the important module for heart waveform prediction, guides global attention to spatio-temporal enhancement and perception of periodic signals according to facial frame differences. The decoder, which is the core strategy pipeline for unwrapping time-varying disturbances, uses waveform gradient variations as the constraint for high-dimensional representations to separate spikes from multiple uncorrelated obstructions. Simultaneously, we integrate the synchronous computation of the encoder's fusion of appearance and frame difference to provide more detailed guidance for in-the-wild spatio-temporal modeling, and design a more reliable regression loss function to coordinate long-term temporal and frame-wise spatial supervision. We train, validate, and practice our proposed model on multiple publicly available datasets, where it achieves competitive performance in pulse estimation by extensive experimental results.

TransPhys: Transformer-based unsupervised contrastive learning for remote heart rate measurement

ST-Phys: Unsupervised Spatio-Temporal Contrastive Remote Physiological Measurement

TranPhys: Spatiotemporal Masked Transformer Steered Remote Photoplethysmography Estimation

Contrast-Phys: Unsupervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast

Contrast-Phys+: Unsupervised and Weakly-supervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast

STFMRPPGFormer:Vision Transformer Network Based on MLP-3D and STFM for Remote Heart Rate Measurement

TransPPG: Two-stream Transformer for Remote Heart Rate Estimate

An effective cross-scenario remote heart rate estimation network based on global-local information and video transformer

rPPG-MAE: Self-supervised Pre-training with Masked Autoencoders for Remote Physiological Measurement

PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer

PhysFormer++: Facial Video-based Physiological Measurement with SlowFast Temporal Difference Transformer

Rppg-Mae: Self-supervised Pretraining with Masked Autoencoders for Remote Physiological Measurements

TranPulse: Remote Photoplethysmography Estimation with Time-Varying Supervision to Disentangle Multi-Physiologically Interference

TCNTransNet: A Semi-Supervised Temporal-Spatial Fusion Framework for Heart Rate Estimation from Camera Video

CDCCA-RPPGFormer: Transformer-Like Network Based on 3D-CDC-ST and ECA for Remote Heart Rate Measurement

Non-Contact Heart Rate Estimation from Photoplethysmography Using EEMD and Convolution-Transformer Network

Video-based Remote Physiological Measurement via Self-supervised Learning

Dual-path TokenLearner for Remote Photoplethysmography-based Physiological Measurement with Facial Videos

Demodulation Based Transformer for rPPG Generation and Heart Rate Estimation

Facial Video-based Remote Physiological Measurement via Self-supervised Learning

Multi-hierarchical Convolutional Network for Efficient Remote Photoplethysmograph Signal and Heart Rate Estimation from Face Video Clips.