PhysBench: A Benchmark Framework for rPPG with a New Dataset and Baseline

Kegang Wang,Yantao Wei,Mingwen Tong,Jie Gao,Yi Tian,YuJian Ma,ZhongJin Zhao

DOI: https://doi.org/10.48550/arXiv.2305.04161

2024-11-04

Abstract:In recent years, due to the widespread use of internet videos, physiological remote sensing has gained more and more attention in the fields of affective computing and telemedicine. Recovering physiological signals from facial videos is a challenging task that involves a series of preprocessing, image algorithms, and post-processing to finally restore waveforms. We propose a complete and efficient end-to-end training and testing framework that provides fair comparisons for different algorithms through unified preprocessing and post-processing. In addition, we introduce a highly synchronized lossless format dataset along with a lightweight algorithm. The dataset contains over 32 hours (3.53M frames) of video from 58 subjects; by training on our collected dataset both our proposed algorithm as well as existing ones can achieve improvements.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper attempts to solve several key problems faced by remote photoplethysmography (rPPG) when recovering the blood volume pulse (BVP) signal from facial videos:

1. **Quality and Synchronization of Datasets**: Many existing rPPG datasets have signal loss due to compression and the problem of asynchronization between physiological signals and videos. These problems affect the training effect and generalization ability of algorithms. Therefore, the author proposes a highly synchronized and lossless dataset - Remote Learning Affect and Physiology (RLAP) - to solve these problems.

2. **Lightweight and Efficiency of Algorithms**: In order to enable the rPPG algorithm to run on resource - limited devices, such as mobile devices or embedded systems, the author designs a lightweight model based on one - dimensional convolutional neural network (1D CNN) - Seq - rPPG. This model not only has extremely low computational cost, but also performs as well as or even better than existing complex models on the RLAP dataset.

3. **Unified Experimental Framework**: In order to facilitate the development, training and testing of new models and improve the reproducibility of experiments, the author develops a new training and testing framework - PhysBench. This framework unifies pre - processing and post - processing steps, provides faster training and testing speeds, and supports the convenient addition of multiple datasets, algorithms and evaluation metrics.

By solving the above problems, this paper aims to promote the development of rPPG technology, especially its applications in affective computing and telemedicine.