Abstract:Recent work has shown that a person's sympathetic arousal can be estimated from facial videos alone using basic signal processing. This opens up new possibilities in the field of telehealth and stress management, providing a non-invasive method to measure stress only using a regular RGB camera. In this paper, we present SympCam, a new 3D convolutional architecture tailored to the task of remote sympathetic arousal prediction. Our model incorporates a temporal attention module (TAM) to enhance the temporal coherence of our sequential data processing capabilities. The predictions from our method improve accuracy metrics of sympathetic arousal in prior work by 48% to a mean correlation of 0.77. We additionally compare our method with common remote photoplethysmography (rPPG) networks and show that they alone cannot accurately predict sympathetic arousal "out-of-the-box". Furthermore, we show that the sympathetic arousal predicted by our method allows detecting physical stress with a balanced accuracy of 90% - an improvement of 61% compared to the rPPG method commonly used in related work, demonstrating the limitations of using rPPG alone. Finally, we contribute a dataset designed explicitly for the task of remote sympathetic arousal prediction. Our dataset contains synchronized face and hand videos of 20 participants from two cameras synchronized with electrodermal activity (EDA) and photoplethysmography (PPG) measurements. We will make this dataset available to the community and use it to evaluate the methods in this paper. To the best of our knowledge, this is the first dataset available to other researchers designed for remote sympathetic arousal prediction.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the problem of remotely measuring sympathetic arousal through facial videos. Specifically, the author proposes a new 3D convolutional neural network architecture (SympCam), combined with a temporal attention module (TAM), to improve the accuracy of predicting sympathetic arousal from facial videos. #### Main problems and background 1. **Limitations of existing methods**: - Traditional methods rely on contact - based sensors (such as electrodermal activity, EDA) to measure sympathetic arousal. These sensors need to be worn on the body, which limits their wide application. - Although remote photoplethysmography (rPPG) can measure physiological signals such as heart rate, it cannot accurately predict sympathetic arousal when used alone. 2. **Research motivation**: - Provide a non - invasive method based on ordinary RGB cameras to measure sympathetic arousal, which will provide new possibilities for telemedicine and stress management. - Solve the problems of high standard deviation and large inter - individual differences in existing methods, and improve the stability and accuracy of prediction. #### Specific objectives 1. **Develop a new model**: - Propose a new 3D convolutional neural network architecture named SympCam, which is specifically used for remote sympathetic arousal prediction. - Introduce a temporal attention module (TAM) to enhance the ability to process time - series data, thereby improving the accuracy of prediction. 2. **Create a dedicated dataset**: - Construct and release a dataset specifically designed for remote sympathetic arousal prediction, which contains synchronous facial and hand videos of 20 participants as well as electrodermal activity (EDA) and photoplethysmography (PPG) measurement data. 3. **Evaluate and compare**: - Evaluate the performance of the new model through cross - validation and compare it with the existing rPPG method to prove its superiority. - Demonstrate the effectiveness of the new method in detecting physical stress (such as stress responses caused by pain), achieving a balanced accuracy of 90%, which is 61% higher than that of existing methods. ### Conclusion By introducing SympCam and its accompanying temporal attention module (TAM), the author has successfully improved the accuracy of predicting sympathetic arousal from facial videos and demonstrated the potential application value of this method in telemedicine and stress management. In addition, they also released the first dataset specifically designed for this task, providing a valuable resource for future research.

SympCam: Remote Optical Measurement of Sympathetic Arousal

Facial-Video-Based Physiological Signal Measurement: Recent Advances and Affective Applications

EmpathicSchool: A multimodal dataset for real-time facial expressions and physiological data analysis under different stress conditions

Remote Heart Rate Measurement from Face Videos under Realistic Situations

The Real-Time Image Sequences-Based Stress Assessment Vision System for Mental Health

Exploring the Feasibility of Seamless Remote Heart Rate Measurement Using Multiple Synchronized Cameras

The physiological measurement and evaluation of empathy of video content

Multimodal Affective State Assessment Using fNIRS + EEG and Spontaneous Facial Expression

Classification of emotional states via transdermal cardiovascular spatiotemporal facial patterns using multispectral face videos

EmoWear: Wearable Physiological and Motion Dataset for Emotion Recognition and Context Awareness

3D Convolutional Neural Networks for Remote Pulse Rate Measurement and Mapping from Facial Video

Camera-Based Remote Physiology Sensing for Hundreds of Subjects Across Skin Tones

Evaluating Visual Photoplethysmography Method

Introducing WESAD, a Multimodal Dataset for Wearable Stress and Affect Detection

A robust non-contact heart rate estimation from facial video based on a non-parametric signal extraction model

Emotion Classification Based on Pulsatile Images Extracted from Short Facial Videos via Deep Learning

Your blush gives you away: detecting hidden mental states with remote photoplethysmography and thermal imaging

Hybrid Spatiotemporal Models for Sentiment Classification Via Galvanic Skin Response

eSEE-d: Emotional State Estimation Based on Eye-Tracking Dataset

Convolutional neural network with spatio-temporal-channel attention for remote heart rate estimation