Abstract:Background Affective states are important aspects of healthy functioning; as such, monitoring and understanding affect is necessary for the assessment and treatment of mood-based disorders. Recent advancements in wearable technologies have increased the use of such tools in detecting and accurately estimating mental states (eg, affect, mood, and stress), offering comprehensive and continuous monitoring of individuals over time. Objective Previous attempts to model an individual’s mental state relied on subjective measurements or the inclusion of only a few objective monitoring modalities (eg, smartphones). This study aims to investigate the capacity of monitoring affect using fully objective measurements. We conducted a comparatively long-term (12-month) study with a holistic sampling of participants’ moods, including 20 affective states. Methods Longitudinal physiological data (eg, sleep and heart rate), as well as daily assessments of affect, were collected using 3 modalities (ie, smartphone, watch, and ring) from 20 college students over a year. We examined the difference between the distributions of data collected from each modality along with the differences between their rates of missingness. Out of the 20 participants, 7 provided us with 200 or more days’ worth of data, and we used this for our predictive modeling setup. Distributions of positive affect (PA) and negative affect (NA) among the 7 selected participants were observed. For predictive modeling, we assessed the performance of different machine learning models, including random forests (RFs), support vector machines (SVMs), multilayer perceptron (MLP), and K-nearest neighbor (KNN). We also investigated the capability of each modality in predicting mood and the most important features of PA and NA RF models. Results RF was the best-performing model in our analysis and performed mood and stress (nervousness) prediction with ~81% and ~72% accuracy, respectively. PA models resulted in better performance compared to NA. The order of the most important modalities in predicting PA and NA was the smart ring, phone, and watch, respectively. SHAP (Shapley Additive Explanations) analysis showed that sleep and activity-related features were the most impactful in predicting PA and NA. Conclusions Generic machine learning–based affect prediction models, trained with population data, outperform existing methods, which use the individual’s historical information. Our findings indicated that our mood prediction method outperformed the existing methods. Additionally, we found that sleep and activity level were the most important features for predicting next-day PA and NA, respectively.

Learning Attention Level Prediction Via Multimodal Physiological Data Using Wearable Wrist Devices

Exploring Skin Potential Signals in Electrodermal Activity: Identifying Key Features for Attention State Differentiation

A Physiological Data-Driven Model for Learners' Cognitive Load Detection Using HRV-PRV Feature Fusion and Optimized XGBoost Classification

Learner's cognitive state recognition based on multimodal physiological signal fusion

Recognizing the Human Attention State Using Cardiac Pulse from the Noncontact and Automatic-Based Measurements

Sustained Attention States Recognition with EEG and Eye-Tracking in the GradCPT.

Evaluating Children’s Engagement Through Wrist-worn Biosensors and LSTM Network

Wearable Device-Based Real-Time Monitoring of Physiological Signals: Evaluating Cognitive Load Across Different Tasks

Emotion Classification with Multi‐modal Physiological Signals Using Multi‐attention‐based Neural Network

Cognitive Load Prediction from Multimodal Physiological Signals Using Multiview Learning.

Combining Facial Behavioral Cues, Eye Movements and EEG-Based Attention to Improve Prediction of Reading Failure

A Data-Driven Framework for Learners' Cognitive Load Detection Using ECG-PPG Physiological Feature Fusion and XGBoost Classification.

Design of Wearable and Portable Physiological Parameter Monitoring System for Attentiveness Evaluation

Learning Immersion Assessment Model Based on Multi-dimensional Physiological Characteristics

Objective Prediction of Next-Day’s Affect Using Multimodal Physiological and Behavioral Data: Algorithm Development and Validation Study

A Deep-Learning Based Method for Analysis of Students’ Attention in Offline Class

Data-driven learning fatigue detection system: A multimodal fusion approach of ECG (electrocardiogram) and video signals

Multimodal Physiological Analysis of Cognitive Affective Effect of Immersive Learning in Virtual Reality

Multi-modality Sensor Data Classification with Selective Attention

A Virtual Reality and Online Learning Immersion Experience Evaluation Model Based on SVM and Wearable Recordings

Wearable Neurophysiological Recordings in Middle-School Classroom Correlate with Students’ Academic Performance