A Self-supervised Framework for Improved Data-Driven Monitoring of Stress via Multi-modal Passive Sensing

Shayan Fazeli,Lionel Levine,Mehrab Beikzadeh,Baharan Mirzasoleiman,Bita Zadeh,Tara Peris,Majid Sarrafzadeh
2023-03-25
Abstract:Recent advances in remote health monitoring systems have significantly benefited patients and played a crucial role in improving their quality of life. However, while physiological health-focused solutions have demonstrated increasing success and maturity, mental health-focused applications have seen comparatively limited success in spite of the fact that stress and anxiety disorders are among the most common issues people deal with in their daily lives. In the hopes of furthering progress in this domain through the development of a more robust analytic framework for the measurement of indicators of mental health, we propose a multi-modal semi-supervised framework for tracking physiological precursors of the stress response. Our methodology enables utilizing multi-modal data of differing domains and resolutions from wearable devices and leveraging them to map short-term episodes to semantically efficient embeddings for a given task. Additionally, we leverage an inter-modality contrastive objective, with the advantages of rendering our framework both modular and scalable. The focus on optimizing both local and global aspects of our embeddings via a hierarchical structure renders transferring knowledge and compatibility with other devices easier to achieve. In our pipeline, a task-specific pooling based on an attention mechanism, which estimates the contribution of each modality on an instance level, computes the final embeddings for observations. This additionally provides a thorough diagnostic insight into the data characteristics and highlights the importance of signals in the broader view of predicting episodes annotated per mental health status. We perform training experiments using a corpus of real-world data on perceived stress, and our results demonstrate the efficacy of the proposed approach in performance improvements.
Machine Learning
What problem does this paper attempt to address?
The main problem that this paper attempts to solve is: **How to improve stress monitoring through multi - modal passive sensing data, especially in the case of a lack of effective methods in the field of remote mental health monitoring**. Specifically, the author points out several challenges in the current field of mental health monitoring: 1. **Limited success of data - driven stress and anxiety monitoring applications**: Although solutions related to physical health have achieved certain success, the applications of mental health monitoring are relatively lagging behind. 2. **Unclear statistical sufficiency of data**: From a data - driven perspective, it is not always intuitively clear whether the observed data is sufficient for accurately assessing mental health conditions (for example, whether the number of phone calls can infer the depressive state). 3. **Difficulty in data acquisition**: Due to privacy and regulatory issues, data that can support artificial intelligence applications are often not easily available or are restricted. 4. **Limitations of existing methods**: Many existing methods only rely on high - level labels (such as classifying patients as bipolar disorder or healthy), ignoring the specific situations of short - term emotional reactions such as stress. To solve these problems, the author proposes a **multi - modal self - supervised learning framework**, aiming to learn the context of stress reactions through continuous physiological readings. This framework has the following characteristics: - **Modularity and extensibility**: It can handle multi - modal data from different sources and resolutions and is suitable for various wearable devices. - **Self - supervised learning component**: It improves training efficiency, especially in the case of limited labeled data. - **Attention mechanism**: It provides a diagnostic view of the system, enabling researchers to gain in - depth understanding of the relationships between different data patterns and enhancing the interpretability of the model. - **Hierarchical structure optimization**: By optimizing local and global features, it makes knowledge transfer and compatibility with other devices easier to achieve. In conclusion, the goal of this research is to provide more effective remote mental health management tools by improving the real - time monitoring of stress and other mental health.