Abstract:Emotional health draws great concern with the enhancement of public health consciousness.Emotional health is closely related to the quality of personal life.Even for some special groups of people,like pilots,soldiers,etc.,their emotional states will have impacts on the stability of communities.Traditionally,to evaluate emotional states of human beings relies on the doctors or psychologists to communicate with subjects and give scores based on various questionnaires.This approach is not scientific enough and leads to the difficulties in the emotional health monitoring in daily-life.Emotion recognition enables lifeless sensors and computers to measure and interpret human emotions.It is a procedure consisting of emotion-related bio signals recording,features extraction and emotional states classification,providing scientific evidence for emotional health monitoring and primary diagnosis of potential mental diseases.In the related works concerning emotion recognition,the application scenarios are usually restricted in the hospitals or labs and the common-used classifiers are not suitable for the daily emotion recognition data set.This paper develops a multimodal biosensor network to simplify the sensing framework so that it can finish emotion recognition tasks when subjects are participating in daily tasks without so many disturbances.Several wearable biosensor nodes record multimodal bio signals (electroencephalography,pulse and blood pressure) and transmit them to a body station employing Arduino UNO3 and its expansion boards.The body station works as a web client connecting to a web data center on the Internet by wireless routers or personal hotspots.The web data center is established on NI-PXI 1065 with a static public IP address.The recognition algorithm is carried out in the data center and the results are displayed for authorized web terminals with the assistance of web publishing service supported by LabVIEW.The multimodal wearable biosensor network can provide emotion-related bio signals from which typical features are extracted based on the existing theories.In particular,due to the uncertainties in signal acquisition and feature extraction,a stacked auto-encoder (based on the deep learning theory) optimized emotion recognition method is proposed to improve the recognition process.The stacked auto-encoder helps to pre-learn the feature vectors and with the fine tuning it generates a better scheme for emotion classification phase.There are 9 emotional states for classification according to the Valence-Arousal dimensional model.A two-layer stacked neural network with a softmax classifier is designed to finish the final classification tasks.The experiment convinces that the feature vectors pre-learned by stacked auto-encoder are of higher quality both in centrality and distinguishability based on the similarity evaluation theory.The final recognition rate is also improved approximately 5％ compared to related works.The main contributions of this paper are the wearable network-based sensing structure,the stacked auto-encoder optimized multimodal emotion recognition algorithm and the quantitative analysis on 71-day experimental data.This is a novel system for daily emotional health monitoring and can provide scientific suggestions for doctors or guardians.However,in the future,large scale of data should be accumulated.Moreover,the dynamic performance and energy efficiency also need improving.

EMSN: an Energy-Efficient Memristive Sequencer Network for Human Emotion Classification in Mental Health Monitoring

Memristive Circuit Design of Sequencer Network for Human Emotion Classification

Memristor-Based Hierarchical Attention Network for Multimodal Affective Computing in Mental Health Monitoring

A Efficient Multimodal Framework for Large Scale Emotion Recognition by Fusing Music and Electrodermal Activity Signals

Temporal Convolutional Network-Enhanced Real-Time Implicit Emotion Recognition with an Innovative Wearable fNIRS-EEG Dual-Modal System

Design and Implementation of a Flexible Neuromorphic Computing System for Affective Communication Via Memristive Circuits

A Neural Network-Based ECG Classification Processor with Exploitation of Heartbeat Similarity

An Optimized Residual Network For Emotion Recognition Based On A Multi-Features Fusion Technology And Electroencephalography Signals

Multi-modal emotion analysis from facial expressions and electroencephalogram.

Memristive Circuit Design for Personalized Emotion Generation with Memory and Retrieval Functions

Stacked Auto-Encoder Optimized Emotion Recognition in Multimodal Wearable Biosensor Network

Design and Implementation of an Emotion Analysis System Based on EEG Signals

Multimodal Neuromorphic Sensory-Processing System With Memristor Circuits for Smart Home Applications

Energy-Efficient Reservoir Computing Based on Solution-Processed Electrolyte/Ferroelectric Memcapacitive Synapses for Biosignal Classification

Arousal-Valence Classification from Peripheral Physiological Signals Using Long Short-Term Memory Networks

MSPAN: A Memristive Spike-Based Computing Engine With Adaptive Neuron for Edge Arrhythmia Detection

A memristor-based associative memory neural network circuit with emotion effect

MEMOS: A Multi-modal Emotion Stream Database for Temporal Spontaneous Emotional State Detection

Energy Efficient ECG Monitoring System for Human Emotional Stress Assessment

Novel multimodal emotion detection method using Electroencephalogram and Electrocardiogram signals

Emotion stimuli-based surface electromyography signal classification employing Markov transition field and deep neural networks