Universal serologic reaction with lipid antigen; in tuberculosis.

R. L. Kahn

DOI: https://doi.org/10.1093/AJCP/19.5.419

1949-05-01

American Journal of Clinical Pathology

Abstract:

What problem does this paper attempt to address?

Arousal and Valence Classification Model Based on Long Short-Term Memory and DEAP Data for Mental Healthcare Management

Eun Jeong Choi,Dong Keun Kim

DOI: https://doi.org/10.4258/hir.2018.24.4.309

2018-01-01

Healthcare Informatics Research

Abstract:OBJECTIVES: Both the valence and arousal components of affect are important considerations when managing mental healthcare because they are associated with affective and physiological responses. Research on arousal and valence analysis, which uses images, texts, and physiological signals that employ deep learning, is actively underway; research investigating how to improve the recognition rate is needed. The goal of this research was to design a deep learning framework and model to classify arousal and valence, indicating positive and negative degrees of emotion as high or low.METHODS: The proposed arousal and valence classification model to analyze the affective state was tested using data from 40 channels provided by a dataset for emotion analysis using electrocardiography (EEG), physiological, and video signals (the DEAP dataset). Experiments were based on 10 selected featured central and peripheral nervous system data points, using long short-term memory (LSTM) as a deep learning method.RESULTS: The arousal and valence were classified and visualized on a two-dimensional coordinate plane. Profiles were designed depending on the number of hidden layers, nodes, and hyperparameters according to the error rate. The experimental results show an arousal and valence classification model accuracy of 74.65 and 78%, respectively. The proposed model performed better than previous other models.CONCLUSIONS: The proposed model appears to be effective in analyzing arousal and valence; specifically, it is expected that affective analysis using physiological signals based on LSTM will be possible without manual feature extraction. In a future study, the classification model will be adopted in mental healthcare management systems.
An Efficient Machine Learning-Based Emotional Valence Recognition Approach Towards Wearable EEG

Lamiaa Abdel-Hamid

DOI: https://doi.org/10.3390/s23031255

IF: 3.9

2023-01-22

Sensors

Abstract:Emotion artificial intelligence (AI) is being increasingly adopted in several industries such as healthcare and education. Facial expressions and tone of speech have been previously considered for emotion recognition, yet they have the drawback of being easily manipulated by subjects to mask their true emotions. Electroencephalography (EEG) has emerged as a reliable and cost-effective method to detect true human emotions. Recently, huge research effort has been put to develop efficient wearable EEG devices to be used by consumers in out of the lab scenarios. In this work, a subject-dependent emotional valence recognition method is implemented that is intended for utilization in emotion AI applications. Time and frequency features were computed from a single time series derived from the Fp1 and Fp2 channels. Several analyses were performed on the strongest valence emotions to determine the most relevant features, frequency bands, and EEG timeslots using the benchmark DEAP dataset. Binary classification experiments resulted in an accuracy of 97.42% using the alpha band, by that outperforming several approaches from literature by ~3–22%. Multiclass classification gave an accuracy of 95.0%. Feature computation and classification required less than 0.1 s. The proposed method thus has the advantage of reduced computational complexity as, unlike most methods in the literature, only two EEG channels were considered. In addition, minimal features concluded from the thorough analyses conducted in this study were used to achieve state-of-the-art performance. The implemented EEG emotion recognition method thus has the merits of being reliable and easily reproducible, making it well-suited for wearable EEG devices.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
A Comparative Study of Machine Learning Techniques for Emotion Recognition from Peripheral Physiological Signals

Sowmya Vijayakumar,Ronan Flynn,Niall Murray

DOI: https://doi.org/10.1109/ISSC49989.2020.9180193

2020-06-01

Abstract:Recent developments in wearable technology have led to increased research interest in using peripheral physiological signals for emotion recognition. The non-invasive nature of peripheral physiological signal measurement via wearables enables ecologically valid long-term monitoring. These peripheral signal measurements can be used in real-time in many ways including health and emotion classification. This paper investigates the utility of peripheral physiological signals for emotion recognition using the publicly available DEAP database. Using this database (which contains electroencephalogram (EEG) signals and peripheral signals), this paper compares eight machine learning models in the classification of valence and arousal emotion dimensions. These were applied to the peripheral physiological signals only. These models operate on three groupings of the peripheral data: (i) the raw peripheral physiological signals; (ii) individual feature sets extracted from each peripheral signal; and (iii) a fusion data set made of the combined features from the individual peripheral signals. The results indicate that support vector machine, linear discriminant analysis and logistic regression give the best recognition results on all three data groups considered. The feature fusion data set, which is made up by fusing all the features from the peripheral signals, gives the best recognition accuracy on both valence and arousal dimensions. In addition, subject dependency for emotion classification from peripheral signals is examined and significant individual variability is observed. The recognition rate varies between each participant from 10% to 87.5%.

Computer Science
End-to-End Emotion Recognition using Peripheral Physiological Signals

M. Pidgeon,N. Kanwal,N. Murray,M. Asghar

DOI: https://doi.org/10.14236/ewic/hci2022.19

2022-07-01

Abstract:The study of emotion recognition from physiological signals has seen a huge growth in recent decades. Studies initially used traditional machine learning classiﬁcation to estimate either discrete emotions or combinations of arousal and valence. However, different feature engineering techniques such as Ensemble Empirical Mode Decomposition (EEMD) Analysis for electroencephalography (EEG) signals and statistical calculations for peripheral signals are used prior to machine learning. Also, several sensors needed to be worn by participants to predict the emotions. This study aims to investigate whether arousal and valence can be predicted from a single peripheral signal using deep learning. The Galvanic Skin Response(GSR), Respiration (RSP), Blood Volume Pulse (BVP) and Temperature (Temp) signals from the DEAP dataset are used. The signals are downsampled to approximately three hertz (Hz) and input to a convolutional network (CNN) to predict arousal and valence. GSR, RSP and BVP had similar F1 and accuracy results. BVP had an F1 result of 0.673 and 0.632 and accuracies of 63.5% and 61.1% respectively for arousal and valence. RSP’s F1 results were 0.677 and 0.669 and accuracies were 61.3% and 64.2% for arousal and valence respectively. GSR had F1 results of 0.699 and 0.663 and accuracies of 62.5% and 60.2% respectively for arousal and valence. Using raw signals and examining the peripheral signals individually, we were able to identify which sensors showed the best potential for further research to bring emotion classiﬁcation into a real-world scenario using non-invasive sensors.

Computer Science
Temporal Convolutional Network-Enhanced Real-Time Implicit Emotion Recognition with an Innovative Wearable fNIRS-EEG Dual-Modal System

Jiafa Chen,Kaiwei Yu,Fei Wang,Zhengxian Zhou,Yifei Bi,Songlin Zhuang,Dawei Zhang

DOI: https://doi.org/10.3390/electronics13071310

IF: 2.9

2024-04-01

Electronics

Abstract:Emotion recognition remains an intricate task at the crossroads of psychology and artificial intelligence, necessitating real-time, accurate discernment of implicit emotional states. Here, we introduce a pioneering wearable dual-modal device, synergizing functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) to meet this demand. The first-of-its-kind fNIRS-EEG ensemble exploits a temporal convolutional network (TC-ResNet) that takes 24 fNIRS and 16 EEG channels as input for the extraction and recognition of emotional features. Our system has many advantages including its portability, battery efficiency, wireless capabilities, and scalable architecture. It offers a real-time visual interface for the observation of cerebral electrical and hemodynamic changes, tailored for a variety of real-world scenarios. Our approach is a comprehensive emotional detection strategy, with new designs in system architecture and deployment and improvement in signal processing and interpretation. We examine the interplay of emotions and physiological responses to elucidate the cognitive processes of emotion regulation. An extensive evaluation of 30 subjects under four emotion induction protocols demonstrates our bimodal system's excellence in detecting emotions, with an impressive classification accuracy of 99.81% and its ability to reveal the interconnection between fNIRS and EEG signals. Compared with the latest unimodal identification methods, our bimodal approach shows significant accuracy gains of 0.24% for EEG and 8.37% for fNIRS. Moreover, our proposed TC-ResNet-driven temporal convolutional fusion technique outperforms conventional EEG-fNIRS fusion methods, improving the recognition accuracy from 0.7% to 32.98%. This research presents a groundbreaking advancement in affective computing that combines biological engineering and artificial intelligence. Our integrated solution facilitates nuanced and responsive affective intelligence in practical applications, with far-reaching impacts on personalized healthcare, education, and human–computer interaction paradigms.

engineering, electrical & electronic,computer science, information systems,physics, applied
Decoding Emotional Experience through Physiological Signal Processing

Maria S. Perez-Rosero,Behnaz Rezaei,Murat Akcakaya,Sarah Ostadabbas

DOI: https://doi.org/10.48550/arXiv.1606.00370

2016-06-02

Abstract:There is an increasing consensus among re- searchers that making a computer emotionally intelligent with the ability to decode human affective states would allow a more meaningful and natural way of human-computer interactions (HCIs). One unobtrusive and non-invasive way of recognizing human affective states entails the exploration of how physiological signals vary under different emotional experiences. In particular, this paper explores the correlation between autonomically-mediated changes in multimodal body signals and discrete emotional states. In order to fully exploit the information in each modality, we have provided an innovative classification approach for three specific physiological signals including Electromyogram (EMG), Blood Volume Pressure (BVP) and Galvanic Skin Response (GSR). These signals are analyzed as inputs to an emotion recognition paradigm based on fusion of a series of weak learners. Our proposed classification approach showed 88.1% recognition accuracy, which outperformed the conventional Support Vector Machine (SVM) classifier with 17% accuracy improvement. Furthermore, in order to avoid information redundancy and the resultant over-fitting, a feature reduction method is proposed based on a correlation analysis to optimize the number of features required for training and validating each weak learner. Results showed that despite the feature space dimensionality reduction from 27 to 18 features, our methodology preserved the recognition accuracy of about 85.0%. This reduction in complexity will get us one step closer towards embedding this human emotion encoder in the wireless and wearable HCI platforms.

Human-Computer Interaction,Machine Learning
An Emotion Recognition System Based on Physiological Signals Obtained by Wearable Sensors

Cheng He,Yun-jin Yao,Xue-song Ye

DOI: https://doi.org/10.1007/978-981-10-2404-7_2

2017-01-01

Abstract:Automatic emotion recognition is a major topic in the area of human--robot interaction. This paper presents an emotion recognition system based on physiological signals. Emotion induction experiments which induced joy, sadness, anger, and pleasure were conducted on 11 subjects. The subjects’ electrocardiogram (ECG) and respiration (RSP) signals were recorded simultaneously by a physiological monitoring device based on wearable sensors. Compared to the non-wearable physiological monitoring devices often used in other emotion recognition systems, the wearable physiological monitoring device does not restrict the subjects’ movement. From the acquired physiological signals, one hundred and forty-five signal features were extracted. A feature selection method based on genetic algorithm was developed to minimize errors resulting from useless signal features as well as reduce computation complexity. To recognize emotions from the selected physiological signal features, a support vector machine (SVM) method was applied, which achieved a recognition accuracy of 81.82, 63.64, 54.55, and 30.00 % for joy, sadness, anger, and pleasure, respectively. The results showed that it is feasible to recognize emotions from physiological signals.
WEARS: Wearable Emotion AI with Real-time Sensor data

Dhruv Limbani,Daketi Yatin,Nitish Chaturvedi,Vaishnavi Moorthy,Pushpalatha M,Harichandana BSS,Sumit Kumar

2023-08-22

Abstract:Emotion prediction is the field of study to understand human emotions. Existing methods focus on modalities like text, audio, facial expressions, etc., which could be private to the user. Emotion can be derived from the subject's psychological data as well. Various approaches that employ combinations of physiological sensors for emotion recognition have been proposed. Yet, not all sensors are simple to use and handy for individuals in their daily lives. Thus, we propose a system to predict user emotion using smartwatch sensors. We design a framework to collect ground truth in real-time utilizing a mix of English and regional language-based videos to invoke emotions in participants and collect the data. Further, we modeled the problem as binary classification due to the limited dataset size and experimented with multiple machine-learning models. We also did an ablation study to understand the impact of features including Heart Rate, Accelerometer, and Gyroscope sensor data on mood. From the experimental results, Multi-Layer Perceptron has shown a maximum accuracy of 93.75 percent for pleasant-unpleasant (high/low valence classification) moods.

Signal Processing,Machine Learning
Emotion Recognition from ECG Signals Using Wavelet Scattering and Machine Learning

Axel Sepúlveda,Francisco Castillo,Carlos Palma,Maria Rodriguez-Fernandez

DOI: https://doi.org/10.3390/app11114945

2021-05-27

Applied Sciences

Abstract:Affect detection combined with a system that dynamically responds to a person’s emotional state allows an improved user experience with computers, systems, and environments and has a wide range of applications, including entertainment and health care. Previous studies on this topic have used a variety of machine learning algorithms and inputs such as audial, visual, or physiological signals. Recently, a lot of interest has been focused on the last, as speech or video recording is impractical for some applications. Therefore, there is a need to create Human–Computer Interface Systems capable of recognizing emotional states from noninvasive and nonintrusive physiological signals. Typically, the recognition task is carried out from electroencephalogram (EEG) signals, obtaining good accuracy. However, EEGs are difficult to register without interfering with daily activities, and recent studies have shown that it is possible to use electrocardiogram (ECG) signals for this purpose. This work improves the performance of emotion recognition from ECG signals using wavelet transform for signal analysis. Features of the ECG signal are extracted from the AMIGOS database using a wavelet scattering algorithm that allows obtaining features of the signal at different time scales, which are then used as inputs for different classifiers to evaluate their performance. The results show that the proposed algorithm for extracting features and classifying the signals obtains an accuracy of 88.8% in the valence dimension, 90.2% in arousal, and 95.3% in a two-dimensional classification, which is better than the performance reported in previous studies. This algorithm is expected to be useful for classifying emotions using wearable devices.
CorrNet: Fine-Grained Emotion Recognition for Video Watching Using Wearable Physiological Sensors

Tianyi Zhang,Abdallah El Ali,Chen Wang,Alan Hanjalic,Pablo Cesar

DOI: https://doi.org/10.3390/s21010052

IF: 3.9

2020-12-24

Sensors

Abstract:Recognizing user emotions while they watch short-form videos anytime and anywhere is essential for facilitating video content customization and personalization. However, most works either classify a single emotion per video stimuli, or are restricted to static, desktop environments. To address this, we propose a correlation-based emotion recognition algorithm (CorrNet) to recognize the valence and arousal (V-A) of each instance (fine-grained segment of signals) using only wearable, physiological signals (e.g., electrodermal activity, heart rate). CorrNet takes advantage of features both inside each instance (intra-modality features) and between different instances for the same video stimuli (correlation-based features). We first test our approach on an indoor-desktop affect dataset (CASE), and thereafter on an outdoor-mobile affect dataset (MERCA) which we collected using a smart wristband and wearable eyetracker. Results show that for subject-independent binary classification (high-low), CorrNet yields promising recognition accuracies: 76.37% and 74.03% for V-A on CASE, and 70.29% and 68.15% for V-A on MERCA. Our findings show: (1) instance segment lengths between 1–4 s result in highest recognition accuracies (2) accuracies between laboratory-grade and wearable sensors are comparable, even under low sampling rates (≤64 Hz) (3) large amounts of neutral V-A labels, an artifact of continuous affect annotation, result in varied recognition performance.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Emotion and Stress Recognition Utilizing Galvanic Skin Response and Wearable Technology: A Real-time Approach for Mental Health Care

H. Homayoun,Ruijie Fang,S. Rafatirad,Ruoyu Zhang,Elahe Hosseini

DOI: https://doi.org/10.1109/BIBM58861.2023.10386049

2023-12-05

Abstract:In modern society, people are exposed to various stressors and negative emotions daily and they may cause mental and physical diseases such as depression, anxiety, high blood pressure, heart attacks, and stroke. Therefore, this paper delves into the potential of modern wearable technologies as a tool for real-time health monitoring. The advent of ubiquitous sensing has ushered in an era where physiological and behavioral measurements can be continuously recorded in daily life. One significant physiological marker is the Galvanic Skin Response (GSR), which exhibits noteworthy changes under different emotional states. We propose a machine learning-based emotion recognition framework. It includes a preprocessing stage that eliminates noise and extracts 87 features from the GSR data. To account for individual differences in physiological responses, we also introduce a novel normalization procedure per subject. Finally, a subset of dominant and discriminative features enhances the proposed framework’s performance. We conducted experiments on two datasets, the wearable stress and affect detection dataset (WESAD) for stress detection, and the multimodal MAHNOB-HCI dataset for emotion recognition. The results show that the Leave-One-Out method is capable of detecting stress with 97.03% accuracy. Moreover, the proposed method classifies arousal and valence with an accuracy of 82.20% and 82.57%, respectively.

Psychology,Engineering,Medicine,Computer Science
An Ensemble Learning Method for Emotion Charting Using Multimodal Physiological Signals

Amna Waheed Awan,Syed Muhammad Usman,Shehzad Khalid,Aamir Anwar,Roobaea Alroobaea,Saddam Hussain,Jasem Almotiri,Syed Sajid Ullah,Muhammad Usman Akram

DOI: https://doi.org/10.3390/s22239480

2022-12-04

Abstract:Emotion charting using multimodal signals has gained great demand for stroke-affected patients, for psychiatrists while examining patients, and for neuromarketing applications. Multimodal signals for emotion charting include electrocardiogram (ECG) signals, electroencephalogram (EEG) signals, and galvanic skin response (GSR) signals. EEG, ECG, and GSR are also known as physiological signals, which can be used for identification of human emotions. Due to the unbiased nature of physiological signals, this field has become a great motivation in recent research as physiological signals are generated autonomously from human central nervous system. Researchers have developed multiple methods for the classification of these signals for emotion detection. However, due to the non-linear nature of these signals and the inclusion of noise, while recording, accurate classification of physiological signals is a challenge for emotion charting. Valence and arousal are two important states for emotion detection; therefore, this paper presents a novel ensemble learning method based on deep learning for the classification of four different emotional states including high valence and high arousal (HVHA), low valence and low arousal (LVLA), high valence and low arousal (HVLA) and low valence high arousal (LVHA). In the proposed method, multimodal signals (EEG, ECG, and GSR) are preprocessed using bandpass filtering and independent components analysis (ICA) for noise removal in EEG signals followed by discrete wavelet transform for time domain to frequency domain conversion. Discrete wavelet transform results in spectrograms of the physiological signal and then features are extracted using stacked autoencoders from those spectrograms. A feature vector is obtained from the bottleneck layer of the autoencoder and is fed to three classifiers SVM (support vector machine), RF (random forest), and LSTM (long short-term memory) followed by majority voting as ensemble classification. The proposed system is trained and tested on the AMIGOS dataset with k-fold cross-validation. The proposed system obtained the highest accuracy of 94.5% and shows improved results of the proposed method compared with other state-of-the-art methods.
CNN and LSTM-Based Emotion Charting Using Physiological Signals

Muhammad Najam Dar,Muhammad Usman Akram,Sajid Gul Khawaja,Amit N. Pujari

DOI: https://doi.org/10.3390/s20164551

IF: 3.9

2020-08-14

Sensors

Abstract:Novel trends in affective computing are based on reliable sources of physiological signals such as Electroencephalogram (EEG), Electrocardiogram (ECG), and Galvanic Skin Response (GSR). The use of these signals provides challenges of performance improvement within a broader set of emotion classes in a less constrained real-world environment. To overcome these challenges, we propose a computational framework of 2D Convolutional Neural Network (CNN) architecture for the arrangement of 14 channels of EEG, and a combination of Long Short-Term Memory (LSTM) and 1D-CNN architecture for ECG and GSR. Our approach is subject-independent and incorporates two publicly available datasets of DREAMER and AMIGOS with low-cost, wearable sensors to extract physiological signals suitable for real-world environments. The results outperform state-of-the-art approaches for classification into four classes, namely High Valence—High Arousal, High Valence—Low Arousal, Low Valence—High Arousal, and Low Valence—Low Arousal. Emotion elicitation average accuracy of 98.73% is achieved with ECG right-channel modality, 76.65% with EEG modality, and 63.67% with GSR modality for AMIGOS. The overall highest accuracy of 99.0% for the AMIGOS dataset and 90.8% for the DREAMER dataset is achieved with multi-modal fusion. A strong correlation between spectral- and hidden-layer feature analysis with classification performance suggests the efficacy of the proposed method for significant feature extraction and higher emotion elicitation performance to a broader context for less constrained environments.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Automated Affective Computing Based on Bio-Signals Analysis and Deep Learning Approach

Chiara Filippini,Adolfo Di Crosta,Rocco Palumbo,David Perpetuini,Daniela Cardone,Irene Ceccato,Alberto Di Domenico,Arcangelo Merla

DOI: https://doi.org/10.3390/s22051789

IF: 3.9

2022-02-24

Sensors

Abstract:Extensive possibilities of applications have rendered emotion recognition ineluctable and challenging in the fields of computer science as well as in human-machine interaction and affective computing. Fields that, in turn, are increasingly requiring real-time applications or interactions in everyday life scenarios. However, while extremely desirable, an accurate and automated emotion classification approach remains a challenging issue. To this end, this study presents an automated emotion recognition model based on easily accessible physiological signals and deep learning (DL) approaches. As a DL algorithm, a Feedforward Neural Network was employed in this study. The network outcome was further compared with canonical machine learning algorithms such as random forest (RF). The developed DL model relied on the combined use of wearables and contactless technologies, such as thermal infrared imaging. Such a model is able to classify the emotional state into four classes, derived from the linear combination of valence and arousal (referring to the circumplex model of affect’s four-quadrant structure) with an overall accuracy of 70% outperforming the 66% accuracy reached by the RF model. Considering the ecological and agile nature of the technique used the proposed model could lead to innovative applications in the affective computing field.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Behavioral and Physiological Signals-Based Deep Multimodal Approach for Mobile Emotion Recognition

Kangning Yang,Chaofan Wang,Yue Gu,Zhanna Sarsenbayeva,Benjamin Tag,Tilman Dingler,Greg Wadley,Jorge Goncalves

DOI: https://doi.org/10.1109/taffc.2021.3100868

IF: 13.99

2021-01-01

IEEE Transactions on Affective Computing

Abstract:With the rapid development of mobile and wearable devices, it is increasingly possible to access users’ affective data in a more unobtrusive manner. On this basis, researchers have proposed various systems to recognize user’s emotional states. However, most of these studies rely on traditional machine learning techniques and a limited number of signals, leading to systems that either do not generalize well or would frequently lack sufficient information for emotion detection in realistic scenarios. In this paper, we propose a novel attention-based LSTM system that uses a combination of sensors from a smartphone (front camera, microphone, touch panel) and a wristband (photoplethysmography, electrodermal activity, and infrared thermopile sensor) to accurately determine user’s emotional states. We evaluated the proposed system by conducting a user study with 45 participants. Using collected behavioral (facial expression, speech, keystroke) and physiological (blood volume, electrodermal activity, skin temperature) affective responses induced by visual stimuli, our system was able to achieve an average accuracy of 89.2 percent for binary positive and negative emotion classification under leave-one-participant-out cross-validation. Furthermore, we investigated the effectiveness of different combinations of data signals to cover different scenarios of signal availability.

computer science, cybernetics, artificial intelligence
Automatic Emotion Recognition Using Temporal Multimodal Deep Learning

Bahareh Nakisa,Mohammad Naim Rastgoo,Andry Rakotonirainy,Frederic Maire,Vinod Chandran

DOI: https://doi.org/10.1109/access.2020.3027026

IF: 3.9

2020-01-01

IEEE Access

Abstract:Emotion recognition using miniaturised wearable physiological sensors has emerged as a revolutionary technology in various applications. However, detecting emotions using the fusion of multiple physiological signals remains a complex and challenging task. When fusing physiological signals, it is essential to consider the ability of different fusion approaches to capture the emotional information contained within and across modalities. Moreover, since physiological signals consist of time-series data, it becomes imperative to consider their temporal structures in the fusion process. In this study, we propose a temporal multimodal fusion approach with a deep learning model to capture the non-linear emotional correlation within and across electroencephalography (EEG) and blood volume pulse (BVP) signals and to improve the performance of emotion classification. The performance of the proposed model is evaluated using two different fusion approaches – early fusion and late fusion. Specifically, we use a convolutional neural network (ConvNet) long short-term memory (LSTM) model to fuse the EEG and BVP signals to jointly learn and explore the highly correlated representation of emotions across modalities, after learning each modality with a single deep network. The performance of the temporal multimodal deep learning model is validated on our dataset collected from smart wearable sensors and is also compared with results of recent studies. The experimental results show that the temporal multimodal deep learning models, based on early and late fusion approaches, successfully classified human emotions into one of four quadrants of dimensional emotions with an accuracy of 71.61% and 70.17%, respectively.

computer science, information systems,telecommunications,engineering, electrical & electronic
Online Learning for Wearable EEG-Based Emotion Classification

Sidratul Moontaha,Franziska Elisabeth Friederike Schumann,Bert Arnrich

DOI: https://doi.org/10.3390/s23052387

IF: 3.9

2023-02-22

Sensors

Abstract:Giving emotional intelligence to machines can facilitate the early detection and prediction of mental diseases and symptoms. Electroencephalography (EEG)-based emotion recognition is widely applied because it measures electrical correlates directly from the brain rather than indirect measurement of other physiological responses initiated by the brain. Therefore, we used non-invasive and portable EEG sensors to develop a real-time emotion classification pipeline. The pipeline trains different binary classifiers for Valence and Arousal dimensions from an incoming EEG data stream achieving a 23.9% (Arousal) and 25.8% (Valence) higher F1-Score on the state-of-art AMIGOS dataset than previous work. Afterward, the pipeline was applied to the curated dataset from 15 participants using two consumer-grade EEG devices while watching 16 short emotional videos in a controlled environment. Mean F1-Scores of 87% (Arousal) and 82% (Valence) were achieved for an immediate label setting. Additionally, the pipeline proved to be fast enough to achieve predictions in real-time in a live scenario with delayed labels while continuously being updated. The significant discrepancy from the readily available labels on the classification scores leads to future work to include more data. Thereafter, the pipeline is ready to be used for real-time applications of emotion classification.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Valence-Arousal Model based Emotion Recognition using EEG, peripheral physiological signals and Facial Expression

Qingyang Zhu,Guanming Lu,Jingjie Yan

DOI: https://doi.org/10.1145/3380688.3380694

2020-01-17

Abstract:Emotion recognition plays a particularly important role in the field of artificial intelligence. However, the emotional recognition of electroencephalogram (EEG) in the past was only a unimodal or a bimodal based on EEG. This paper aims to use deep learning to perform emotional recognition based on the multimodal with valence-arousal dimension of EEG, peripheral physiological signals, and facial expressions. The experiment uses the complete data of 18 experimenters in the Database for Emotion Analysis Using Physiological Signals (DEAP) to classify the EEG, peripheral physiological signals and facial expression video in unimodal and multimodal fusion. The experiment demonstrates that Multimodal fusion's accuracy is excelled that in unimodal and bimodal fusion. The multimodal compensates for the defects of unimodal and bimodal information sources.
Emotion recognition based on multiple physiological signals

Qi Li,Yunqing Liu,Fei Yan,Qiong Zhang,Cong Liu

DOI: https://doi.org/10.1016/j.bspc.2023.104989

IF: 5.1

2023-05-20

Biomedical Signal Processing and Control

Abstract:Physiological signals can more realistically reflect human emotional states. To overcome the limitations imposed in single-modal emotion recognition, emotion recognition of multimodal physiological signals has received increasingly widespread attention. However, the original fusion models usually ignore the different distributions of multiple signals and how to capture complementary features from multimodal information effectively. This paper proposes an effective classification model for multimodal physiological signals to address the above issues based on modeling the heterogeneity and correlation between multimodal signals. First, differential entropy features are extracted from Electroencephalography (EEG) signals and peripheral physiological signals (PPS) such as Electrocardiographic (ECG) signals, Electromyographic (EMG) signals, and other physiological signals. Then, according to the different distributions and frequency characteristics of the acquired signals, the EEG signal features are made into a three-dimensional feature map and input to the neural network to extract the frequency spatial dimension features. Further temporal features are extracted from the peripheral physiological signals using a long and short-term memory network. Finally, the EEG and peripheral physiological signal features were fused and input to a multimodal long and short-term memory network to extract the association between different modalities and perform classification. The experiments were conducted on the benchmark DEAP dataset, and the results showed that the classification accuracy of the proposed model in this paper was 95.89% and 94.99% in the arousal dimension and the valence dimension, respectively, which were 2.77% and 3.11% higher compared to the unimodal EEG model, respectively. This paper also analyzed the effects of different peripheral physiological signals on emotion recognition.

engineering, biomedical
Emotion Recognition Using Electrodermal Activity Signals and Multiscale Deep Convolution Neural Network

Nagarajan Ganapathy,Ramakrishnan Swaminathan

Abstract:Automated emotion recognition plays a vital role in problem solving, decision making and social activities of human life. An emotion is a set of reactions and experience to a given conditions, which are modeled as a linear combination of arousal and valence dimensions. Emotional pattern recognition based on physiological signals is a relatively new and fast growing area of research. Several physiological signals such as ECG, EEG, and EMG have been used for emotion recognition. Analysis of Electrodermal Activity (EDA) signals is one of the popular technique for emotion state analysis. In this work, an attempt is made to discriminate arousal-valence dimensions using EDA signals and multiscale one dimensional convolution neural network (MCNN). For this, EDA signals are obtained from publically available online DEAP database. These signals are normalized using channel normalization and subjected to MCNN for event-related robust features and classification. K-fold cross validation is used to investigate the performance of classifier. The result shows that the MCNN are able to discriminate the emotional states in arousal/valence dimensions. The proposed approach obtained an overall classification accuracy of 83.75% and 81.25% for arousal and valence scale, respectively. The network yields better classification performance for arousal scale then valence diemension. This might be due to the fact that arousal represent the intensity of emotions. The result also show that the proposed approach is better than the conventional hand-crafted feature based approach. Thus, it appears that the proposed approach can be used to differentiate autonomic and clinical conditions.

Universal serologic reaction with lipid antigen; in tuberculosis.

Arousal and Valence Classification Model Based on Long Short-Term Memory and DEAP Data for Mental Healthcare Management

An Efficient Machine Learning-Based Emotional Valence Recognition Approach Towards Wearable EEG

A Comparative Study of Machine Learning Techniques for Emotion Recognition from Peripheral Physiological Signals

End-to-End Emotion Recognition using Peripheral Physiological Signals

Temporal Convolutional Network-Enhanced Real-Time Implicit Emotion Recognition with an Innovative Wearable fNIRS-EEG Dual-Modal System

Decoding Emotional Experience through Physiological Signal Processing

An Emotion Recognition System Based on Physiological Signals Obtained by Wearable Sensors

WEARS: Wearable Emotion AI with Real-time Sensor data

Emotion Recognition from ECG Signals Using Wavelet Scattering and Machine Learning

CorrNet: Fine-Grained Emotion Recognition for Video Watching Using Wearable Physiological Sensors

Emotion and Stress Recognition Utilizing Galvanic Skin Response and Wearable Technology: A Real-time Approach for Mental Health Care

An Ensemble Learning Method for Emotion Charting Using Multimodal Physiological Signals

CNN and LSTM-Based Emotion Charting Using Physiological Signals

Automated Affective Computing Based on Bio-Signals Analysis and Deep Learning Approach

Behavioral and Physiological Signals-Based Deep Multimodal Approach for Mobile Emotion Recognition

Automatic Emotion Recognition Using Temporal Multimodal Deep Learning

Online Learning for Wearable EEG-Based Emotion Classification

Valence-Arousal Model based Emotion Recognition using EEG, peripheral physiological signals and Facial Expression

Emotion recognition based on multiple physiological signals

Emotion Recognition Using Electrodermal Activity Signals and Multiscale Deep Convolution Neural Network