Topical application of DEET for schistosomiasis.

K. Ramaswamy,Yi-Xun He,B. Salafsky,T. Shibuya

DOI: https://doi.org/10.1016/J.PT.2003.10.001

2003-12-01

Trends in Parasitology

Abstract:

What problem does this paper attempt to address?

Multimodal Sensing and Computational Intelligence for Situation Awareness Classification in Autonomous Driving

Jing Yang,Nade Liang,Brandon J. Pitts,Kwaku O. Prakah-Asante,Reates Curry,Mike Blommer,Radhakrishnan Swaminathan,Denny Yu

DOI: https://doi.org/10.1109/thms.2023.3234429

2023-03-18

IEEE Transactions on Human-Machine Systems

Abstract:Maintaining situation awareness (SA) is essential for drivers to deal with the situations that Society of Automotive Engineers (SAE) Level 3 automated vehicle systems are not designed to handle. Although advanced physiological sensors can enable continuous SA assessments, previous single-modality approaches may not be sufficient to capture SA. To address this limitation, the current study demonstrates a multimodal sensing approach for objective SA monitoring. Physiological sensor data from electroencephalogram and eye-tracking were recorded for 30 participants as they performed three secondary tasks during automated driving scenarios that consisted of a pre-takeover (pre-TOR) request segment and a post-TOR segment. The tasks varied in terms of how visual attention was allocated in the pre-TOR segment. In the post-TOR segment, drivers were expected to gather information from the driving environment in preparation for a vehicle-to-driver transition. Participants' ground-truth SA level was measured using the Situation Awareness Global Assessment Techniques (SAGAT) after the post-TOR segment. A total of 23 physiological features were extracted from the post-TOR segment to train computational intelligence models. Results compared the performance of five different classifiers, the ground-truth labeling strategies, and the features included in the model. Overall, the proposed neural network model outperformed other machine learning models and achieved the best classification accuracy (90.6%). A model with 11 features was optimal. In addition, the multi-physiological sensor-model outperformed the single sensing model by comparing prediction performance. Our results suggest that multimodal sensing model can objectively predict SA. The results of this study provide new insight into how physiological features contribute to the SA assessment.

computer science, cybernetics, artificial intelligence
Driving Drowsiness Detection With Eeg Using A Modified Hierarchical Extreme Learning Machine Algorithm With Particle Swarm Optimization: A Pilot Study

Yuliang Ma,Songjie Zhang,Donglian Qi,Zhizeng Luo,Rihui Li,Thomas Potter,Yingchun Zhang

DOI: https://doi.org/10.3390/electronics9050775

IF: 2.9

2020-01-01

Electronics

Abstract:Driving fatigue accounts for a large number of traffic accidents in modern life nowadays. It is therefore of great importance to reduce this risky factor by detecting the driver's drowsiness condition. This study aimed to detect drivers' drowsiness using an advanced electroencephalography (EEG)-based classification technique. We first collected EEG data from six healthy adults under two different awareness conditions (wakefulness and drowsiness) in a virtual driving experiment. Five different machine learning techniques, including the K-nearest neighbor (KNN), support vector machine (SVM), extreme learning machine (ELM), hierarchical extreme learning machine (H-ELM), and the proposed modified hierarchical extreme learning machine algorithm with particle swarm optimization (PSO-H-ELM), were applied to classify the subject's drowsiness based on the power spectral density (PSD) feature extracted from the EEG data. The mean accuracies of the five classifiers were 79.31%, 79.31%, 74.08%, 81.67%, and 83.12%, respectively, demonstrating the superior performance of our new PSO-H-ELM algorithm in detecting drivers' drowsiness, compared to the other techniques.
A Multimodal Data-Driven Approach for Driving Risk Assessment

Congcong Bai,Sheng Jin,Jun Jing,Chengcheng Yang,Wenbin Yao,Donglei Rong,Jeremie Adje Alagbe

DOI: https://doi.org/10.1016/j.tre.2024.103678

IF: 10.047

2024-01-01

Transportation Research Part E Logistics and Transportation Review

Abstract:Real-time assessment and short-term warning of driving risks are critical for AI-assisted vehicles to significantly improve the safety and reliability of mobility. However, existing methods do not comprehensively consider these factors, making it difficult to achieve more accurate risk assessments. Aiming at this problem, this paper proposes a new driving risk assessment framework by integrating multimodal data. First, based on naturalistic driving experiments, we collected multimodal data encompassing human-vehicle-road factors. Then, using the Latent Dirichlet Allocation (LDA) model, we identified three risk levels based on driving behavior features: normal driving, longitudinal risky driving, and lateral risky driving. To better understand the spatiotemporal importance of multiple factors, a spatiotemporal dual-channel neural network based on a multi-layer attention mechanism (MLA-DCNN) is developed. This model has a spatiotemporal dual-channel structure, which can integrate "low-level" historical sequences and "high-level" extract statistical features of multiple features. In addition, it adopts three layers of attention mechanism, respectively used to capture the differences of features in temporal, spatial, and extracted-level dimensions. Results reveal that the LDA model is more effective than traditional clustering methods in uncovering latent patterns of driving risk. The proposed model achieved an impressive accuracy of 91.04%, demonstrating higher risk assessment capabilities than the other alternative models. In addition, the multilayer attention enhances the interpretability of the model and is able to capture the spatiotemporal importance of different factors across various road environments. This method can be applied to connected and automated vehicles (CAVs) using multimodal natural driving data collected by in-vehicle sensors. It enhances the risk warning capabilities of driving assistance systems, and the multidimensional importance analysis also supports decision-making for traffic management authorities.
Multimodal Estimation of Change Points of Physiological Arousal in Drivers

Kleanthis Avramidis,Tiantian Feng,Digbalay Bose,Shrikanth Narayanan

DOI: https://doi.org/10.48550/arXiv.2210.15826

IF: 4.729

2022-10-28

Signal Processing

Abstract:Detecting unsafe driving states, such as stress, drowsiness, and fatigue, is an important component of ensuring driving safety and an essential prerequisite for automatic intervention systems in vehicles. These concerning conditions are primarily connected to the driver's low or high arousal levels. In this study, we describe a framework for processing multimodal physiological time-series from wearable sensors during driving and locating points of prominent change in drivers' physiological arousal state. These points of change could potentially indicate events that require just-in-time intervention. We apply time-series segmentation on heart rate and breathing rate measurements and quantify their robustness in capturing change points in electrodermal activity, treated as a reference index for arousal, as well as on self-reported stress ratings, using three public datasets. Our experiments demonstrate that physiological measures are veritable indicators of change points of arousal and perform robustly across an extensive ablation study.
Attention-Guided Multi-scale Convolutional Neural Network for Driving Fatigue Detection

Jinbu Tang,Wei Zhou,Weilong Zheng,Zheng Zeng,Jiayi Li,Ruizhi Su,Tuheti Adili,Wei Chen,Chen,Jingchun Luo

DOI: https://doi.org/10.1109/jsen.2024.3406047

IF: 4.3

2024-01-01

IEEE Sensors Journal

Abstract:Driver fatigue poses a significant safety risk in road transport, prompting heightened public awareness. Electroencephalography (EEG), a key physiological signal for fatigue characterization, has garnered substantial interest and served as de facto gold standard. Existing EEG-based fatigue detection methods often utilize limited forehead lobe channels or the entire brain’s EEG signals, potentially leading to information loss or redundancy. This oversight neglects the crucial aspect of optimizing channels. Moreover, many studies prioritize augmenting model performance through additional sub-networks, neglecting refined features, deepening network and creating overfitting. To address these gaps, this paper introduces an attention-guided multi-scale convolutional neural network for driving fatigue detection, comprising a multi-scale convolutional neural network (MSCNN) and a channel attention mechanism (CAM) module. The MSCNN optimizes network depth, capturing richer temporal and frequency features. The CAM module supplements the MSCNN by emphasizing valuable information, exploring channel contributions, and assessing the impact of feature refinement on model performance. The proposed method demonstrates promising results on the public SEED-VIG dataset with an average root mean square error (RMSE) of 0.051 and an average correlation coefficient (COR) of 0.966 across 23 subjects, outperforming alternative approaches in driving fatigue detection. These findings show that our proposed method perform more effectively for driving fatigue detection based on EEG.
[Interconvertibility of molecular forms of 3'5' cyclic AMP phosphodiesterase of human platelets].

A. Pichard,J. Kaplan

Abstract:Three forms of cyclic AMP phosphodiesterase from human platelets are resolved upon sucrose gradient centrifugation : 2.5S (monomer), 4.8S (dimer) and 7S (tetramer). They are interconvertible and form an association-dissociation equilibrium depending on the concentration of enzyme. The dissociated form has a high Km for cyclic AMP (Km : 3-5.10(-4) M) whereas the associated form has a low Km (Km : 3-5.10(-5) M).
Driver inattention monitoring system based on multimodal fusion with visual cues to improve driving safety

Xuanpeng Li,Emmanuel Seignez

DOI: https://doi.org/10.1177/0142331216670451

IF: 2.146

2016-10-07

Transactions of the Institute of Measurement and Control

Abstract:Driver inattention, either driver drowsiness or distraction, is a major contributor to serious traffic crashes. In general, most research on this topic studies driver drowsiness and distraction separately, and is often conducted in a well-controlled, simulated environment. By considering the reliability and flexibility of real-time driver monitoring systems, it is possible to evaluate driver inattention by the fusion of multiple selected cues in real life scenarios. This paper presents a real-time, visual-cue-based driver monitoring system, which can track both multi-level driver drowsiness and distraction simultaneously. A set of visual cues are adopted via analysis of drivers’ physical behaviour and driving performance. Driver drowsiness is evaluated using a multi-level scale, by applying evidence theory. Additionally, a general framework of extensive hierarchical combinations is used to generate a probabilistic evaluation of driving risk in real time. This driver inattention monitoring system with multimodal fusion has been proven to improve the accuracy of risk evaluation and reduce the rate of false alarms, and acceptance of the system is recommended.

automation & control systems,instruments & instrumentation
Detecting driving stress in physiological signals based on multimodal feature analysis and kernel classifiers

Lan-lan Chen,Yu Zhao,Peng-fei Ye,Jian Zhang,Jun-zhong Zou

DOI: https://doi.org/10.1016/j.eswa.2017.01.040

IF: 8.5

2017-11-01

Expert Systems with Applications

Abstract:Monitoring driving status has great potential in helping us decline the occurrence probability of traffic accidents and the aim of this research is to develop a novel system for driving stress detection based on multimodal feature analysis and kernel-based classifiers. Physiological signals such as electrocardiogram, galvanic skin response and respiration were record from fourteen drives executed in a prescribed route at real drive environments. Features were widely extracted from time, spectral and wavelet multi-domains. In order to search for the optimal feature sets, Sparse Bayesian Learning (SBL) and Principal Component Analysis (PCA) were combined and adopted. Kernel-based classifiers were employed to improve the accuracy of stress detection task. Analysis I used features from 10s intervals of data which were recorded during well-defined rest, highway and city driving conditions to discriminate three levels of diving stress achieving an averaging accuracy over 99% at per-drive level and 89% in cross-drive validation. Analysis II made continuous stress evaluation throughout a complete driving test attaining a high coincidence with the true road situation especially at the switching interval of traffic conditions. Experimental results reveal that different levels of driving stress can be characterized by specific set of physiological measures. These physiological measures could be applied to in-vehicle intelligent systems in various approaches to help the drivers better manage their negative driving status. Our design scheme for driving stress detection could also facilitate the development of similar in-vehicle expert systems, such as driver's emotion management, driver's sleeping onset monitoring, and human-computer interaction (HCI).

computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science
Capsule Attention for Multimodal EEG-EOG Representation Learning With Application to Driver Vigilance Estimation

Guangyi Zhang,Ali Etemad

DOI: https://doi.org/10.1109/tnsre.2021.3089594

IF: 4.9

2021-01-01

IEEE Transactions on Neural Systems and Rehabilitation Engineering

Abstract:Driver vigilance estimation is an important task for transportation safety. Wearable and portable brain-computer interface devices provide a powerful means for real-time monitoring of the vigilance level of drivers to help with avoiding distracted or impaired driving. In this paper, we propose a novel multimodal architecture for in-vehicle vigilance estimation from Electroencephalogram and Electrooculogram. To enable the system to focus on the most salient parts of the learned multimodal representations, we propose an architecture composed of a capsule attention mechanism following a deep Long Short-Term Memory (LSTM) network. Our model learns hierarchical dependencies in the data through the LSTM and capsule feature representation layers. To better explore the discriminative ability of the learned representations, we study the effect of the proposed capsule attention mechanism including the number of dynamic routing iterations as well as other parameters. Experiments show the robustness of our method by outperforming other solutions and baseline techniques, setting a new state-of-the-art. We then provide an analysis on different frequency bands and brain regions to evaluate their suitability for driver vigilance estimation. Lastly, an analysis on the role of capsule attention, multimodality, and robustness to noise is performed, highlighting the advantages of our approach.

engineering, biomedical,rehabilitation
Efficient Mixture-of-Expert for Video-based Driver State and Physiological Multi-task Estimation in Conditional Autonomous Driving

Jiyao Wang,Xiao Yang,Zhenyu Wang,Ximeng Wei,Ange Wang,Dengbo He,Kaishun Wu

2024-10-28

Abstract:Road safety remains a critical challenge worldwide, with approximately 1.35 million fatalities annually attributed to traffic accidents, often due to human errors. As we advance towards higher levels of vehicle automation, challenges still exist, as driving with automation can cognitively over-demand drivers if they engage in non-driving-related tasks (NDRTs), or lead to drowsiness if driving was the sole task. This calls for the urgent need for an effective Driver Monitoring System (DMS) that can evaluate cognitive load and drowsiness in SAE Level-2/3 autonomous driving contexts. In this study, we propose a novel multi-task DMS, termed VDMoE, which leverages RGB video input to monitor driver states non-invasively. By utilizing key facial features to minimize computational load and integrating remote Photoplethysmography (rPPG) for physiological insights, our approach enhances detection accuracy while maintaining efficiency. Additionally, we optimize the Mixture-of-Experts (MoE) framework to accommodate multi-modal inputs and improve performance across different tasks. A novel prior-inclusive regularization method is introduced to align model outputs with statistical priors, thus accelerating convergence and mitigating overfitting risks. We validate our method with the creation of a new dataset (MCDD), which comprises RGB video and physiological indicators from 42 participants, and two public datasets. Our findings demonstrate the effectiveness of VDMoE in monitoring driver states, contributing to safer autonomous driving systems. The code and data will be released.

Computer Vision and Pattern Recognition,Artificial Intelligence
Multimodal Driver Condition Monitoring System Operating in the Far-Infrared Spectrum

Mateusz Knapik,Bogusław Cyganek,Tomasz Balon

DOI: https://doi.org/10.3390/electronics13173502

IF: 2.9

2024-09-04

Electronics

Abstract:Monitoring the psychophysical conditions of drivers is crucial for ensuring road safety. However, achieving real-time monitoring within a vehicle presents significant challenges due to factors such as varying lighting conditions, vehicle vibrations, limited computational resources, data privacy concerns, and the inherent variability in driver behavior. Analyzing driver states using visible spectrum imaging is particularly challenging under low-light conditions, such as at night. Additionally, relying on a single behavioral indicator often fails to provide a comprehensive assessment of the driver's condition. To address these challenges, we propose a system that operates exclusively in the far-infrared spectrum, enabling the detection of critical features such as yawning, head drooping, and head pose estimation regardless of the lighting scenario. It integrates a channel fusion module to assess the driver's state more accurately and is underpinned by our custom-developed and annotated datasets, along with a modified deep neural network designed for facial feature detection in the thermal spectrum. Furthermore, we introduce two fusion modules for synthesizing detection events into a coherent assessment of the driver's state: one based on a simple state machine and another that combines a modality encoder with a large language model. This latter approach allows for the generation of responses to queries beyond the system's explicit training. Experimental evaluations demonstrate the system's high accuracy in detecting and responding to signs of driver fatigue and distraction.

engineering, electrical & electronic,computer science, information systems,physics, applied
Multimodal Driver State Modeling through Unsupervised Learning

Arash Tavakoli,Arsalan Heydarian

DOI: https://doi.org/10.48550/arXiv.2110.01727

2021-10-05

Abstract:Naturalistic driving data (NDD) can help understand drivers' reactions to each driving scenario and provide personalized context to driving behavior. However, NDD requires a high amount of manual labor to label certain driver's state and behavioral patterns. Unsupervised analysis of NDD can be used to automatically detect different patterns from the driver and vehicle data. In this paper, we propose a methodology to understand changes in driver's physiological responses within different driving patterns. Our methodology first decomposes a driving scenario by using a Bayesian Change Point detection model. We then apply the Latent Dirichlet Allocation method on both driver state and behavior data to detect patterns. We present two case studies in which vehicles were equipped to collect exterior, interior, and driver behavioral data. Four patterns of driving behaviors (i.e., harsh brake, normal brake, curved driving, and highway driving), as well as two patterns of driver's heart rate (HR) (i.e., normal vs. abnormal high HR), and gaze entropy (i.e., low versus high), were detected in these two case studies. The findings of these case studies indicated that among our participants, the drivers' HR had a higher fraction of abnormal patterns during harsh brakes, accelerating and curved driving. Additionally, free-flow driving with close to zero accelerations on the highway was accompanied by more fraction of normal HR as well as a lower gaze entropy pattern. With the proposed methodology we can better understand variations in driver's psychophysiological states within different driving scenarios. The findings of this work, has the potential to guide future autonomous vehicles to take actions that are fit to each specific driver.

Human-Computer Interaction
Quality control of leucocyte‐depleted platelet concentrates obtained by buffy‐coat method

T. Vuk,M. Strauss Patko,J. Gulan-Harcet,T. Očić,D. Šarlija,I. Jukić

DOI: https://doi.org/10.1111/tme.12051

2013-10-01

Transfusion Medicine

Abstract:Quality control results on leukoreduced buffy‐coat platelet pools during the 7‐year period (2005–2011) are presented with the aim to assess their overall quality and trends recorded during the study period.
A Multimodal Fusion Fatigue Driving Detection Method Based on Heart Rate and PERCLOS

Guanglong Du,Linlin Zhang,Kang Su,Xueqian Wang,Shaohua Teng,Peter X. Liu

DOI: https://doi.org/10.1109/tits.2022.3176973

IF: 8.5

2022-11-13

IEEE Transactions on Intelligent Transportation Systems

Abstract:Existing visual-based fatigue detection methods usually monitor drivers' fatigue by capturing their facial features, including eyelid movements, yawn frequency and head pose. However, these approaches typically do not take drivers' biological signals into consideration. An accurate model for fatigue detection requires combining both facial behavior and biological data. This paper proposes a novel non-intrusive method for driver multimodal fusion fatigue detection by extracting eyelid features and heart rate signals from the RGB video. The multimodal feature fusion method could significantly increase the accuracy of fatigue detection. Specifically, we established two fatigue detection models based on heart rate and the PERCLOS value respectively with one-dimensional Convolutional Neural Network (1D CNN), where the PERCLOS refers to the percentage of eyelid closure over the pupil. Finally, the outputs of the two models are weighted to achieve the multimodal fusion fatigue detection. Simulation results show that our method yield better performance than traditional methods.

engineering, electrical & electronic,transportation science & technology, civil
EEG Based Driver Inattention Identification via Feature Profiling and Dimensionality Reduction

Omid Dehzangi,Mojtaba Taherisadr

DOI: https://doi.org/10.1007/978-3-030-02819-0_9

IF: 5.711

2018-12-30

Internet of Things

Abstract:More than 90% of the persistently increasing traffic fatalities is related to human choice/error. Monitoring driver attention has a direct effect on decreasing injury/fatality rates. In recent years, there has been much effort to estimate drivers’ state with the goal of improving their driving behavior and preventing vehicle crashes in the first place. Physiological based detection has shown to be the most direct method of measuring driver state among which, electroencephalogram (EEG) is the most comprehensive method. EEGs are recorded from multiple channels that are processed separately. However, contribution of a fairly large number of the channels might be minimal to the target application. The computational load and the redundancy induced by those channels can hurt the identification performance. In this study, we propose an EEG-based systematic methodology for the assessment of driver state of inattention. Our proposed framework includes three major modules: (1) We first characterize each EEG channel rigorously via extraction of various categories of descriptors as features, (2) we then capture the contribution of each channel toward the identification task via channel specific feature dimensionality reduction, (3) we then conduct channel selection in order to find key brain regions of impact. Eight subjects participated in our naturalistic driving study. Our proposed method resulted in the accuracy of 98.99 ± 1.2% inattention identification accuracy. We also discovered that the first and second best channels are consistently selected from frontal and parietal regions for participating subjects.
Assessing the Effects of Various Physiological Signal Modalities on Predicting Different Human Cognitive States

Ayca Aygun,Thuan Nguyen,Matthias Scheutz

DOI: https://doi.org/10.1101/2024.02.29.582708

2024-03-04

Abstract:Robust estimation of systemic human cognitive states is critical for a variety of applications, from simply detecting inefficiencies in task assignments, to the adaptation of artificial agents’ behaviors to improve team performance in mixed-initiative human-machine teams. This study showed that human eye gaze, in particular, the percentage change in pupil size (PCPS), is the most reliable biomarker for assessing three human cognitive states including workload, sense of urgency, and mind wandering compared to electroencephalogram (EEG), functional near-infrared spectroscopy (fNIRS), respiration, and skin conductance. We used comprehensive multi-modal driving dataset to examine the accuracy of signals to assess these cognitive states. We performed comprehensive statistical tests to validate the performance of several physiological signals to determine human cognitive states and demonstrated that PCPS shows noticeably superior performance. We also characterized the link between workload and sense of urgency with eye gaze and observed that consecutive occurrences of higher sense of urgency were prone to increase overall workload. Finally, we trained five machine learning (ML) models and showed that four of them had similar accuracy in cognitive state classification (with one, random forest, showing inferior performance). The results provided evidence that the PCPS is a reliable physiological marker for cognitive state estimation.

Neuroscience
Driving-Induced Neurological Biomarkers in an Advanced Driver-Assistance System

Iqram Hussain,Seo Young,Se-Jin Park

DOI: https://doi.org/10.3390/s21216985

IF: 3.9

2021-10-21

Sensors

Abstract:Physiological signals are immediate and sensitive to neurological changes resulting from the mental workload induced by various driving environments and are considered a quantifying tool for understanding the association between neurological outcomes and driving cognitive workloads. Neurological assessment, outside of a highly-equipped clinical setting, requires an ambulatory electroencephalography (EEG) headset. This study aimed to quantify neurological biomarkers during a resting state and two different scenarios of driving states in a virtual driving environment. We investigated the neurological responses of seventeen healthy male drivers. EEG data were measured in an initial resting state, city-roadways driving state, and expressway driving state using a portable EEG headset in a driving simulator. During the experiment, the participants drove while experiencing cognitive workloads due to various driving environments, such as road traffic conditions, lane changes of surrounding vehicles, the speed limit, etc. The power of the beta and gamma bands decreased, and the power of the delta waves, theta, and frontal theta asymmetry increased in the driving state relative to the resting state. Delta-alpha ratio (DAR) and delta-theta ratio (DTR) showed a strong correlation with a resting state, city-roadways driving state, and expressway driving state. Binary machine-learning (ML) classification models showed a near-perfect accuracy between the resting state and driving state. Moderate classification performances were observed between the resting state, city-roadways state, and expressway state in multi-class classification. An EEG-based neurological state prediction approach may be utilized in an advanced driver-assistance system (ADAS).

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Multimodal Vigilance Estimation Using Deep Learning

Wei Wu,Wei Sun,Q. M. Jonathan Wu,Yimin Yang,Hui Zhang,Wei-Long Zheng,Bao-Liang Lu

DOI: https://doi.org/10.1109/tcyb.2020.3022647

IF: 11.8

2022-01-01

IEEE Transactions on Cybernetics

Abstract:The phenomenon of increasing accidents caused by reduced vigilance does exist. In the future, the high accuracy of vigilance estimation will play a significant role in public transportation safety. We propose a multimodal regression network that consists of multichannel deep autoencoders with subnetwork neurons (MCDAE <sub>sn</sub> ). After we define two thresholds of "0.35" and "0.70" from the percentage of eye closure, the output values are in the continuous range of 0-0.35, 0.36-0.70, and 0.71-1 representing the awake state, the tired state, and the drowsy state, respectively. To verify the efficiency of our strategy, we first applied the proposed approach to a single modality. Then, for the multimodality, since the complementary information between forehead electrooculography and electroencephalography features, we found the performance of the proposed approach using features fusion significantly improved, demonstrating the effectiveness and efficiency of our method.

automation & control systems,computer science, cybernetics, artificial intelligence
Driver stress detection via multimodal fusion using attention-based CNN-LSTM

Luntian Mou,Chao Zhou,Pengfei Zhao,Bahareh Nakisa,Mohammad Naim Rastgoo,Ramesh Jain,Wen Gao

DOI: https://doi.org/10.1016/j.eswa.2021.114693

IF: 8.5

2021-07-01

Expert Systems with Applications

Abstract:<p>Stress has been identified as one of major contributing factors in car crashes due to its negative impact on driving performance. It is in urgent need that the stress levels of drivers can be detected in real time with high accuracy so that intervening or navigating measures can be taken in time to mitigate the situation. Existing driver stress detection models mainly rely on traditional machine learning techniques to fuse multimodal data. However, due to the non-linear correlations among modalities, it is still challenging for traditional multimodal fusion methods to handle the real-time influx of complex multimodal and high dimensional data, and report drivers' stress levels accurately. To solve this issue, a framework of driver stress detection through multimodal fusion using attention based deep learning techniques is proposed in this paper. Specifically, an attention based convolutional neural networks (CNN) and long short-term memory (LSTM) model is proposed to fuse non-invasive data, including eye data, vehicle data, and environmental data. Then, the proposed model can automatically extract features separately from each modality and give different levels of attention to features from different modalities through self-attention mechanism. To verify the validity of the proposed method, extensive experiments have been carried out on our dataset collected using an advanced driving simulator. Experimental results demonstrate that the performance of the proposed method on driver stress detection outperforms the state-of-the-art models with an average accuracy of 95.5%.</p>

computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science
Detecting Driver Cognition Alertness State from Visual Activities in Normal and Emergency Scenarios.

Longxi Luo,Jianping Wu,Weijie Fei,Luzheng Bi,Xinan Fan

DOI: https://doi.org/10.1109/tits.2022.3166251

IF: 8.5

2022-01-01

IEEE Transactions on Intelligent Transportation Systems

Abstract:Current driving behavior studies have limits in obtaining driver’s state information, and recent studies involving driver state focus on driver distraction or inattention. But it is more common that drivers operate in an intermediate subconscious state, where the drivers are neither cognitively fully focused nor distracted. There was little research to address this topic. In this study, the driver cognition alertness state information, which indicates driver subconscious alertness, is detected from eye and iris activities by non-contact computer vision methods. And a novel analysis is conducted and reveals the strong correlation between driver performance and driver cognition alertness state from experiment results. In detail, the driver cognition alertness state is quantified by the proposed metric– iris movement index, which is calculated from iris-eye relative displacements. The developed computer vision method produces high precision results in detecting faces, eyes, and irises in the experiment based on multiple deep learning networks applied in cascade, and enables displacement tracking of eye and iris targets. A filtering method is proposed and removes artifacts due to eye blinks in displacement measurements. The driving performance is estimated by a proposed performance evaluation method in a series of traffic scenarios designed with normal and emergent conditions.

Topical application of DEET for schistosomiasis.

Multimodal Sensing and Computational Intelligence for Situation Awareness Classification in Autonomous Driving

Driving Drowsiness Detection With Eeg Using A Modified Hierarchical Extreme Learning Machine Algorithm With Particle Swarm Optimization: A Pilot Study

A Multimodal Data-Driven Approach for Driving Risk Assessment

Multimodal Estimation of Change Points of Physiological Arousal in Drivers

Attention-Guided Multi-scale Convolutional Neural Network for Driving Fatigue Detection

[Interconvertibility of molecular forms of 3'5' cyclic AMP phosphodiesterase of human platelets].

Driver inattention monitoring system based on multimodal fusion with visual cues to improve driving safety

Detecting driving stress in physiological signals based on multimodal feature analysis and kernel classifiers

Capsule Attention for Multimodal EEG-EOG Representation Learning With Application to Driver Vigilance Estimation

Efficient Mixture-of-Expert for Video-based Driver State and Physiological Multi-task Estimation in Conditional Autonomous Driving

Multimodal Driver Condition Monitoring System Operating in the Far-Infrared Spectrum

Multimodal Driver State Modeling through Unsupervised Learning

Quality control of leucocyte‐depleted platelet concentrates obtained by buffy‐coat method

A Multimodal Fusion Fatigue Driving Detection Method Based on Heart Rate and PERCLOS

EEG Based Driver Inattention Identification via Feature Profiling and Dimensionality Reduction

Assessing the Effects of Various Physiological Signal Modalities on Predicting Different Human Cognitive States

Driving-Induced Neurological Biomarkers in an Advanced Driver-Assistance System

Multimodal Vigilance Estimation Using Deep Learning

Driver stress detection via multimodal fusion using attention-based CNN-LSTM

Detecting Driver Cognition Alertness State from Visual Activities in Normal and Emergency Scenarios.