Abstract:Developing real-time automatic intuitive devices based on electroencephalography (EEG) to improve hazard recognition performance (HRP) is essential in construction engineering management. However, existing studies generally developed devices based on unimodal data, failing to distinguish the role played by brain activities in different visual areas and ignoring the possible existence of other psychological mechanisms. Therefore, this study aims to use multimodal data based on brain-eye synergy to predict HRP and reveal the cognitive mechanism of hazard recognition to develop intuitive hazard recognition devices. Accordingly, we designed experiments to investigate this problem: (1) 53 construction workers completed a 120-trial hazard recognition task in a laboratory environment (N=6,360), (2) an EEG recorder collected the participants' EEG activities and an eye-tracking device recorded the data of eye movement, and (3) the power spectral density (PSD) of EEG was calculated and then imported into Matlab2018a in combination with EEGLAB for analysis. The results showed that the prediction results of hazard recognition have a time-variant effect. For fall-related hazards, EEG signals within 0-320 ms after the onset were significant in predicting HRP; for electric-related and fire-related hazards, EEG signals within 560-640 ms after the starting point were the most valuable. Also, compared with fall-related hazards, it took a significantly longer time to recognize electric-related and fire-related hazards, which consumed more attentional resources. Finally, the frontal lobe requires more attentional resources to finish cognitive activities than the occipital lobe to finish visual information processing activities, and the occipital lobe needs to process visual information first, and then transmit it to the frontal lobe for analysis and judgment. For the body of knowledge, this study, finding the time-variant effect of HRP, improves the theory of construction hazard recognition from an eye-brain synergy perspective and provides guidance for developing intuitive hazard recognition devices and improving management performance. This study combined eye-movement data and EEG data to predict the performance of hazard recognition using multimodal data. Not only did this study reveal the mechanism of hazard recognition, but it also proposed suggestions for project management and device development. Managers are suggested to implement refined management and regularly monitor the physiological signals of employees. Some construction companies started to use smart devices to measure workers' arousal, emotional valence, emotional regulation ability, fatigue, workload, or safety attention level before work every day. And it is recommended to provide safety reminders to employees based on whether there are any abnormalities in their physiological indicators. In addition to project management, the results of this study can serve the development or optimization of hazard recognition equipment. Developing devices cannot only consider EEG information, but also combine EEG information with eye-movement information. Meanwhile, developers can also consider different types of hazards and different brain regions to improve device performance.

Paving the Way for Future EEG Studies in Construction: Dependent Component Analysis for Automatic Ocular Artifact Removal from Brainwave Signals

Eye Blink Artifact Detection Based on Multi-Dimensional EEG Feature Fusion and Optimization.

Eye Blink Artifact Detection with Novel Optimized Multi-Dimensional Electroencephalogram Features

Monitoring workers' attention and vigilance in construction activities through a wireless and wearable electroencephalography system

Construction Site Hazard Identification and Worker Adverse Reaction Monitoring Using Electroencephalograms: A Review

Understanding construction workers’ cognitive processes under risky scenarios through electroencephalography

Toward an Intuitive Device for Construction Hazard Recognition Management: Eye Fixation–Related Potentials in Reinvestigation of Hazard Recognition Performance Prediction

Developing Construction Workers' Mental Vigilance Indicators Through Wavelet Packet Decomposition on EEG Signals

Study on removal of eye movement and blink artifacts from EEG using PCA and JADE

Measuring and Computing Cognitive Statuses of Construction Workers Based on Electroencephalogram: A Critical Review

Assessing Working Vulnerability of Construction Labor through EEG signal processing

EEG-Based Performance-Driven Adaptive Automated Hazard Alerting System in Security Surveillance Support

Revealing the "invisible Gorilla" in Construction: Estimating Construction Safety Through Mental Workload Assessment

Monitoring Distraction of Construction Workers Caused by Noise Using a Wearable Electroencephalography (EEG) Device

Detecting and Measuring Construction Workers' Vigilance Through Hybrid Kinematic-Eeg Signals

Artifacts Removing Techniques on EEG Waveforms

State-of-the-art EEG artifact removal evaluation

A Novel Method Based on Combination of Independent Component Analysis and Ensemble Empirical Mode Decomposition for Removing Electrooculogram Artifacts From Multichannel Electroencephalogram Signals

[Research on automatic removal of ocular artifacts from single channel electroencephalogram signals based on wavelet transform and ensemble empirical mode decomposition]

Identification and Removal of Physiological Artifacts From Electroencephalogram Signals: A Review

Intelligent Method for Real-Time Portable EEG Artifact Annotation in Semiconstrained Environment Based on Computer Vision.