Brain informed transfer learning for categorizing construction hazards

Xiaoshan Zhou,Pin-Chao Liao

DOI: https://doi.org/10.1111/mice.13078

2022-11-18

Abstract:A transfer learning paradigm is proposed for "knowledge" transfer between the human brain and convolutional neural network (CNN) for a construction hazard categorization task. Participants' brain activities are recorded using electroencephalogram (EEG) measurements when viewing the same images (target dataset) as the CNN. The CNN is pretrained on the EEG data and then fine-tuned on the construction scene images. The results reveal that the EEG-pretrained CNN achieves a 9 % higher accuracy compared with a network with same architecture but randomly initialized parameters on a three-class classification task. Brain activity from the left frontal cortex exhibits the highest performance gains, thus indicating high-level cognitive processing during hazard recognition. This work is a step toward improving machine learning algorithms by learning from human-brain signals recorded via a commercially available brain-computer interface. More generalized visual recognition systems can be effectively developed based on this approach of "keep human in the loop".

Neurons and Cognition,Machine Learning,Neural and Evolutionary Computing

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to improve the performance of machine - learning algorithms in identifying potential safety hazards on construction sites by combining human brain activity signals. Specifically, the author proposes a transfer - learning method, which uses electroencephalogram (EEG) data to pre - train a convolutional neural network (CNN), and then fine - tunes it on construction - scene images to achieve the classification task of three potential safety hazards on construction sites (i.e., unenclosed electrical compartments, lack of edge protection, and unstable temporary structures). This method aims to enable machine - learning algorithms to learn more effectively from human brain signals in a "keep human in the loop" manner, thereby improving their performance in small - sample classification tasks. The main contribution of the paper lies in exploring how to use the internal knowledge of the human brain to guide machine learning, especially for the specific task of identifying potential safety hazards on construction sites. Through experiments, the author found that the CNN pre - trained with EEG has a 9% higher accuracy rate in the three - class classification task than the network with randomly initialized parameters. In addition, the study also revealed the high - performance gain in the left prefrontal cortex during the danger - recognition process, indicating a high - level cognitive processing during the danger - recognition process. This provides a new direction for developing more general visual - recognition systems.

Brain informed transfer learning for categorizing construction hazards

Transfer learning-based EEG analysis of visual attention and working memory on motor cortex for BCI

Brain‐regulated Learning for Classifying On‐site Hazards with Small Datasets

Exploring construction workers’ brain connectivity during hazard recognition: a cognitive psychology perspective

Brain–computer interface‐based target recognition system using transfer learning: A deep learning approach

A Transfer Learning-Based CNN Deep Learning Model for Unfavorable Driving State Recognition

Deep Learning and Transfer Learning for Brain Tumor Detection and Classification

DEEP TRANSFER LEARNING FOR EEG-BASED BRAIN COMPUTER INTERFACE

Decoding human brain activity with deep learning

Multimodal Classification with Deep Convolutional-Recurrent Neural Networks for Electroencephalography

Adaptive transfer learning for EEG motor imagery classification with deep Convolutional Neural Network

Transferring a deep learning model from healthy subjects to stroke patients in a motor imagery brain–computer interface

Know Your Mind: Adaptive Cognitive Activity Recognition with Reinforced CNN

Hazard Differentiation Embedded in the Brain: A Near-Infrared Spectroscopy-Based Study

Brain-mediated Transfer Learning of Convolutional Neural Networks

Convolutional Neural Networks with A Topographic Representation Module for EEG-Based Brain-Computer Interfaces

Deep Learning Human Mind for Automated Visual Classification

EEG-based performance-driven adaptive automated hazard alerting system in security surveillance support

A 1D CNN for high accuracy classification and transfer learning in motor imagery EEG-based brain-computer interface

Weighing votes in human-machine collaboration for hazard recognition: Inferring hazard perceptual threshold and decision confidence from electroencephalogram wavelets

Basic Level Categorization Facilitates Visual Object Recognition