Automatic construction site hazard identification integrating construction scene graphs with BERT based domain knowledge

Lite Zhang,Junjie Wang,Yanbo Wang,Hai Sun,Xuebing Zhao
DOI: https://doi.org/10.1016/j.autcon.2022.104535
IF: 10.3
2022-08-29
Automation in Construction
Abstract:Hazards often arise from interactions of different parties and can cost great loss. Proactively identification can prevent it from happening. Computer vision currently can identify the entities and attributes inside the construction scenes but fail to generate interaction-level scene descriptions and integrate them with domain knowledge for hazards inference. This paper proposed an automatic hazard inference method using construction scene graphs and C-BERT network. First, computer vision was utilized to build the construction scene graphs with interaction-level scene descriptions including entities, attributes, and their interactions. Second, C-BERT network was designed to infer hazards by integrating scene graphs with domain knowledge like construction regulations. 5 different working scenes were used to demonstrate the validity of the proposed approach and reached 97.82% of hazard identification accuracy. It provided an efficient method for combining visual information and domain knowledge for automated safety monitoring, and path for the industry's massive multimodal information fusion.
construction & building technology,engineering, civil
What problem does this paper attempt to address?