Vision-based size distribution analysis of rock fragments using multi-modal deep learning and interactive annotation

Yudi Tang,Yulin Wang,Guangyao Si
DOI: https://doi.org/10.1016/j.autcon.2024.105276
IF: 10.3
2024-01-19
Automation in Construction
Abstract:Real-time size distribution analysis is of great significance to rock fragments in practical engineering. Traditional methods have struggled to strike a balance between analysis speed and precision, prompting the recent adoption of deep learning. However, prevalent approaches for estimating rock fragment sizes from RGB images (single-modality) suffer from two defects: (a)time-consuming and labour-intensive dataset annotation, (b) poor transferability between cases. To solve the above problems, a comprehensive multi-modal framework for size distribution prediction of rock fragments (SDPRF) is proposed in this paper. This framework comprises three essential components: a multi-modal image dataset generation method, a multi-modal rock surface net (Mrsnet) for fragment edge detection and a 2-step breakpoints connection algorithm. The test results indicate:(1) The generation method of SDPRF dataset greatly reduces the time required for dataset annotation, (2) Mrsnet shows better generalization ability for other cases outside the training set than traditional single-modal learning.
construction & building technology,engineering, civil
What problem does this paper attempt to address?