Abstract:With the development of signal processing and deep learning, vision-based methods for loosening detection have progressed in recent years. However, existing visual inspection methods are mainly used to measure the loosening angle, which requires prior knowledge of the initial tightening states, and is only applicable to hexagon bolts and nuts with clear edges. This paper proposes a method to quantitatively calculate the length of the exposed bolt for detecting loosening using vision-based deep learning and geometric imaging theory. The newly proposed method consists of three modules: the region of interest (RoI) location, keypoint detection, and length calculation modules. The RoI location module applies a faster regional convolutional neural network (Faster-RCNN)-based deep learning algorithm to locate the exposed bolt, whereas the keypoint detection module employs a cascaded pyramid network (CPN)-based deep learning algorithm to detect five keypoints on the exposed bolt. An accurate expression is derived from the geometric imaging theory, and the coordinates of the five keypoints and camera parameters are imported into the expression to calculate the length of the exposed bolt. In the experiment, the resolution of input image was 640 × 640 pixels, and the test speed of the proposed method was approximately 200 milliseconds per image. Experimental results showed an average measurement error of 0.61 mm using our method, outperforming other measurement methods and the state-of-the-art networks of human pose estimation. Moreover, the robustness of the proposed method was validated by varying the shooting conditions, bolt sizes and colours. Finally, some factors influencing detection and measurement accuracies are discussed, and prospects for further research are proposed.

Bolt loosening angle measurement along full range of screw exposure length based on 3D point cloud

Bolt loosening angle detection through arrangement shape-independent perspective transformation and corner point extraction based on semantic segmentation results

3D vision-based bolt loosening assessment using photogrammetry, deep neural networks, and 3D point-cloud processing

Quantitative loosening detection of threaded fasteners using vision-based deep learning and geometric imaging theory

Bolt 3D Point Cloud Segmentation and Measurement Based on DBSCAN Clustering

Quantitative Analysis of Bolt Loosening Angle Based on Deep Learning

Autonomous 3D vision‐based bolt loosening assessment using micro aerial vehicles

A Novel Method for Analyzing the Influence of Relative Attitude of Laser Profile Scanner on 3D Measurement System

Computer Vision-Based Autonomous Method for Quantitative Detection of Loose Bolts in Bolted Connections of Steel Structures

A novel anti-loosening bolt looseness diagnosis of bolt connections using a vision-based technique

Early Bolt Loosening Detection Method Based on Digital Image Correlation

Image-Based Bolt-Loosening Detection Using a Checkerboard Perspective Correction Method

Vision-based Detection of Bolt Tension Considering Non-Rotatory Loosening Via a New Calculation Method of Bolt Flexibility Coefficient

Bolt loosening assessment using ensemble vision models for automatic localization and feature extraction with target‐free perspective adaptation

Image Registration-Based Bolt Loosening Detection of Steel Joints

Automatic Lockbolt Detection Method for Riveting Quality Evaluation based on Specific Geometric Feature Descriptors of 3D Point Cloud

Bolt loosening angle detection technology using deep learning

Vision-Based Detection of Bolt Loosening Using YOLOv5

Vision-Based Real-Time Bolt Loosening Detection by Identifying Anti-Loosening Lines

Detection of the Bolt Loosening Angle through Semantic Key Point Extraction Detection by Using an Hourglass Network

Image-based bolt self-localization and bolt-loosening detection using deep learning and an improved homography-based prospective rectification method