Abstract:In the process of aircraft assembly, there exist numerous and ubiquitous cable brackets that shall be installed on frames and subsequently need to be manually verified with CAD models. Such a task is usually performed by special operators, hence is time-consuming, labor-intensive, and error-prone. In order to save the inspection time and increase the reliability of results, many researchers attempt to develop intelligent inspection systems using robotic, AR , or AI technologies. However, there is no comprehensive method to achieve enough portability, intelligence, efficiency, and accuracy while providing intuitive task assistance for inspectors in real time. In this paper, a combined AR+AI system is introduced to assist brackets inspection in a more intelligent yet efficient manner. Especially, AR-based Mask R-CNN is proposed by skillfully integrating markerless AR into deep learning-based instance segmentation to generate more accurate and fewer region proposals, and thus alleviates the computation load of the deep learning program. Based on this, brackets segmentation can be performed robustly and efficiently on mobile devices such as smartphones or tablets. By using the proposed system, CAD model checking can be automatically performed between the segmented physical brackets and the corresponding virtual brackets rendered by AR in real time. Furthermore, the inspection results can be directly projected on the corresponding physical brackets for the convenience of maintenance. To verify the feasibility of the proposed method, experiments are carried out on a full-scale mock-up of C919 aircraft main landing gear cabin. The experimental results indicate that the inspection accuracy is up to 97.1%. Finally, the system has been deployed in the real C919 aircraft final-assembly workshop. The preliminary evaluation reveals that the proposed real-time AR-assisted intelligent inspection approach is effective and promising for large-scale industrial applications.

Intelligent Itinerant Inspection Technology of Aircraft Based on Deep Learning and AR

An AR-Assisted Deep Learning-Based Approach for Automatic Inspection of Aviation Connectors.

AR Aircraft Exterior Inspection System Based on Visual SLAM Technology

AR-based deep learning for real-time inspection of cable brackets in aircraft

A Comprehensive Solution for Inspecting Cable Brackets in Aircraft Based on Augmented Reality and Deep Learning

Exploration and Practice of Aircraft Assembly AR Intelligent Pilot Technology

Automating the Visual Inspection of Aircraft

Automating Aircraft Scanning For Inspection With A UAV And Reinforcement Learning Technique

Visual Inspection of the Aircraft Surface Using a Teleoperated Reconfigurable Climbing Robot and Enhanced Deep Learning Technique

Edge-Feature-Based Aircraft Cover Recognition and Pose Estimation for AR-aided Inner Components Inspection

Intelligent Technology for Aircraft Detection and Recognition Through SAR Imagery: Advancements and Prospects

Design of airport road surface inspection system based on machine vision and deep learning

Defects Recognition Algorithm Development from Visual UAV Inspections

Augmented Imagefication: A Data-driven Fault Detection Method for Aircraft Air Data Sensors

Increasing Trust with Augmented Reality in Human-in-the-Loop Intelligent System: A Case Study with Inspection Task.

FC-YOLO: An aircraft skin defect detection algorithm based on multi-scale collaborative feature fusion

Edge-based cover recognition and tracking method for an AR-aided aircraft inspection system

3D Point Cloud Analysis for Detection and Characterization of Defects on Airplane Exterior Surface

Detection and Analysis of Aircraft Composite Material Structures Using UAV

Aviation Rivet Classification and Anomaly Detection Based on Deep Learning

A Deep-Learning-Based Approach for Aircraft Engine Defect Detection