Abstract:This paper focuses on the design of Convolution Neural Networks to visually guide an autonomous Unmanned Aerial Vehicle required to inspect power towers. The network is required to precisely segment images taken by a camera mounted on a UAV in order to allow a motion module to generate collision-free and inspection-relevant manoeuvres of the UAV along different types of towers. The images segmentation process is particularly challenging not only because of the different structures of the towers but also because of the enormous variability of the background, which can vary from the uniform blue of the sky to the multi-colour complexity of a rural, forest, or urban area. To be able to train networks that are robust enough to deal with the task variability, without incurring into a labour-intensive and costly annotation process of physical-world images, we have carried out a comparative study in which we evaluate the performances of networks trained either with synthetic images (i.e., the synthetic dataset), physical-world images (i.e., the physical-world dataset), or a combination of these two types of images (i.e., the hybrid dataset). The network used is an attention-based U-NET. The synthetic images are created using photogrammetry, to accurately model power towers, and simulated environments modelling a UAV during inspection of different power towers in different settings. Our findings reveal that the network trained on the hybrid dataset outperforms the networks trained with the synthetic and the physical-world image datasets. Most notably, the networks trained with the hybrid dataset demonstrates a superior performance on multiples evaluation metrics related to the image-segmentation task. This suggests that, the combination of synthetic and physical-world images represents the best trade-off to minimise the costs related to capturing and annotating physical-world images, and to maximise the task performances. Moreover, the results of our study demonstrate the potential of photogrammetry in creating effective training datasets to design networks to automate the precise movement of visually-guided UAVs.

UAV Imagery-based Automatic Classification of Ground Surface Types for Earthworks

Video surveillance-based multi-task learning with swin transformer for earthwork activity classification

Surface sediment classification using a deep learning model and unmanned aerial vehicle data of tidal flats

Ground material classification for UAV-based photogrammetric 3D data A 2D-3D Hybrid Approach

Automatic classification of trees using a UAV onboard camera and deep learning

Survey of Built Environment in the Era of UAV: from Aerial Photogrammetry to Point Cloud Classification

Automatic identification of building structure types using unmanned aerial vehicle oblique images and deep learning considering facade prior knowledge

Investigation of Unsafe Construction Site Conditions Using Deep Learning Algorithms Using Unmanned Aerial Vehicles

Surface Defect-Extended BIM Generation Leveraging UAV Images and Deep Learning

IUNet-IF: identification of construction waste using unmanned aerial vehicle remote sensing and multi-layer deep learning methods

Robust Landslide Recognition Using UAV Datasets: A Case Study in Baihetan Reservoir

Feature extraction and analysis of landscape imaging using drones and machine vision

On the design of deep learning-based control algorithms for visually guided UAVs engaged in power tower inspection tasks

Real-Time Detection of Ground Objects Based on Unmanned Aerial Vehicle Remote Sensing with Deep Learning: Application in Excavator Detection for Pipeline Safety

UAV Remote Sensing for Campus Monitoring: A Comparative Evaluation of Nearest Neighbor and Rule-Based Classification

Robust Autonomous Landing of UAVs in Non-Cooperative Environments Based on Comprehensive Terrain Understanding

Vision-Based On-Site Construction Waste Localization Using Unmanned Aerial Vehicle

Automatic extraction and 3D modeling of real road scenes using UAV imagery and deep learning semantic segmentation

Open-Pit Granite Mining Area Extraction Using UAV Aerial Images and the Novel GIPNet

Computer Vision Based Inspection on Post-Earthquake With UAV Synthetic Dataset

Vegetation Classification of UAV Multispectral Remote Sensing Images Based on Deep Learning