Abstract:Building façades elements detection plays a key point role in façade defects detection and street scene reconstruction tasks for sustainable city development. Although the artificial intelligence technology has made a breakthrough in image segmentation, it is nontrivial to directly apply standard deep learning approaches for building façade element detection. The main reason is that the existing semantic segmentation networks have a bad performance in predicting highly regularized shapes. This research develops a hieratical deep learning framework with a symmetric loss function to automatically detect building façade elements from images. The new framework contains two types of attention modules, namely, the spatial attention module, and the channel attention module. A new loss function is designed to integrate prior engineering knowledge, which can be used to force the detection of façade elements (e.g., windows, doors, concrete walls, and sunshades) to have a highly proportionate shape. The effectiveness of the developed approach is demonstrated in two public datasets. Experimental results indicate that the developed deep learning framework with a new loss function outperforms state-of-the-art models significantly, where the achieved a Mean Intersection over Union (IoU) on the Ecole Centrale Paris(ECP) dataset (81.9%) brings an improvement of 11.3% over Fully Convolutional Network (FCN) and 4.0% over Deepfaçade, respectively, and the achieved a Mean IoU on the ArtDeco dataset (85.6%) yields an improvement of 11.4% over FCN and 5.8% over Deepfaçade, respectively. Moreover, the developed approach is more practical and effective to detect regularized façade elements, where the detection of the wall components has an IoU of 93.6% on the ECP dataset and 88.6% on the ArtDeco dataset, respectively. Overall, the contribution from the technical aspect is to develop a hieratical deep learning framework consisting of attention modules together with the newly designed loss function and the prior engineering knowledge. The contribution from the practical aspect is to realize the automatic and accurate detection for various building façade elements in complex environments, which can be potentially helpful for the infrastructure monitoring and maintenance operation.

Curtain wall frame segmentation using a dual-flow aggregation network: Application to robot pose estimation

Automatic segmentation of curtain wall frame using a context collaboration pyramid network

Automatic curtain wall frame detection based on deep learning and cross-modal feature fusion

End-Face Localization and Segmentation of Steel Bar Based on Convolution Neural Network.

ObjectFusion: an Object Detection and Segmentation Framework with RGB-D SLAM and Convolutional Neural Networks

3D Object Segmentation Using Cross-Window Point Transformer with Latent Semantic Boundary Guidance

A Multi-channel Input Framework for Structure Displacement Response Prediction Using Convolutional Neural Network

Research on Non-Destructive Testing of Curtain Wall Hangings Based on Efficient Channel Attention Vision Transformer Network

Resformer-Unet: A U-shaped Framework Combining ResNet and Transformer for Segmentation of Strip Steel Surface Defects

Autonomous Concrete Crack Semantic Segmentation Using Deep Fully Convolutional Encoder–Decoder Network in Concrete Structures Inspection

RGB-D-Based Pose Estimation of Workpieces with Semantic Segmentation and Point Cloud Registration

Semantic segmentation of mechanical assembly using selective kernel convolution UNet with fully connected conditional random field

Deep learning for detecting building façade elements from images considering prior knowledge

Window Detection In Facade Imagery: A Deep Learning Approach Using Mask R-CNN

Intelligent recognition of joints and fissures in tunnel faces using an improved mask region‐based convolutional neural network algorithm

WallNet: Hierarchical Visual Attention-Based Model for Putty Bulge Terminal Points Detection

Clothes Grasping and Unfolding Based on RGB-D Semantic Segmentation

Weld Seam Tracking and Detection Robot Based on Artificial Intelligence Technology

Automatic crack detection on concrete and asphalt surfaces using semantic segmentation network with hierarchical Transformer

EFDCNet: Encoding Fusion and Decoding Correction Network for RGB-D Indoor Semantic Segmentation

A robot grasping detection network based on flexible selection of multi-modal feature fusion structure