Abstract:Due to the large intra-class differences between the same categories and the scale imbalance between different categories in the remote sensing image dataset, the semantic segmentation task presents the problem of small-scale object information loss, the imbalance between foreground and background, and simultaneously the background dominates, which seriously affects the performance of the network model. To solve the above problems, this paper proposes an efficient bilateral branch depth neural network model based on the U-Net depth neural network, named BBU-Net. Firstly, one branch of the network learns the distribution characteristics of the original data, and the other focuses on difficult samples. Then the two branches improve the representation and classification ability of the neural network by accumulating learning strategies. Finally, considering the geometric diversity of remote sensing images, this paper adopts test time augmentation and reflection padding strategies and proposes a balanced weighted loss function named CombineLoss to alleviate the imbalance in the training process. The depth neural network proposed in this paper was first tested on the Inria Aerial Image Labeling Dataset, and 87.53% of mean intersection over union and 97.4% of mean pixel accuracy were obtained, respectively. At the same time, to verify the model's complexity, the model proposed in this paper is compared with the neural network based on integrated learning. The comparison results show that the spatial complexity of the network proposed in this paper is much lower than the neural network obtained by integrated learning, and the parameters are also much smaller than the neural network based on integrated learning. Then use the satellite building dataset I in the WHU Building Dataset and mainstream semantic segmentation methods for multiple groups of comparative experiments. The experimental results show that the method proposed in this paper can effectively extract the semantic information of remote sensing images, significantly improve the imbalance of remote sensing image data, improve the performance of the network model, and achieve a good semantic segmentation effect, which fully proves the effectiveness of this method.

JSH-Net: joint semantic segmentation and height estimation using deep convolutional networks from single high-resolution remote sensing imagery

IMG2HEIGHT: height estimation from single remote sensing image using a deep convolutional encoder-decoder network

Hierarchical Self-Attention Embedded Neural Network With Dense Connection for Remote-Sensing Image Semantic Segmentation

JSNet: Joint Instance and Semantic Segmentation of 3D Point Clouds

Transformer and CNN Hybrid Deep Neural Network for Semantic Segmentation of Very-High-Resolution Remote Sensing Imagery

JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds

HSNet: an Intelligent Hierarchical Semantic-Aware Network System for Real-Time Semantic Segmentation

Multi-Attention-Based Semantic Segmentation Network for Land Cover Remote Sensing Images

Joint Learning of Semantic Segmentation and Height Estimation for Remote Sensing Image Leveraging Contrastive Learning

SSNet: a joint learning network for semantic segmentation and disparity estimation

HTC-DC Net: Monocular Height Estimation from Single Remote Sensing Images

An Attention-Fused Network for Semantic Segmentation of Very-High-Resolution Remote Sensing Imagery

SOSD-Net: Joint Semantic Object Segmentation and Depth Estimation from Monocular Images

Joint Classification of Hyperspectral and LiDAR Data Using Height Information Guided Hierarchical Fusion-and-Separation Network

Densely Based Multi-Scale and Multi-Modal Fully Convolutional Networks for High-Resolution Remote-Sensing Image Semantic Segmentation

Semantic Segmentation of Very-High-Resolution Remote Sensing Images via Deep Multi-Feature Learning

Semantic segmentation of remote sensing image based on bilateral branch network

CI-Net: a joint depth estimation and semantic segmentation network using contextual information

Full Semantic Constructed Network for Urban Use Classification from Very High-Resolution Optical Remote Sensing Imagery

Semantic Labeling Of High Resolution Aerial Imagery And Lidar Data With Fine Segmentation Network

Multi-Task Learning of Relative Height Estimation and Semantic Segmentation from Single Airborne RGB Images