Abstract:3D geo-information is of great significance for understanding the living environment; however, 3D perception from remote sensing data, especially on a large scale, is restricted. To tackle this problem, we propose a method for monocular height estimation from optical imagery, which is currently one of the richest sources of remote sensing data. As an ill-posed problem, monocular height estimation requires well-designed networks for enhanced representations to improve performance. Moreover, the distribution of height values is long-tailed with the low-height pixels, e.g., the background, as the head, and thus trained networks are usually biased and tend to underestimate building heights. To solve the problems, instead of formalizing the problem as a regression task, we propose HTC-DC Net following the classification-regression paradigm, with the head-tail cut (HTC) and the distribution-based constraints (DCs) as the main contributions. HTC-DC Net is composed of the backbone network as the feature extractor, the HTC-AdaBins module, and the hybrid regression process. The HTC-AdaBins module serves as the classification phase to determine bins adaptive to each input image. It is equipped with a vision transformer encoder to incorporate local context with holistic information and involves an HTC to address the long-tailed problem in monocular height estimation for balancing the performances of foreground and background pixels. The hybrid regression process does the regression via the smoothing of bins from the classification phase, which is trained via DCs. The proposed network is tested on three datasets of different resolutions, namely ISPRS Vaihingen (0.09 m), DFC19 (1.3 m) and GBH (3 m). Experimental results show the superiority of the proposed network over existing methods by large margins. Extensive ablation studies demonstrate the effectiveness of each design component.

IMG2HEIGHT: height estimation from single remote sensing image using a deep convolutional encoder-decoder network

IM2HEIGHT: Height Estimation from Single Monocular Imagery via Fully Residual Convolutional-Deconvolutional Network

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

Height estimation from single aerial images using a deep ordinal regression network

HTC-DC Net: Monocular Height Estimation from Single Remote Sensing Images

HGDNet: A Height-Hierarchy Guided Dual-Decoder Network for Single View Building Extraction and Height Estimation

JSH-Net: joint semantic segmentation and height estimation using deep convolutional networks from single high-resolution remote sensing imagery

HeightFormer: A Multilevel Interaction and Image-adaptive Classification-regression Network for Monocular Height Estimation with Aerial Images

IM2ELEVATION: Building Height Estimation from Single-View Aerial Imagery

HeightFormer: A Multilevel Interaction and Image-Adaptive Classification–Regression Network for Monocular Height Estimation with Aerial Images

HeightFormer: Single-Imagery Height Estimation Transformer With Bilateral Feature Pyramid Fusion

Multi-Task Learning of Relative Height Estimation and Semantic Segmentation from Single Airborne RGB Images

Hierarchical Self-Attention Embedded Neural Network With Dense Connection for Remote-Sensing Image Semantic Segmentation

MF-BHNet: A Hybrid Multimodal Fusion Network for Building Height Estimation Using Sentinel-1 and Sentinel-2 Imagery

A Cnn-Based Depth Estimation Approach With Multi-Scale Sub-Pixel Convolutions And A Smoothness Constraint

FusionHeightNet: A Multi-Level Cross-Fusion Method from Multi-Source Remote Sensing Images for Urban Building Height Estimation

Disentangled Latent Transformer for Interpretable Monocular Height Estimation

StyHighNet: Semi-Supervised Learning Height Estimation from a Single Aerial Image via Unified Style Transferring

Surface Height Map Estimation from a Single Image Using Convolutional Neural Networks

Attention-Based Monocular Depth Estimation Considering Global and Local Information in Remote Sensing Images

Stereollax Net: Stereo Parallax-Based Deep Learning Network for Building Height Estimation