Abstract:3D geo-information is of great significance for understanding the living environment; however, 3D perception from remote sensing data, especially on a large scale, is restricted. To tackle this problem, we propose a method for monocular height estimation from optical imagery, which is currently one of the richest sources of remote sensing data. As an ill-posed problem, monocular height estimation requires well-designed networks for enhanced representations to improve performance. Moreover, the distribution of height values is long-tailed with the low-height pixels, e.g., the background, as the head, and thus trained networks are usually biased and tend to underestimate building heights. To solve the problems, instead of formalizing the problem as a regression task, we propose HTC-DC Net following the classification-regression paradigm, with the head-tail cut (HTC) and the distribution-based constraints (DCs) as the main contributions. HTC-DC Net is composed of the backbone network as the feature extractor, the HTC-AdaBins module, and the hybrid regression process. The HTC-AdaBins module serves as the classification phase to determine bins adaptive to each input image. It is equipped with a vision transformer encoder to incorporate local context with holistic information and involves an HTC to address the long-tailed problem in monocular height estimation for balancing the performances of foreground and background pixels. The hybrid regression process does the regression via the smoothing of bins from the classification phase, which is trained via DCs. The proposed network is tested on three datasets of different resolutions, namely ISPRS Vaihingen (0.09 m), DFC19 (1.3 m) and GBH (3 m). Experimental results show the superiority of the proposed network over existing methods by large margins. Extensive ablation studies demonstrate the effectiveness of each design component.

THE Benchmark: Transferable Representation Learning for Monocular Height Estimation

THE Benchmark: Transferable Representation Learning for Monocular Height Estimation

Benchmarking Large-Scale Multi-View 3D Reconstruction Using Realistic Synthetic Images

Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer

HeightFormer: A Multilevel Interaction and Image-Adaptive Classification–Regression Network for Monocular Height Estimation with Aerial Images

HTC-DC Net: Monocular Height Estimation from Single Remote Sensing Images

HeightFormer: A Multilevel Interaction and Image-adaptive Classification-regression Network for Monocular Height Estimation with Aerial Images

HA-Bins: Hierarchical Adaptive Bins for Robust Monocular Depth Estimation across Multiple Datasets

Every Dataset Counts: Scaling up Monocular 3D Object Detection with Joint Datasets Training

Disentangled Latent Transformer for Interpretable Monocular Height Estimation

Monocular Image-Based 3-D Model Retrieval: A Benchmark

Towards Robust Monocular Depth Estimation: A New Baseline and Benchmark

IM2HEIGHT: Height Estimation from Single Monocular Imagery via Fully Residual Convolutional-Deconvolutional Network

A Simple and Reliable Method for Estimating Building-Scale Height Based on Multisource Datasets

Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision

Boosting Monocular Depth Estimation with Sparse Guided Points

HeightFormer: A Semantic Alignment Monocular 3D Object Detection Method from Roadside Perspective

A New Dataset for Monocular Depth Estimation Under Viewpoint Shifts

Building3D: An Urban-Scale Dataset and Benchmarks for Learning Roof Structures from Point Clouds

DiverseDepth: Affine-invariant Depth Prediction Using Diverse Data

Enhancing Monocular Height Estimation from Aerial Images with Street-view Images