Abstract:Scene segmentation via unsupervised domain adaptation (UDA) enables the transfer of knowledge acquired from source synthetic data to real-world target data, which largely reduces the need for manual pixel-level annotations in the target domain. To facilitate domain-invariant feature learning, existing methods typically mix data from both the source domain and target domain by simply copying and pasting the pixels. Such vanilla methods are usually sub-optimal since they do not take into account how well the mixed layouts correspond to real-world scenarios. Real-world scenarios are with an inherent layout. We observe that semantic categories, such as sidewalks, buildings, and sky, display relatively consistent depth distributions, and could be clearly distinguished in a depth map. Based on such observation, we propose a depth-aware framework to explicitly leverage depth estimation to mix the categories and facilitate the two complementary tasks, i.e., segmentation and depth learning in an end-to-end manner. In particular, the framework contains a Depth-guided Contextual Filter (DCF) forndata augmentation and a cross-task encoder for contextual learning. DCF simulates the real-world layouts, while the cross-task encoder further adaptively fuses the complementing features between two tasks. Besides, it is worth noting that several public datasets do not provide depth annotation. Therefore, we leverage the off-the-shelf depth estimation network to generate the pseudo depth. Extensive experiments show that our proposed methods, even with pseudo depth, achieve competitive performance on two widely-used bench-marks, i.e. 77.7 mIoU on GTA to Cityscapes and 69.3 mIoU on Synthia to Cityscapes.

Depth Segmentation in Real-World Scenes Based on U-V Disparity Analysis

A New Approach of Point Cloud Processing and Scene Segmentation for Guiding the Visually Impaired

Unifying Terrain Awareness Through Real-Time Semantic Segmentation

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

Spatio-Temporal Segmentation with Depth-Inferred Videos of Static Scenes

Object segmentation based on disparity estimation

Depth Extraction Method Based on Multi-view Stereo Matching

Spatio-Temporal Video Segmentation of Static Scenes and Its Applications

Object segmentation based on accurate disparity estimation

New Real-Time Algorithm for Multiview Disparity-Based 3D-Object Segmentation.

A Depth-Based Land-Use Semantic Extraction Method for Landscape Images

Uplifting Range-View-based 3D Semantic Segmentation in Real-Time with Multi-Sensor Fusion

Transferring to Real-World Layouts: A Depth-aware Framework for Scene Adaptation

Exploiting Depth from Single Monocular Images for Object Detection and Semantic Segmentation

Uncertainty-aware RGBD Image Segmentation.

Omnidirectional Depth Estimation for Semantic Segmentation

Human Segmentation Based on Disparity Map and GrabCut

DiverseDepth: Affine-invariant Depth Prediction Using Diverse Data

Depth-Guided Semi-Supervised Instance Segmentation

Semantic-Guided Representation Enhancement for Self-supervised Monocular Trained Depth Estimation

Depth-guided Texture Diffusion for Image Semantic Segmentation