Abstract:Glass is very common in the real world. Influenced by the uncertainty about the glass region and the varying complex scenes behind the glass, the existence of glass poses severe challenges to many computer vision tasks, making glass segmentation as an important computer vision task. Glass does not have its own visual appearances but only transmit/reflect the appearances of its surroundings, making it fundamentally different from other common objects. To address such a challenging task, existing methods typically explore and combine useful cues from different levels of features in the deep network. As there exists a characteristic gap between level-different features, i.e., deep layer features embed more high-level semantics and are better at locating the target objects while shallow layer features have larger spatial sizes and keep richer and more detailed low-level information, fusing these features naively thus would lead to a sub-optimal solution. In this paper, we approach the effective features fusion towards accurate glass segmentation in two steps. First, we attempt to bridge the characteristic gap between different levels of features by developing a Discriminability Enhancement (DE) module which enables level-specific features to be a more discriminative representation, alleviating the features incompatibility for fusion. Second, we design a Focus-and-Exploration Based Fusion (FEBF) module to richly excavate useful information in the fusion process by highlighting the common and exploring the difference between level-different features. Combining these two steps, we construct a Progressive Glass Segmentation Network (PGSNet) which uses multiple DE and FEBF modules to progressively aggregate features from high-level to low-level, implementing a coarse-to-fine glass segmentation. In addition, we build the first home-scene-oriented glass segmentation dataset for advancing househo-d robot applications and in-depth research on this topic. Extensive experiments demonstrate that our method outperforms 26 cutting-edge models on three challenging datasets under four standard metrics. The code and dataset will be made publicly available.

Leveraging RGB-D Data with Cross-Modal Context Mining for Glass Surface Detection

Glass Detection and Recognition Based on the Fusion of Ultrasonic Sensor and RGB-D Sensor for the Visually Impaired

Large-Field Contextual Feature Learning for Glass Detection

Robustifying Semantic Cognition of Traversability Across Wearable RGB-depth Cameras

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

PanoGlassNet: Glass Detection With Panoramic RGB and Intensity Images

Progressive Glass Segmentation

Glass Segmentation With RGB-Thermal Image Pairs

GhostingNet: A Novel Approach for Glass Surface Detection With Ghosting Cues

Cross-Modal Attentional Context Learning for RGB-D Object Detection

GEM: Boost Simple Network for Glass Surface Segmentation via Segment Anything Model and Data Synthesis

Glass Makes Blurs: Learning the Visual Blurriness for Glass Surface Detection

Transparency-Aware Segmentation of Glass Objects to Train RGB-Based Pose Estimators

Internal-External Boundary Attention Fusion for Glass Surface Segmentation

GlassNet: Label Decoupling-based Three-stream Neural Network for Robust Image Glass Detection

GEM: Boost Simple Network for Glass Surface Segmentation via Vision Foundation Models

Surface-biased Multi-Level Context 3D Object Detection

Enhanced Boundary Learning for Glass-like Object Segmentation

Spatial-information Guided Adaptive Context-aware Network for Efficient RGB-D Semantic Segmentation

RGB-D Grasp Detection via Depth Guided Learning with Cross-modal Attention

Salient Object Detection for RGBD Video Via Spatial Interaction and Depth-Based Boundary Refinement