Abstract:In the field of referring segmentation, top-down methods and bottom-up methods are the two prevailing approaches. Both of these methods inevitably exhibit certain drawbacks. Top-down methods are susceptible to Polar Negative (PN) errors due to their limited understanding of multi-modal fine-grained features. Bottom-up methods lack macro-level object positional information, making them susceptible to Inferior Positive (IP) errors. However, we find that the two approaches are highly complementary in addressing their respective weaknesses, but combining them directly through a simple average does not yield complementary advantages. Therefore, we proposed a hierarchical collaboration approach to explore the complementary characteristics of the existing two methods from the perspectives of fusion and interaction, aiming to achieve more precise segmentation results. We proposed the Complementary Feature Interaction (CFI) module, which enables top-down methods to access fine-grained information and allows bottom-up approaches to obtain object positional information interactively. Regarding integration, Gaussian Scoring Integration (GSI) models the Gaussian performance distributions of two branches and performs weighted integration by sampling confidence scores from these distributions. We integrate various top-down and bottom-up methods within the proposed architecture and conduct experiments on three standard datasets. The experimental results demonstrate that our method outperforms the state-of-the-art independent segmentation algorithms. On the RefCOCO validation, test A and test B datasets, our proposed method achieved IoU scores of 77.51, 79.12, and 72.79, respectively. Extensive experiments demonstrate that our method can significantly improve segmentation accuracy when fusing different sub-methods.

A survey of methods for addressing the challenges of referring image segmentation

A Survey on Semantic Segmentation.

Beyond One-to-One: Rethinking the Referring Image Segmentation

A Brief Survey on Semantic Segmentation with Deep Learning

Locate then Segment: A Strong Pipeline for Referring Image Segmentation

A Survey on Deep Learning Technique for Video Segmentation

RRSIS: Referring Remote Sensing Image Segmentation

Image Segmentation with Language Referring Expression and Comprehension

Techniques and Challenges of Image Segmentation: A Review

Semantics-Aware Dynamic Localization and Refinement for Referring Image Segmentation

Deep Learning based 3D Segmentation: A Survey

Deep-Learning-Based Point Cloud Semantic Segmentation: A Survey

Advancing Referring Expression Segmentation Beyond Single Image

Intelligent Image Semantic Segmentation: A Review Through Deep Learning Techniques for Remote Sensing Image Analysis

Referring Image Segmentation via Cross-Modal Progressive Comprehension

Image Segmentation in Foundation Model Era: A Survey

Automatic image co-segmentation: a survey

Referring Image Segmentation via Text Guided Multi-Level Interaction

Hierarchical Collaboration for Referring Image Segmentation

A Comprehensive Review of Modern Object Segmentation Approaches