Abstract:In the field of referring segmentation, top-down methods and bottom-up methods are the two prevailing approaches. Both of these methods inevitably exhibit certain drawbacks. Top-down methods are susceptible to Polar Negative (PN) errors due to their limited understanding of multi-modal fine-grained features. Bottom-up methods lack macro-level object positional information, making them susceptible to Inferior Positive (IP) errors. However, we find that the two approaches are highly complementary in addressing their respective weaknesses, but combining them directly through a simple average does not yield complementary advantages. Therefore, we proposed a hierarchical collaboration approach to explore the complementary characteristics of the existing two methods from the perspectives of fusion and interaction, aiming to achieve more precise segmentation results. We proposed the Complementary Feature Interaction (CFI) module, which enables top-down methods to access fine-grained information and allows bottom-up approaches to obtain object positional information interactively. Regarding integration, Gaussian Scoring Integration (GSI) models the Gaussian performance distributions of two branches and performs weighted integration by sampling confidence scores from these distributions. We integrate various top-down and bottom-up methods within the proposed architecture and conduct experiments on three standard datasets. The experimental results demonstrate that our method outperforms the state-of-the-art independent segmentation algorithms. On the RefCOCO validation, test A and test B datasets, our proposed method achieved IoU scores of 77.51, 79.12, and 72.79, respectively. Extensive experiments demonstrate that our method can significantly improve segmentation accuracy when fusing different sub-methods.

Hierarchical Image Segmentation Based on Multi-feature Fusion and Graph Cut Optimization.

Decompose Image into Meaningful Regions Based on Contour Detector and Watershed Algorithm

An efficient hierarchical graph based image segmentation

Image Segmentation Using Hierarchical Merge Tree

Object-specific Optimization of Hierarchical Multiscale Segmentations for High-Spatial Resolution Remote Sensing Images

Hierarchical Segmentation Evaluation of Region-Based Image Hierarchy

From Pixels to Objects: A Hierarchical Approach for Part and Object Segmentation Using Local and Global Aggregation

HFS: Hierarchical Feature Selection for Efficient Image Segmentation

Remotely sensed imagery segmentation based on the hierarchical representation of image content

Hierarchical Collaboration for Referring Image Segmentation

Fusion of hierarchical class graphs for remote sensing semantic segmentation

An Improved Hierarchical Segmentation Method for Remote Sensing Images

Deep Hierarchical Representation And Segmentation Of High Resolution Remote Sensing Images

Hierarchical Attention Guided Framework for Multi-resolution Collaborative Whole Slide Image Segmentation

Image semantic segmentation with hierarchical feature fusion based on deep neural network

Multi-value Image Segmentation Based on FCM Algorithm and Graph Cut Theory.

A Multi-Step Fusion Network for Semantic Segmentation of High-Resolution Aerial Images

Adaptive fusion with multi-scale features for interactive image segmentation

Interactive RGB-D Image Segmentation Using Hierarchical Graph Cut and Geodesic Distance

Fast Hierarchical Segmentation of High-Resolution Remote Sensing Image with Adaptive Edge Penalty

A Hierarchical Segmentation Method for Multispectral Imagery