Abstract:Feature fusion is a commonly used strategy in image retrieval tasks, which aggregates the matching responses of multiple visual features. Feasible sets of features can be either descriptors (SIFT, HSV) for an entire image or the same descriptor for different local parts (face, body). Ideally, the to-be-fused heterogeneous features are pre-assumed to be discriminative and complementary to each other. However, the effectiveness of different features varies dramatically according to different queries. That is to say, for some queries, a feature may be neither discriminative nor complementary to existing ones, while for other queries, the feature suffices. As a result, it is important to estimate the effectiveness of features in a query-adaptive manner. To this end, this article proposes a new late fusion scheme at the score level. We base our method on the observation that the sorted score curves contain patterns that describe their effectiveness. For example, an "L"-shaped curve indicates that the feature is discriminative while a gradually descending curve suggests a bad feature. As such, this paper introduces a query-adaptive late fusion pipeline. In the hand-crafted version, it can be an unsupervised approach to tasks like particular object retrieval. In the learning version, it can also be applied to supervised tasks like person recognition and pedestrian retrieval, based on a trainable neural module. Extensive experiments are conducted on two object retrieval datasets and one person recognition dataset. We show that our method is able to highlight the good features and suppress the bad ones, is resilient to distractor features, and achieves very competitive retrieval accuracy compared with the state of the art. In an additional person re-identification dataset, the application scope and limitation of the proposed method are studied.

Comprehensive Framework of Early and Late Fusion for Image-Sentence Retrieval

Combine Early and Late Fusion Together: A Hybrid Fusion Framework for Image-Text Matching

Query Adaptive Late Fusion for Image Retrieval.

Scene Graph Based Fusion Network For Image-Text Retrieval

Where Elegance Meets Precision: Towards a Compact, Automatic, and Flexible Framework for Multi-modality Image Fusion and Applications

Searching a Hierarchically Aggregated Fusion Architecture for Fast Multi-Modality Image Fusion

A Unified Image Fusion Framework with Flexible Bilevel Paradigm Integration

Image Retrieval with Composed Query by Multi-Scale Multi-Modal Fusion.

An End-to-End Framework Based on Vision-Language Fusion for Remote Sensing Cross-Modal text-Image Retrieval

Visual and textual fusion for semantically supervised region-based retrieval

Visual & textual fusion for region retrieval: from both fuzzy matching and bayesian reasoning aspects.

DCMFNet: Deep Cross-Modal Fusion Network for Different Modalities with Iterative Gated Fusion

TextFusion: Unveiling the Power of Textual Semantics for Controllable Image Fusion

From Text to Pixels: A Context-Aware Semantic Synergy Solution for Infrared and Visible Image Fusion

Improving Fusion of Region Features and Grid Features Via Two-Step Interaction for Image-Text Retrieval

Image Fusion via Vision-Language Model

A multimodal framework for unsupervised feature fusion

Cross-Modal Hybrid Feature Fusion for Image-Sentence Matching

Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network

A Multi-Modal Image Fusion Framework Based on Guided Filter and Sparse Representation

DCMFNet: Deep Cross-Modal Fusion Network for Referring Image Segmentation with Iterative Gated Fusion