Abstract:Semantic correspondence remains a challenging task for establishing correspondences between a pair of images with the same category or similar scenes due to the large intra-class appearance. In this paper, we introduce a novel problem called 'Small Object Semantic Correspondence (SOSC).' This problem is challenging due to the close proximity of keypoints associated with small objects, which results in the fusion of these respective features. It is difficult to identify the corresponding key points of the fused features, and it is also difficult to be recognized. To address this challenge, we propose the Keypoint Bounding box-centered Cropping (KBC) method, which aims to increase the spatial separation between keypoints of small objects, thereby facilitating independent learning of these keypoints. The KBC method is seamlessly integrated into our proposed inference pipeline and can be easily incorporated into other methodologies, resulting in significant performance enhancements. Additionally, we introduce a novel framework, named KBCNet, which serves as our baseline model. KBCNet comprises a Cross-Scale Feature Alignment (CSFA) module and an efficient 4D convolutional decoder. The CSFA module is designed to align multi-scale features, enriching keypoint representations by integrating fine-grained features and deep semantic features. Meanwhile, the 4D convolutional decoder, based on efficient 4D convolution, ensures efficiency and rapid convergence. To empirically validate the effectiveness of our proposed methodology, extensive experiments are conducted on three widely used benchmarks: PF-PASCAL, PF-WILLOW, and SPair-71k. Our KBC method demonstrates a substantial performance improvement of 7.5\% on the SPair-71K dataset, providing compelling evidence of its efficacy.

Barycentric Coordinates Based Soft Assignment for Object Classification

Object Recognition Based on the Region of Interest and Optimal Bag of Words Model.

Improved Soft Assignment Coding for Image Classification

In defense of soft-assignment coding

Context-Aware and Locality-Constrained Coding for Image Categorization

Soft-assigned bag of features tracking

Robust Feature Encoding with Neighborhood Information for Image Classification

Spherical Soft Assignment: Improving Image Representation in Content-Based Image Retrieval

Adjacent Coding for Image Classification.

Spatial Encoding of Visual Words for Image Classification.

Independently Keypoint Learning for Small Object Semantic Correspondence

Incorporating Contextual Information into Bag-of-Visual-Words Framework for Effective Object Categorization

Spatial Locality-Preserving Feature Coding for Image Classification

Integrating ILSR to Bag-of-Visual Words Model Based on Sparse Codes of SIFT Features Representations

A discriminative spatial bag-of-word scheme with distinct patch

Towards Optimal Bag-of-features for Object Categorization and Semantic Video Retrieval

Soft-Assigned Bag of Features for Object Tracking

An image classification method based on bag of words model

Create visual word pairs dynamically based on sparse codes of SIFT features for image categorization

Object categorization based on a supervised mean shift algorithm

Improving Bag-of-words Scheme for Scene Categorization