Abstract:The main challenge of cross-modal matching is to construct a shared subspace reflecting semantic closeness. Asymmetric relevance, especially the one-to-many matching case where multiple correspondences of a given query from the other modality can be obtained, has seriously exacerbated this difficulty. Recently, many approaches based on deep metric learning and probability distribution fitting have made great progress. However, these methods can not generalize well to other approaches for their too many hyper-parameters or uncontrollable instability. In addition, we argue that the representational ability of common space learned in real space is insufficient and the symmetric similarity calculations adopted by the previous work fail to capture the asymmetric relevance. To seek a remedy for these problems, this work introduces a novel and effective approach called Q uaternion R epresentation L earning ( QRL ) for better cross-modal matching. Specifically, benefiting from the strong expressive power and richer representation capability of quaternion space with three imaginary components, our proposed QRL method can better represent the shared semantic. Due to the inherent asymmetry of the Hamilton product and its latent inter-dependencies of components, our QRL method can model the asymmetric relevance and capture the complex similarity among intra- and inter-modal interaction. Another advantage is that our QRL method can be utilized in conjunction with other existing methods to promote cross-modal matching. Extensive experiments on two commonly used image-text matching benchmarks, i.e., MSCOCO and Flickr30K, and two widely used video-text retrieval datasets, including MSRVTT and TGIF, demonstrate the effectiveness and superiority of our proposed QRL method.

Improved Representation of Asymmetrical Distances with Interval Quasimetric Embeddings

Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learning

Distance-Adaptive Quaternion Knowledge Graph Embedding with Bidirectional Rotation

Sliced Maximal Information Coefficient: A Training-Free Approach for Image Quality Assessment Enhancement

Quaternion Representation Learning for cross-modal matching

Learning event representation: As sparse as possible, but not sparser

Abstracted Shapes as Tokens -- A Generalizable and Interpretable Model for Time-series Classification

Quantifying Visual Image Quality: A Bayesian View

Perceptual Quality Analysis in Deep Domains Using Structure Separation and High-Order Moments

DeepWSD: Projecting Degradations in Perceptual Space to Wasserstein Distance in Deep Feature Space

Learning Hierarchy-Aware Quaternion Knowledge Graph Embeddings with Representing Relations as 3D Rotations.

Image Quality Assessment: Integrating Model-Centric and Data-Centric Approaches

SQuadMDS: a lean Stochastic Quartet MDS improving global structure preservation in neighbor embedding like t-SNE and UMAP

DeepDC: Deep Distance Correlation as a Perceptual Image Quality Evaluator

Quadra-embedding: Binary code embedding with low quantization error

I-SEA: Importance Sampling and Expected Alignment-Based Deep Distance Metric Learning for Time Series Analysis and Embedding

Semantic Data Augmentation based Distance Metric Learning for Domain Generalization

Coordinate Quantized Neural Implicit Representations for Multi-view Reconstruction

Visual Embedding Augmentation in Fourier Domain for Deep Metric Learning

Modeling Noncommutative Composition of Relations for Knowledge Graph Embedding

Social Embedding Image Distance Learning