Abstract:The emerging research line of cross-modal learning focuses on the issue of transferring feature representation manner learned from limited multimodal data with labelings to the testing phase with partial modalities. This is essentially common and practical in the remote sensing (RS) community when only modal-incomplete data are in users' hands due to inevitable imaging or access restrictions under large-scale observation scenarios. However, most of the existing cross-modal learning methods have been designed with exclusive reliance on labeling, which can be either limited or noisy due to their costly production. To address this issue, we explore, in this article, the possibility to learn cross-modal feature representation in an unsupervised fashion. By integrating the multimodal data into a fully recombined matrix form, we propose the following: 1) the use of common subspace representation as the regression target instead of conventionally adopted binary labels and 2) the orthogonality and manifold alignment regularization terms to shrink the solution space while preserving the pairwise manifold correlations. Through this manner, the modality-specific and mutual latent representations in this common subspace as well as their corresponding projections can be learned simultaneously, and their optimums can be efficiently reached through a nearly one-step computation with the help of eigendecomposition. Finally, we show the superiority of our method through extensive image classification experiments on three multimodal datasets with four remotely sensed modalities involved (i.e., hyperspectral (HS), multispectral, synthetic aperture radar, and light detection and ranging (LiDAR) data). The code and dataset will be made freely available at https://github.com/jingyao16/UCSL to encourage the reproduction of our method and further use.

Unimodal Biometric System Based on Subspace Learning Method

Unimodal biometric system based on local topology structure preserving projections

Learning a Locality Preserving Subspace for Visual Recognition.

Multimodal biometrics using geometry preserving projections

A Robust Face and Ear Based Multimodal Biometric System Using Sparse Representation

Face Recognition Based on Local Binary Pattern and Improved Pairwise-constrained Multiple Metric Learning

Face Recognition Using Data Driven Local Appearance Features

Bimodal Biometrics Based on a Representation and Recognition Approach

Joint multi-subspace feature learning with singular value decomposition for robust single-sample face recognition

A Sparse Representation Method of Bimodal Biometrics and Palmprint Recognition Experiments

A Modified Chaotic Binary Particle Swarm Optimization Scheme and Its Application in Face-Iris Multimodal Biometric Identification

Uncorrelated Multilinear Geometry Preserving Projections For Multimodal Biometrics Recognition

Cost-sensitive Subspace Learning for Face Recognition

Human Gait Recognition Via Sparse Discriminant Projection Learning.

Cost-Sensitive Subspace Analysis and Extensions for Face Recognition

Single Image Subspace For Face Recognition

A Multi-Modal Face Recognition Method Using Complete Local Derivative Patterns And Depth Maps

A Doubly Weighted Approach for Appearance-Based Subspace Learning Methods

Biometric identity recognition based on contrastive positive-unlabeled learning

A Vision System for Multi-View Face Recognition

UCSL: Toward Unsupervised Common Subspace Learning for Cross-Modal Image Classification