Abstract:This research explores the complex yet applicable domain of universal visual recognition, where unlabeled datasets may encompass additional, as-of-yet unidentified classes and present a skewed feature distribution within labeled datasets. The primary challenge lies in concurrently distinguishing instances of known classes and uncovering new classes within the unlabeled data. This difficulty is compounded by two factors: category shift and distribution shift. Current methodologies typically overlook the potential of harnessing the inherent structural relationship between labeled and unlabeled datasets, and they fail to effectively isolate the semantic variability of the newly identified classes from the established ones. To overcome these hurdles, we introduce an innovative framework, which we have termed MESA (short for Manifold structure Exploitation with double Spaces dual Alignment), to advance the field of universal visual recognition. Our approach begins by identifying geometrically and semantically related pairs of nearest neighbors between labeled and unlabeled data, which serve as anchor points to foster an intrinsic match between the two datasets. By integrating an affinity score for similarity, we establish a novel cross-view, anchor-guided, self-supervised learning paradigm. This model is designed to spread the learned labels’ information and to conjoin novel semantic insights within the predictive space. To intensify the matching accuracy for the known classes and enhance the differentiation of novel classes, we further implement a confident-prototype-driven, self-supervised learning paradigm in a cross-view manner. This strategy is aimed at revealing the macroscopic category configuration within the embedding space. MESA employs a pioneering bidirectional dual-alignment mechanism that operates simultaneously in the embedding and prediction spaces, hence providing a more robust approach to confronting challenges brought about by distribution and category shifts. Our exhaustive evaluation across several extensive image recognition benchmarks substantiates MESA’s superior performance over the latest cutting-edge methods, marking a significant leap forward in the sphere of universal visual recognition. Finally, MESA is implemented in Python using the Pytorch machine-learning library, and it is freely available at https://github.com/aimeeyaoyao/MESA.

Dual low-rank structure embedding for robust visual information processing

Robust sparse low-rank embedding for image dimension reduction

Low-rank 2D Local Discriminant Graph Embedding for Robust Image Feature Extraction

LGSLRR: Towards fusing discriminative ordinal local and global structured low-rank representation for image recognition

A New Low-Rank Structurally Incoherent Algorithm for Robust Image Feature Extraction

Structure Preserving Projections Learning Via Low-Rank Embedding for Image Classification

Robust Multiview Feature Learning for RGB-D Image Understanding

Low-rank Representation Based Discriminative Projection for Robust Feature Extraction

Joint Structural Learning to Rank with Deep Linear Feature Learning

Joint enhanced low-rank constraint and kernel rank-order distance metric for low level vision processing

Sparse fixed-rank representation for robust visual analysis.

Intrinsic Structure Exploitation with Dual Alignment for Universal Visual Recognition

Feature Extraction Based on Low-rank Affinity Matrix for Biological Recognition

Learning Low-Rank Sparse Representations With Robust Relationship Inference for Image Memorability Prediction

Robust low rank representation via feature and sample scaling

ROMIR: Robust Multi-View Image Re-Ranking

Robust Face Recognition Based on Non-Negative Sparse Discriminative Low-Rank Representation

Robust auto-weighted projective low-rank and sparse recovery for visual representation

Joint Embedding Learning and Low-Rank Approximation: A Framework for Incomplete Multi-view Learning

Learning Robust Visual-Semantic Embeddings

Low-Rank Discriminative Embedding Regression for Robust Feature Extraction of Hyperspectral Images via Weighted Schatten p-Norm Minimization