Abstract:By characterizing each image set as a nonsingular covariance matrix on the symmetric positive definite (SPD) manifold, the approaches of visual content classification with image sets have made impressive progress. However, the key challenge of unhelpfully large intraclass variability and interclass similarity of representations remains open to date. Although, several recent studies have mitigated the two problems by jointly learning the embedding mapping and the similarity metric on the original SPD manifold, their inherent shallow and linear feature transformation mechanism are not powerful enough to capture useful geometric features, especially in complex scenarios. To this end, this article explores a novel approach, termed SPD manifold deep metric learning (SMDML), for image set classification. Specifically, SMDML first selects a prevailing SPD manifold neural network (SPDNet) as the backbone (encoder) to derive an SPD matrix nonlinear representation. To counteract the degradation of structural information during multistage feature embedding, we construct a Riemannian decoder at the end of the encoder, trained by a reconstruction error term (RT), to induce the generated low-dimensional feature manifold of the hidden layer to capture the pivotal information about the visual data describing the imaged scene. We demonstrate through theory and experiments that it is feasible to replace the Riemannian metric with Euclidean distance in RT. Then, the ReCov layer is introduced into the established Riemannian network to regularize the local statistical information within each input feature matrix, which enhances the effectiveness of the learning process. The theoretical analysis of the activation function used in the ReCov layer in terms of continuity and conditions for generating positive definite matrices is beneficial for network design. Inspired by the fact that the single cross-entropy loss used for training is unable to effectively parse the geometric distribution of the deep representations, we finally endow the suggested model with a novel metric learning regularization term. By explicitly incorporating the encoding and processing of the data variations into the network learning process, this term can not only derive a powerful Riemannian representation but also train an effective classifier. The experimental results show the superiority of the proposed approach on three typical visual classification tasks.

A New Supervised Image Classification Using Data Set Partition

A lightweight and stochastic depth residual attention network for remote sensing scene classification

Fast Low Rank Representation Based Spatial Pyramid Matching for Image Classification.

Hybrid Classification Approach Using Extreme Learning Machine and Sparse Representation Classifier with Adaptive Threshold

Robust Self-Paced Incremental Learning for Multitemporal Remote Sensing Image Classification

Alleviating Long-Tailed Image Classification via Dynamical Classwise Splitting

SPD Manifold Deep Metric Learning for Image Set Classification

Data Classification Algorithm Based on Sparse Decomposition

Superimposed Sparse Parameter Classifiers for Face Recognition

Extreme Sparse Multinomial Logistic Regression: A Fast and Robust Framework for Hyperspectral Image Classification

Singular Value Decomposition Projection for solving the small sample size problem in face recognition

SCS-SLSP: Hard Uncertain Pixels Mining and Utilization for Semi-Supervised Cardiac Image Segmentation Using Subjective Logic Theory and Subset Prototype Generation

Instance-Level Scaling and Dynamic Margin-Alignment Knowledge Distillation for Remote Sensing Image Scene Classification

Deep Metric Learning on the SPD Manifold for Image Set Classification

Embrace Sustainable AI: Dynamic Data Subset Selection for Image Classification

Classification of Spatiotemporal Events Based on Random Forest

Semi-supervised Regression with Data Partitioning and Feature Mapping

SSRNet: A Lightweight Successive Spatial Rectified Network With Noncentral Positional Sampling Strategy for Hyperspectral Images Classification

Large-scale support vector machine classification with redundant data reduction

SP-ILC: Concurrent Single-Pixel Imaging, Object Location, and Classification by Deep Learning

DECOR: Dynamic Decoupling and Multi-Objective Optimization for Long-tailed Remote Sensing Image Classification