Abstract:We propose a novel discriminative feature learning method via Max-Min Ratio Analysis (MMRA) for exclusively dealing with the long-standing "worst-case class separation" problem. Existing technologies simply consider maximizing the minimal pairwise distance on all class pairs in the low-dimensional subspace, which is unable to separate overlapped classes entirely especially when the distribution of samples within same class is diverging. We propose a new criterion, i.e., Max-Min Ratio Analysis (MMRA) that focuses on maximizing the minimal ratio value of between-class and within-class scatter to extremely enlarge the separability on the overlapped pairwise classes. Furthermore, we develop two novel discriminative feature learning models for dimensionality reduction and metric learning based on our MMRA criterion. However, solving such a non-smooth non-convex max-min ratio problem is challenging. As an important theoretical contribution in this paper, we systematically derive an alternative iterative algorithm based on a general max-min ratio optimization framework to solve a general max-min ratio problem with rigorous proofs of convergence. More importantly, we also present another solver based on bisection search strategy to solve the SDP problem efficiently. To evaluate the effectiveness of proposed methods, we conduct extensive pattern classification and image retrieval experiments on several artificial datasets and real-world ScRNA-seq datasets, and experimental results demonstrate the effectiveness of proposed methods.

Minimizing Nearest Neighbor Classification Error for Nonparametric Dimension Reduction.

Neighborhood MinMax Projections

Nonparametric Dimension Reduction Via Maximizing Pairwise Separation Probability.

Simultaneously Learning Neighborship and Projection Matrix for Supervised Dimensionality Reduction

On high-dimensional modifications of the nearest neighbor classifier

Dimension Reduction by Minimum Error Minimax Probability Machine

Distance metric learning by minimal distance maximization

NCLDR: Nearest-Neighbor Contrastive Learning with Dual Correlation Loss for Dimensionality Reduction

Discriminative Dimension Reduction Via Maximin Separation Probability Analysis

Max-Min Distance Analysis by Using Sequential SDP Relaxation for Dimension Reduction

Efficiently Learning a Distance Metric for Large Margin Nearest Neighbor Classification.

Worst-Case Discriminative Feature Learning via Max-Min Ratio Analysis

Nonparametric Discriminant Multi-Manifold Learning

Two-dimensional Nearest Neighbor Discriminant Analysis

Nonparametric Discriminant Multi-Manifold Learning for Dimensionality Reduction

Dimension Reduction by Maximizing Pairwise Discriminations.

Dimension reduction for covariates in network data

Optimal Dimensionality of Metric Space for Classification

Risk-based Adaptive Metric Learning for Nearest Neighbour Classification

Reverse nearest neighbors Bhattacharyya bound linear discriminant analysis for multimodal classification