Abstract:For many machine learning algorithms such as k-nearest neighbor ( k-NN) classifiers and k-means clustering, often their success heavily depends on the metric used to calculate distances between different data points. An effective solution for defining such a metric is to learn it from a set of labeled training samples. In this work, we propose a fast and scalable algorithm to learn a Mahalanobis distance metric. The Mahalanobis metric can be viewed as the Euclidean distance metric on the input data that have been linearly transformed. By employing the principle of margin maximization to achieve better generalization performances, this algorithm formulates the metric learning as a convex optimization problem and a positive semidefinite (p.s.d.) matrix is the unknown variable. Based on an important theorem that a p.s.d. trace-one matrix can always be represented as a convex combination of multiple rank-one matrices, our algorithm accommodates any differentiable loss function and solves the resulting optimization problem using a specialized gradient descent procedure. During the course of optimization, the proposed algorithm maintains the positive semidefiniteness of the matrix variable that is essential for a Mahalanobis metric. Compared with conventional methods like standard interior-point algorithms or the special solver used in large margin nearest neighbor , our algorithm is much more efficient and has a better performance in scalability. Experiments on benchmark data sets suggest that, compared with state-of-the-art metric learning algorithms, our algorithm can achieve a comparable classification accuracy with reduced computational complexity.

Two Dimensional Large Margin Nearest Neighbor for Matrix Classification.

Data classification algorithm based on two-step matrix projection

Efficiently Learning a Distance Metric for Large Margin Nearest Neighbor Classification.

Hierarchical Large Margin Nearest Neighbor Classification

Rank-k 2-D Multinomial Logistic Regression for Matrix Data Classification.

Hierarchical Distance Metric Learning for Large Margin Nearest Neighbor Classification.

Large margin nearest local mean classifier

Scalable Large-Margin Mahalanobis Distance Metric Learning

CLE_LMNN

Multi-distance Support Matrix Machines.

Adaptive Nearest Neighbor: A General Framework for Distance Metric Learning

Multiple Rank Multi-Linear SVM for Matrix Data Classification.

A depth-based nearest neighbor algorithm for high-dimensional data classification

A Portmanteau Local Feature Discrimination Approach to the Classification with High-dimensional Matrix-variate Data

Parameter Free Large Margin Nearest Neighbor For Distance Metric Learning

Two Dimensional Discriminative Projection Based on Nearest Orthogonal Matrix and its Application to Face Recognition

Two Dimensional Maximum Margin Criterion

A Novel Two-Level Nearest Neighbor Classification Algorithm Using an Adaptive Distance Metric

Learning K for Knn Classification

Two-dimensional Nearest Neighbor Discriminant Analysis

Md-knn: an Instance-based Approach for Multi-Dimensional Classification