Abstract:For many machine learning algorithms such as k-nearest neighbor ( k-NN) classifiers and k-means clustering, often their success heavily depends on the metric used to calculate distances between different data points. An effective solution for defining such a metric is to learn it from a set of labeled training samples. In this work, we propose a fast and scalable algorithm to learn a Mahalanobis distance metric. The Mahalanobis metric can be viewed as the Euclidean distance metric on the input data that have been linearly transformed. By employing the principle of margin maximization to achieve better generalization performances, this algorithm formulates the metric learning as a convex optimization problem and a positive semidefinite (p.s.d.) matrix is the unknown variable. Based on an important theorem that a p.s.d. trace-one matrix can always be represented as a convex combination of multiple rank-one matrices, our algorithm accommodates any differentiable loss function and solves the resulting optimization problem using a specialized gradient descent procedure. During the course of optimization, the proposed algorithm maintains the positive semidefiniteness of the matrix variable that is essential for a Mahalanobis metric. Compared with conventional methods like standard interior-point algorithms or the special solver used in large margin nearest neighbor , our algorithm is much more efficient and has a better performance in scalability. Experiments on benchmark data sets suggest that, compared with state-of-the-art metric learning algorithms, our algorithm can achieve a comparable classification accuracy with reduced computational complexity.

Regression-Based Metric Learning

Metric Learning Via Penalized Optimization.

Kernel Regression with Sparse Metric Learning

Beyond Similar and Dissimilar Relations : A Kernel Regression Formulation for Metric Learning

A Novel Metric Learning Framework by Exploiting Global and Local Information

Low-Rank Similarity Metric Learning In High Dimensions

Adaptive Metric Learning with the Low Rank Constraint

Improved Least Squares Support Vector Machine Based on Metric Learning

Scalable Large-Margin Mahalanobis Distance Metric Learning

Multi-Task Sparse Regression Metric Learning For Heterogeneous Classification

Mixture Correntropy-Based Robust Distance Metric Learning for Classification

Relationship constraint deep metric learning

Constrained Empirical Risk Minimization Framework for Distance Metric Learning.

Metric Learning from Imbalanced Data

Distance Metric Learning for Kernel Machines

Distance Metric Learning Via Iterated Support Vector Machines

Low rank metric learning for social image retrieval.

Multi-Level Metric Learning Via Smoothed Wasserstein Distance

Deep Multiple Metric Learning for Time Series Classification

Deep Metric Learning-Based Semi-Supervised Regression with Alternate Learning

Exploiting Inconsistency Problem in Multi-label Classification Via Metric Learning