Abstract:Distance metric learning is of fundamental interest in machine learning because the employed distance metric can significantly affect the performance of many learning methods. Quadratic Mahalanobis metric learning is a popular approach to the problem, but typically requires solving a semidefinite programming (SDP) problem, which is computationally expensive. The worst case complexity of solving an SDP problem involving a matrix variable of size D×D with O(D) linear constraints is about O(D(6.5)) using interior-point methods, where D is the dimension of the input data. Thus, the interior-point methods only practically solve problems exhibiting less than a few thousand variables. Because the number of variables is D(D+1)/2, this implies a limit upon the size of problem that can practically be solved around a few hundred dimensions. The complexity of the popular quadratic Mahalanobis metric learning approach thus limits the size of problem to which metric learning can be applied. Here, we propose a significantly more efficient and scalable approach to the metric learning problem based on the Lagrange dual formulation of the problem. The proposed formulation is much simpler to implement, and therefore allows much larger Mahalanobis metric learning problems to be solved. The time complexity of the proposed method is roughly O(D(3)), which is significantly lower than that of the SDP approach. Experiments on a variety of data sets demonstrate that the proposed method achieves an accuracy comparable with the state of the art, but is applicable to significantly larger problems. We also show that the proposed method can be applied to solve more general Frobenius norm regularized SDP problems approximately.

A Scalable Dual Approach to Semidefinite Metric Learning

Efficient Dual Approach to Distance Metric Learning

A scalable algorithm for learning a mahalanobis distance metric

Scalable Large-Margin Mahalanobis Distance Metric Learning

Positive Semidefinite Metric Learning Using Boosting-Like Algorithms

Distributed Semi-Supervised Metric Learning.

Positive Semidefinite Metric Learning with Boosting

Towards Scalable Semidefinite Programming: Optimal Metric ADMM with A Worst-case Performance Guarantee

Semi-supervised Sparse Metric Learning Using Alternating Linearization Optimization

Towards Making High Dimensional Distance Metric Learning Practical

A Distributed Approach Toward Discriminative Distance Metric Learning

Efficient Distance Metric Learning by Adaptive Sampling and Mini-Batch Stochastic Gradient Descent (SGD)

A Riemannian Primal-Dual Algorithm Based On Proximal Operator And Its Application In Metric Learning

An Efficient Sparse Metric Learning in High-Dimensional Space Via L1-Penalized Log-Determinant Regularization

Metric Learning to Accelerate Convergence of Operator Splitting Methods for Differentiable Parametric Programming

Efficient Stochastic Optimization for Low-Rank Distance Metric Learning.

Efficiently Learning a Distance Metric for Large Margin Nearest Neighbor Classification.

Constrained Metric Learning Via Distance Gap Maximization

Metric Learning Via Penalized Optimization.

Revisiting Metric Learning for SPD Matrix Based Visual Representation.

The Translation Invariant Solution To Quadratic Metric Learning