Abstract:Distance metric learning (DML) aims to find a suitable measure to compute a distance between instances. Facilitated by side information, the learned metric can often improve the performance of similarity or distance based methods such as kNN. Theoretical analyses of DML focus on the learning effectiveness for squared Mahalanobis distance. Specifically, whether the Mahalanobis metric learned from the empirically sampled pairwise constraints is in accordance with the optimal metric optimized over the paired samples generated from the true distribution, and the sample complexity of this process. The excess risk could measure the quality of the generalization, i.e., the gap between the expected objective of empirical metric learned from a regularized objective with convex loss function and the one with the optimal metric. Given N training examples, existing analyses over this non-i.i.d. learning problem have proved the excess risk of DML converges to zero at a rate of \({\mathcal {O}}\left( \frac{1}{\sqrt{N}}\right) \). In this paper, we obtain a faster convergence rate of DML, \({\mathcal {O}}\left( \frac{1}{N}\right) \), when learning the distance metric with a smooth loss function and a strongly convex objective. In addition, when the problem is relatively easy, and the number of training samples is large enough, this rate can be further improved to \({\mathcal {O}}\left( \frac{1}{N^2}\right) \). Synthetic experiments validate that DML can achieve the specified faster generalization rate, and results under various settings help explore the theoretical properties of DML a lot.

Smooth Approximation Method for Non-Smooth Empirical Risk Minimization Based Distance Metric Learning

Constrained Empirical Risk Minimization Framework for Distance Metric Learning.

Learning a Distance Metric by Empirical Loss Minimization

Distance Metric Learning Via Iterated Support Vector Machines

Regularized Large Margin Distance Metric Learning.

Fast Generalization Rates for Distance Metric Learning

Metric Learning Via Penalized Optimization.

Robust Distance Metric Learning Via Simultaneous L1-Norm Minimization and Maximization.

Iterated Support Vector Machines for Distance Metric Learning

Multi-Level Metric Learning Via Smoothed Wasserstein Distance

Efficient Distance Metric Learning by Adaptive Sampling and Mini-Batch Stochastic Gradient Descent (SGD)

Mixture Correntropy-Based Robust Distance Metric Learning for Classification

Towards Making High Dimensional Distance Metric Learning Practical

Low-Rank Robust Online Distance/Similarity Learning based on the Rescaled Hinge Loss

Distance metric learning by minimal distance maximization

Error Risk Minimization

Person Re-identification by Smooth Metric Learning.

Error Density-dependent Empirical Risk Minimization

Empirical Risk Minimization for Metric Learning Using Privileged Information

On the Performance of Empirical Risk Minimization with Smoothed Data

Metric Learning with Adaptive Density Discrimination