Abstract:In this paper, we reveal that metric learning would suffer from serious inseparable problem if without informative sample mining. Since the inseparable samples are often mixed with hard samples, current informative sample mining strategies used to deal with inseparable problem may bring up some side-effects, such as instability of objective function, etc. To alleviate this problem, we propose a novel distance metric learning algorithm, named adaptive neighborhood metric learning (ANML). In ANML, we design two thresholds to adaptively identify the inseparable similar and dissimilar samples in the training procedure, thus inseparable sample removing and metric parameter learning are implemented in the same procedure. Due to the non-continuity of the proposed ANML, we develop an ingenious function, named \emph{log-exp mean function} to construct a continuous formulation to surrogate it, which can be efficiently solved by the gradient descent method. Similar to Triplet loss, ANML can be used to learn both the linear and deep embeddings. By analyzing the proposed method, we find it has some interesting properties. For example, when ANML is used to learn the linear embedding, current famous metric learning algorithms such as the large margin nearest neighbor (LMNN) and neighbourhood components analysis (NCA) are the special cases of the proposed ANML by setting the parameters different values. When it is used to learn deep features, the state-of-the-art deep metric learning algorithms such as Triplet loss, Lifted structure loss, and Multi-similarity loss become the special cases of ANML. Furthermore, the \emph{log-exp mean function} proposed in our method gives a new perspective to review the deep metric learning methods such as Prox-NCA and N-pairs loss. At last, promising experimental results demonstrate the effectiveness of the proposed method.

Non-linear neighborhood component analysis based on constructive neural networks

Image Feature Optimization Based on Nonlinear Dimensionality Reduction

Unsupervised Neighborhood Component Analysis for Clustering.

Fast Neighborhood Component Analysis.

Convolutional Nonlinear Neighbourhood Components Analysis for Time Series Classification.

Neighborhood and Graph Constructions using Non-Negative Kernel Regression

Nonlinear Metric Learning with Kernel Density Estimation

Discovering the Latent Similarities of the KNN Graph by Metric Transformation

Nearest Hit-Misses Component Analysis For Supervised Metric Learning

Adaptive neighborhood Metric learning

Neighborhood MinMax Projections

Adaptive Nearest Neighbor: A General Framework for Distance Metric Learning

Metric Learning Via Penalized Optimization.

Learning nonlinear manifolds based on mixtures of localized linear manifolds under a self-organizing framework

A Graph-Based Semi-Supervised K Nearest-Neighbor Method for Nonlinear Manifold Distributed Data Classification.

Probabilistic Nearest Neighbors Based Locality Preserving Projections for Unsupervised Metric Learning

Linear Neighborhood Propagation and Its Applications

Modeling Nonlinear Manifolds with Mixtures of Localized Principal Subspaces under a Self-Organizing Framework

A Modified Constructive Neural Networks And Its Application For Large-Scale Data Mining

Kernel Density Metric Learning

Nonlinear Local Metric Learning for Person Re-identification.