Abstract:Metric learning, the task of learning a good distance metric, is a key problem in machine learning with ample applications. This paper introduces a novel framework for nonlinear metric learning, called kernel density metric learning (KDML), which is easy to use and provides nonlinear, probability-based distance measures. KDML constructs a direct nonlinear mapping from the original input space into a feature space based on kernel density estimation. The nonlinear mapping in KDML embodies established distance measures between probability density functions, and leads to accurate classification on datasets for which existing linear metric learning methods would fail. It addresses the severe challenge to distance-based classifiers when features are from heterogeneous domains and, as a result, the Euclidean or Mahalanobis distance between original feature vectors is not meaningful. We also propose two ways to determine the kernel bandwidths, including an adaptive local scaling approach and an integrated optimization algorithm that learns the Mahalanobis matrix and kernel bandwidths together. KDML is a general framework that can be combined with any existing metric learning algorithm. As concrete examples, we combine KDML with two leading metric learning algorithms, large margin nearest neighbors (LMNN) and neighborhood component analysis (NCA). KDML can naturally handle not only numerical features, but also categorical ones, which is rarely found in previous metric learning algorithms. Extensive experimental results on various datasets show that KDML significantly improves existing metric learning algorithms in terms of classification accuracy.

Learning Metrics via Discriminant Kernels and Multidimensional Scaling: Toward Expected Euclidean Representation

Distance-Based Classifier Via the Kernel Trick.

Metric learning by discriminant neighborhood embedding

Learning Distance Metric for Support Vector Machine: A Multiple Kernel Learning Approach.

Metric Learning Via Penalized Optimization.

A Distributed Approach Toward Discriminative Distance Metric Learning

Towards Making High Dimensional Distance Metric Learning Practical

Optimal Dimensionality of Metric Space for Classification

From Point to Set: Extend the Learning of Distance Metrics

Data-Dependent Kernel Discriminant Analysis for Feature Extraction and Classification

A Kernel Classification Framework for Metric Learning.

Feature Space Distance Metric Learning for Discriminant Graph Embedding

Distance Metric Learning Via Iterated Support Vector Machines

Nonlinear Metric Learning with Kernel Density Estimation

Mixed-type Distance Shrinkage and Selection for Clustering via Kernel Metric Learning

Robust dimensionality reduction via feature space to feature space distance metric learning.

Kernel Density Metric Learning

Distance metric learning by minimal distance maximization

A Metric-learning based framework for Support Vector Machines and Multiple Kernel Learning

Learning A Mahalanobis Distance Metric for Data Clustering and Classification