Abstract:Classification with indefinite similarities has attracted attention in the machine learning community. This is partly due to the fact that many similarity functions that arise in practice are not symmetric positive semidefinite, i.e. the Mercer condition is not satisfied, or the Mercer condition is difficult to verify. Examples of such indefinite similarities in machine learning applications are ample including, for instance, the BLAST similarity score between protein sequences, human-judged similarities between concepts and words, and the tangent distance or the shape matching distance in computer vision. Nevertheless, previous works on classification with indefinite similarities are not fully satisfactory. They have either introduced sources of inconsistency in handling past and future examples using kernel approximation, settled for local-minimum solutions using non-convex optimization, or produced non-sparse solutions by learning in Krein spaces. Despite the large volume of research devoted to this subject lately, we demonstrate in this paper how an old idea, namely the 1-norm support vector machine (SVM) proposed more than 15 years ago, has several advantages over more recent work. In particular, the 1-norm SVM method is conceptually simpler, which makes it easier to implement and maintain. It is competitive, if not superior to, all other methods in terms of predictive accuracy. Moreover, it produces solutions that are often sparser than more recent methods by several orders of magnitude. In addition, we provide various theoretical justifications by relating 1-norm SVM to well-established learning algorithms such as neural networks, SVM, and nearest neighbor classifiers. Finally, we conduct a thorough experimental evaluation, which reveals that the evidence in favor of 1-norm SVM is statistically significant.

Learning Similarity Metric with SVM.

Learning Similarity Function between Objects in Heterogeneous Spaces

A metric learning perspective of SVM: on the relation of SVM and LMNN

Distance Metric Learning for Kernel Machines

Learning Distance Metric for Support Vector Machine: A Multiple Kernel Learning Approach.

Similarity Measure Learning in Closed-Form Solution for Image Classification.

Learning Non-Metric Partial Similarity Based on Maximal Margin Criterion

SLMOML: Online Metric Learning with Global Convergence

Metricfusion: Generalized Metric Swarm Learning For Similarity Measure

Scalable Large-Margin Mahalanobis Distance Metric Learning

Distance Metric Learning Via Iterated Support Vector Machines

Semi-supervised LDA Based Method for Similarity Distance Metric Learning.

Iterated Support Vector Machines for Distance Metric Learning

Online Similarity Learning for Visual Tracking

Metric learning with geometric mean for similarities measurement

Metric Learning Via Penalized Optimization.

A Coupled Similarity Kernel for Pairwise Support Vector Machine

Large Margin Classification with Indefinite Similarities

A dissimilarity based kernel space for support vector machines

Multi-Level Metric Learning Via Smoothed Wasserstein Distance

Deep Metric Learning using Similarities from Nonlinear Rank Approximations