Abstract:One of the main challenges for feature representation in deep learning-based classification is the design of appropriate loss functions that exhibit strong discriminative power. The classical softmax loss does not explicitly encourage discriminative learning of features. A popular direction of research is to incorporate margins in well-established losses in order to enforce extra intra-class compactness and inter-class separability, which, however, were developed through heuristic means, as opposed to rigorous mathematical principles. In this work, we attempt to address this limitation by formulating the principled optimization objective as learning towards the largest margins. Specifically, we firstly propose to employ the class margin as the measure of inter-class separability, and the sample margin as the measure of intra-class compactness. Accordingly, to encourage discriminative representation of features, the loss function should promote the largest possible margins for both classes and samples. Furthermore, we derive a generalized margin softmax loss to draw general conclusions for the existing margin-based losses. Not only does this principled framework offer new perspectives to understand and interpret existing margin-based losses, but it also provides new insights that can guide the design of new tools, including \textit{sample margin regularization} and \textit{largest margin softmax loss} for class balanced cases, and \textit{zero centroid regularization} for class imbalanced cases. Experimental results demonstrate the effectiveness of our strategy for multiple tasks including visual classification, imbalanced classification, person re-identification, and face verification.

Large-margin Nearest Neighbor Classifiers Via Sample Weight Learning.

Efficiently Learning a Distance Metric for Large Margin Nearest Neighbor Classification.

The Nearest Neighbor Algorithm of Local Probability Centers

Learning Towards The Largest Margins.

On high-dimensional modifications of the nearest neighbor classifier

Improving Nearest Neighbor Classification with Cam Weighted Distance.

Under-bagging Nearest Neighbors for Imbalanced Classification

Parameter Free Large Margin Nearest Neighbor For Distance Metric Learning

Scalable Large-Margin Mahalanobis Distance Metric Learning

Nearest neighbor classification using cam weighted distance

A Novel Pseudo Nearest Neighbor Classification Method Using Local Harmonic Mean Distance

Nearest Feature Line Classifier Based on Collaborative Representation with Nearest Neighbour Search Algorithm

Towards Optimal Attribute Weight Setting for 1-Nearest Neighbor Learning Algorithms

CLE_LMNN

Distributionally Robust Weighted $k$-Nearest Neighbors

Large Margin Discriminative Loss for Classification

Sample Weighting: an Inherent Approach for Outlier Suppressing Discriminant Analysis

Improving classifier decision boundaries using nearest neighbors

K-Ns: A Classifier by the Distance to the Nearest Subspace.

Risk-based Adaptive Metric Learning for Nearest Neighbour Classification