Abstract:In many real-world applications, data is not collected as one batch, but sequentially over time, and often it is not possible or desirable to wait until the data is completely gathered before analyzing it. Thus, we propose a framework to sequentially update a maximum margin classifier by taking advantage of the Maximum Entropy Discrimination principle. Our maximum margin classifier allows for a kernel representation to represent large numbers of features and can also be regularized with respect to a smooth sub-manifold, allowing it to incorporate unlabeled observations. We compare the performance of our classifier to its non-sequential equivalents in both simulated and real datasets.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to effectively update the maximum margin classifier in partially - labeled data streams. Specifically, the author proposes a framework to sequentially update the maximum margin classifier by leveraging the Maximum Entropy Discrimination (MED) principle. This method allows the model to be continuously updated during the gradual collection of data, without having to wait for all data to be collected before performing one - time training. Additionally, this method can also handle unlabeled data, incorporating unlabeled observations into the model through regularization on smooth sub - manifolds. The paper also compares the performance of the proposed sequential classifier with its non - sequential version on both simulated and real - world datasets. ### Core Problems of the Paper 1. **Processing of Partially - Labeled Data**: In practical applications, data is often partially - labeled, meaning that only some samples have labels. How to effectively train a classifier in such a situation is a challenge. 2. **Sequential Model Update**: Data is usually collected gradually rather than all at once. How to sequentially update the classifier in the data stream so that it can promptly reflect the information of new data is also an important issue. 3. **Utilization of Unlabeled Data**: Although unlabeled data has no labels, it still contains useful information. How to effectively utilize this unlabeled data during the training process to improve the generalization ability of the model is another research focus. ### Solutions - **Maximum Entropy Discrimination (MED)**: Use the MED principle to construct a classifier and optimize model parameters by maximizing the posterior probability. - **Sequential Update Framework**: Propose a recursive Bayesian estimation method, represent the likelihood function as a log - linear model under a series of constraints, and weight it with Lagrange multipliers. - **Kernel Representation**: Use kernel functions to represent the feature space, thereby being able to handle high - dimensional features and a large number of feature variables. - **Smooth Sub - manifold Regularization**: Incorporate unlabeled data into the model through regularization on smooth sub - manifolds, improving the robustness and generalization ability of the model. ### Experimental Verification - **Simulated Data**: The performance of the sequential classifier on partially - labeled data is verified through generated simulated data. The results show that the sequential classifier can approach the performance of the classifier trained with the full batch after multiple updates. - **Real - World Data**: Experiments are conducted on the Isolet speech database. The results indicate that the performance of the sequential classifier on partially - labeled data is better than that of the classifier that ignores historical data and is close to that of the classifier re - trained with the full batch. In conclusion, this paper proposes an effective sequential maximum margin classifier that can continuously update the model in partially - labeled data streams while using unlabeled data to improve the performance of the model.

Sequential Maximum Margin Classifiers for Partially Labeled Data

Max-margin Based Bayesian Classifier.

Maximum Margin Partial Label Learning

A Multi-Class Large Margin Classifier

A Max-Margin Learning Algorithm with Additional Features

A Large Margin Classifier with Additional Features

A New Constrained Maximum Margin Approach to Discriminative Learning of Bayesian Classifiers

Maximum Margin Based Semi-supervised Spectral Kernel Learning.

Semi-Supervised Partial Label Learning via Confidence-Rated Margin Maximization

Maximum Margin Multi-Dimensional Classification

Learning Imbalanced Datasets with Maximum Margin Loss

Online Partial Label Learning

Learning from Multi-Dimensional Partial Labels.

Nonlinear Online Classification Algorithm with Probability Margin

Maximum entropy method for multiple-instance classification

A Sequential Model for Multi-Class Classification

Piecewise Large Margin Learning for Partially Annotated Sequences

Active Learning Via Sequential Design and Uncertainty Sampling

Online active classification via margin-based and feature-based label queries

Sequential Labeling with Structural SVM under Nondecomposable Losses.

Partial Label Learning with Discrimination Augmentation