Abstract:The probabilistic classification vector machine (PCVM) synthesizes the advantages of both the support vector machine and the relevant vector machine, delivering a sparse Bayesian solution to classification problems. However, the PCVM is currently only applicable to binary cases. Extending the PCVM to multi-class cases via heuristic voting strategies such as one-vs-rest or one-vs-one often results in a dilemma where classifiers make contradictory predictions, and those strategies might lose the benefits of probabilistic outputs. To overcome this problem, we extend the PCVM and propose a multi-class probabilistic classification vector machine (mPCVM). Two learning algorithms, i.e., one top-down algorithm and one bottom-up algorithm, have been implemented in the mPCVM. The top-down algorithm obtains the maximum a posteriori (MAP) point estimates of the parameters based on an expectation-maximization algorithm, and the bottom-up algorithm is an incremental paradigm by maximizing the marginal likelihood. The superior performance of the mPCVMs, especially when the investigated problem has a large number of classes, is extensively evaluated on synthetic and benchmark data sets.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the deficiencies of existing classification methods in multi - classification tasks, especially the limitations of Support Vector Machines (SVM) and Relevance Vector Machines (RVM) when dealing with multi - classification problems. Specifically: 1. **Non - probability output**: SVM does not directly provide probability output, which is a drawback in some application scenarios. 2. **Computational complexity**: The computational complexity of SVM is linearly related to the size of the training set, resulting in a decline in performance when training on large - scale data sets. 3. **Instability**: Although RVM is superior to SVM in some aspects, the consistency between its weight sign and class label cannot be guaranteed, resulting in instability in classification problems and sensitivity to kernel parameters. 4. **Multi - classification extension problem**: Existing multi - classification strategies (such as one - against - all and one - against - one) are prone to generate contradictory predictions among classifiers when extending binary classifiers to multi - classifiers, and these strategies cannot directly generate probability output. To overcome these problems, the author proposes the Multi - class Probabilistic Classification Vector Machine (mPCVM), which combines the advantages of Support Vector Machines and Relevance Vector Machines while ensuring the consistency between weight sign and class label. mPCVM ensures the sparsity and computational efficiency of the model by introducing a truncated Gaussian prior to constrain the weights. The paper also proposes two learning algorithms: a top - down algorithm (mPCVM 1) and a bottom - up algorithm (mPCVM 2), which optimize the model parameters by maximum a posteriori estimation and maximizing the marginal likelihood respectively. Through experiments on synthetic data sets and benchmark data sets, the superior performance of mPCVM in multi - classification tasks has been verified, especially in cases where the number of classes is large and the classes are unbalanced.

Probabilistic Classification Vector Machine for Multi-Class Classification

Multiclass Probabilistic Classification Vector Machine

Probabilistic Classification Vector Machines for Multiclass Classification

Probabilistic Classification Vector Machines.

A Multi-Class Large Margin Classifier

Efficient Probabilistic Classification Vector Machine with Incremental Basis Function Selection.

MPC: Multi-view Probabilistic Clustering

Probabilistic Feature Selection and Classification Vector Machine

Linear Multi-Class Classification Support Vector Machine

Probabilistic Methods in Multi-Class Brain-Computer Interface

Multi-sphere Classification Algorithm Based on 1-SVM

Support Vector Machine-Based Multi-Model Predictive Control

Multi-classification Algorithm Based on Truncated Gaussian Prior and Variational Bayesian

Support Vector Machines for Multi-class Classification Problems

Multi-class Classification Method Based on SVM Probability Output and Evidence Theory

Multi-Class Support Vector Machine Via Maximizing Multi-Class Margins

Comparative Analysis For Probability Modeling Of Multi-Class Svm

Multi-class Support Vector Machine with Maximizing Minimum Margin

A New Support Vector Machine for Multi-class Classification

Incremental Probabilistic Classification Vector Machine with Linear Costs

DCSVM: fast multi-class classification using support vector machines