Abstract:Matrices are appropriate for representing a wealth of data with complex structures such as images and electroencephalogram data (EEG). To learn a classifier dealing with these matrix data, the structure information of the feature matrix is useful. In this paper, we focus on the regularized matrix classifiers whose input samples and weight parameters are both in the form of a matrix. Some existing approaches assume that the weight matrix has a low-rank structure and then utilize the popular nuclear norm of the weight matrix as a regularization term. However, the optimization methods for these matrix classifiers often involve numbers of expensive singular value decomposition (SVD) operations, which prevents scaling beyond moderate matrix sizes. To reduce the time complexity, we propose a novel learning algorithm called Atom Decomposition Based Subgradient Descent (ADBSD), which solves the optimization problem for the matrix classifier whose objective function is the combination of the Frobenius matrix norm and nuclear norm of the weight matrix along with the hinge loss function. Our ADBSD is an iterative scheme which selects the most informative rank-one matrices from the subgradient of the objective function in each iteration. We consider using the atom decomposition based methods to minimize nuclear norm because they mainly rely on the computation of top singular vector pair which leads to great advantages on efficiency. We empirically evaluate the performance of the proposed algorithm ADBSD on both synthetic and real-world datasets. Results show that our approach is more efficient and robust than the state-of-the-art methods.

Gradient-based Quadratic Multiform Separation

A Multi-Class Large Margin Classifier

A new improved support vector machine: QGA-SVM

Kernel-Free Quadratic Surface Regression for Multi-Class Classification

Nonlinear kernel-free quadratic hyper-surface support vector machine with 0-1 loss function

Random Matrix Analysis to Balance between Supervised and Unsupervised Learning under the Low Density Separation Assumption

Gradient-Free optimization algorithm for single-qubit quantum classifier

An Efficient and Versatile Variational Method for High-Dimensional Data Classification

Classifier-guided Gradient Modulation for Enhanced Multimodal Learning

Asymmetric and robust loss function driven least squares support vector machine

Multinomial Logistic Regression Algorithms via Quadratic Gradient

Adjustable entropy funtion method for misclassification minimization problems

Kernel-Free Universum Quadratic Surface Twin Support Vector Machines for Imbalanced Data

Homocentric quadratic surfaces and maximum margin approach for imbalanced data classification

Variational quantum state discriminator for supervised machine learning

Cross Distance Minimization for Solving the Nearest Point Problem Based on Scaled Convex Hull

English OCR Multi-template Classification Based on Modified Quadratic Discriminant Function

DCSVM: fast multi-class classification using support vector machines

Variational Quantum Linear Solver enhanced Quantum Support Vector Machine

Atom Decomposition Based Subgradient Descent for Matrix Classification

Discriminative Dimension Reduction Via Maximin Separation Probability Analysis