Abstract:Feature selection can help to reduce data redundancy and improve algorithm performance in actual tasks. Most of the embedded feature selection models are constructed based on square loss and hinge loss. However, these models based on the square loss cannot directly evaluate the discriminability of the sam-ples in the feature subspace, and these methods based on the hinge loss are difficult to solve due to their complex objective functions. To deal with these problems, a Feature Selection method with Multi-class Logistic Regression (FSMLR) is proposed in this paper. Firstly, we construct a linear function to measure the difference between the distance from samples to their regression hyperplane and the distance from these samples to regression hyperplanes of other classes, which could be used to strengthen the discrim-inant property of the embedded model. Then, we design a re-weighting matrix with a 2,0-norm pound sparse condition as well as a discrete condition, which is used to select features in the subspace. Considering that it is difficult to solve the re-weighting matrix with the discrete and sparse conditions in an optimization problem, we relax these two conditions and present a feature selection model via a re-weighted multi -class logistic regression with the two relaxed constraints. Finally, we add the F-norm regularization in our model to avoid overfitting, and its unconstrained equivalent transformation with 2 pound,p-norm regular-ization is derived to explore the function of the re-weighting matrix. The gradient descent algorithm could be used to solve the FSMLR. Especially, when the regularization term in the equivalence problem is set to 2,1-norm pound, the global optimal solution can be obtained. Extensive experiments on multiple public data sets prove that FSMLR outperforms other competitors.(c) 2023 Elsevier B.V. All rights reserved.

Feature Selection Based on Loss-Margin of Nearest Neighbor Classification

Feature Selection Based on Dependency Margin

Feature Evaluation and Selection Based on Neighborhood Soft Margin

Feature Selection Method Based on Margin Loss and L1-norm Regulation

Feature selection using nearest attributes

Feature Selection Via Maximizing Neighborhood Soft Margin.

A Constrained Feature Selection Approach Based on Feature Clustering and Hypothesis Margin Maximization

Improving Performance of the k-Nearest Neighbor Classifier by Combining Feature Selection with Feature Weighting.

Face Feature Selection and Recognition Based on Different Types of Margin

Trace Ratio Criterion Based Large Margin Subspace Learning for Feature Selection.

Unsupervised Feature Analysis with Class Margin Optimization

Large-margin Nearest Neighbor Classifiers Via Sample Weight Learning.

Feature Selection Based on Classes Margin

Large Margin Feature Weighting Method Via Linear Programming

A Novel Feature Selection Algorithm Based on LVQ Hypothesis Margin

Efficient Linear Feature Extraction Based on Large Margin Nearest Neighbor.

Feature selection with multi-class logistic regression

Selecting Discrete and Continuous Features Based on Neighborhood Decision Error Minimization

A feature selection approach to estimate discrimination capability of feature subset category

Feature Selection for Imbalanced Data Based on Neighborhood Rough Sets.

Feature Weighting Algorithm Based on Margin and Linear Programming