A regularized correntropy framework for robust pattern recognition

HeRan,ZhengWei-Shi,HuBao-Gang,KongXiang-Wei

IF: 3.278

2011-01-01

Neural Computation

Abstract:This letter proposes a new multiple linear regression model using regularized correntropy for robust pattern recognition. First, we motivate the use of correntropy to improve the robustness of the ...

What problem does this paper attempt to address?

A Regularized Correntropy Framework for Robust Pattern Recognition

Ran He,Wei-Shi Zheng,Bao-Gang Hu,Xiang-Wei Kong

DOI: https://doi.org/10.1162/neco_a_00155

IF: 3.278

2011-01-01

Neural Computation

Abstract:This letter proposes a new multiple linear regression model using regularized correntropy for robust pattern recognition. First, we motivate the use of correntropy to improve the robustness of the classical mean square error (MSE) criterion that is sensitive to outliers. Then an l1 regularization scheme is imposed on the correntropy to learn robust and sparse representations. Based on the half-quadratic optimization technique, we propose a novel algorithm to solve the nonlinear optimization problem. Second, we develop a new correntropy-based classifier based on the learned regularization scheme for robust object recognition. Extensive experiments over several applications confirm that the correntropy-based l1 regularization can improve recognition accuracy and receiver operator characteristic curves under noise corruption and occlusion.
Robust spectral regression for face recognition

Yanqing Guo,Ran He,Wei-Shi Zheng,Xiangwei Kong,Zhaofeng He

DOI: https://doi.org/10.1016/j.neucom.2013.02.011

IF: 6

2013-01-01

Neurocomputing

Abstract:Spectral regression has been an efficient and powerful tool for face recognition. However, spectral regression is sensitive to the errors incurred by inaccurate annotation and occlusion. This paper studies robust spectral regression based discriminant subspace learning from correntropy and spatially smooth structure of facial subspace. First, we formulate the robust discriminant subspace learning problem as a maximum correntropy problem, which finds the most correlation solution between spectral targets and predictions. Second, total variation (TV) regularization is imposed on the correntropy objective to learn a spatially smooth face structure. Lastly, based on the additive form of half-quadratic optimization, we cast the maximum correntropy problem into a compound regularization model, which can be efficiently optimized via an accelerated proximal gradient algorithm. Compared with iteratively reweighted least squares based methods, the proposed method can not only improve recognition rates but also reduce computational cost. Experimental results on a couple of face recognition datasets demonstrate the robustness and effectiveness of our method against inaccurate annotation and occlusion.
Learning with the Maximum Correntropy Criterion Induced Losses for Regression.

Yunlong Feng,Xiaolin Huang,Lei Shi,Yuning Yang,Johan A. K. Suykens

DOI: https://doi.org/10.5555/2789272.2886783

2015-01-01

Journal of Machine Learning Research

Abstract:Within the statistical learning framework, this paper studies the regression model associated with the correntropy induced losses. The correntropy, as a similarity measure, has been frequently employed in signal processing and pattern recognition. Motivated by its empirical successes, this paper aims at presenting some theoretical understanding towards the maximum correntropy criterion in regression problems. Our focus in this paper is twofold: first, we are concerned with the connections between the regression model associated with the correntropy induced loss and the least squares regression model. Second, we study its convergence property. A learning theory analysis which is centered around the above two aspects is conducted. From our analysis, we see that the scale parameter in the loss function balances the convergence rates of the regression model and its robustness. We then make some efforts to sketch a general view on robust loss functions when being applied into the learning for regression problems. Numerical experiments are also implemented to verify the effectiveness of the model.
Error Analysis on Regularized Regression Based on the Maximum Correntropy Criterion

Bingzheng Li,Zhengzhan Dai

DOI: https://doi.org/10.3934/mfc.2020003

2020-01-01

Mathematical Foundations of Computing

Abstract:This paper aims at the regularized learning algorithm for regression associated with the correntropy induced losses in reproducing kernel Hilbert spaces. The main target is the error analysis for the regression problem in learning theory based on the maximum correntropy. Explicit learning rates are provided. From our analysis, when choosing a suitable parameter of the loss function, we obtain satisfactory learning rates. The rates depend on the regularization error and the covering numbers of the reproducing kernel Hilbert space.
Robust non-negative matrix factorization with multiple correntropy-induced hypergraph regularizer

Jie Liu,Jiawei Wang,Kai Zhang,Jianshe Zhou,Yongdong Zhang

DOI: https://doi.org/10.1016/j.sigpro.2020.107569

IF: 4.729

2020-09-01

Signal Processing

Abstract:<p>Non-negative matrix factorization (NMF) is a popular learning tool, which has widely used in computer vision and image processing. Many variants and extensions of NMF have been proposed, where the manifold regularized NMF methods have achieved promising performance due to the preservation of the geometric structures of the data. However, for many applications, the data is usually contaminated by complex noise. The noise leads the data to deviate from the intrinsic manifold, resulting in the degenerate performance. To make the NMF methods reflect the underlying manifold structure well, we propose a novel robust non-negative matrix factorization model. Our model proposes a novel ensemble manifold regularizer, which combines multiple robust hypergraphs to estimate the underlying manifold. Specificially, the correntropy-induced hypergraph is used as the initial manifold estimation. By incorporating the proposed regularizer into the original NMF framework, two novel manifold regularized NMF methods are proposed. The clustering results on the noisy image datasets demonstrate that our model is effective, which achieves the state-of-the-art performance.</p>

engineering, electrical & electronic
Robust Regularized Kernel Regression.

Jianke Zhu,Steven C. H. Hoi,Michael Rung-Tsong Lyu

DOI: https://doi.org/10.1109/tsmcb.2008.927279

2009-01-01

Abstract:Robust regression techniques are critical to fitting data with noise in real-world applications. Most previous work of robust kernel regression is usually formulated into a dual form, which is then solved by some quadratic program solver consequently. In this correspondence, we propose a new formulation for robust regularized kernel regression under the theoretical framework of regularization networks and then tackle the optimization problem directly in the primal. We show that the primal and dual approaches are equivalent to achieving similar regression performance, but the primal formulation is more efficient and easier to be implemented than the dual one. Different from previous work, our approach also optimizes the bias term. In addition, we show that the proposed solution can be easily extended to other noise-reliable loss function, including the Huber-epsiv insensitive loss function. Finally, we conduct a set of experiments on both artificial and real data sets, in which promising results show that the proposed method is effective and more efficient than traditional approaches.
Correntropy-Induced Discriminative Nonnegative Sparse Coding for Robust Palmprint Recognition

Kunlei Jing,Xinman Zhang,Guokun Song

DOI: https://doi.org/10.3390/s20154250

2020-07-30

Abstract:Palmprint recognition has been widely studied for security applications. However, there is a lack of in-depth investigations on robust palmprint recognition. Regression analysis being intuitively interpretable on robustness design inspires us to propose a correntropy-induced discriminative nonnegative sparse coding method for robust palmprint recognition. Specifically, we combine the correntropy metric and l1-norm to present a powerful error estimator that gains flexibility and robustness to various contaminations by cooperatively detecting and correcting errors. Furthermore, we equip the error estimator with a tailored discriminative nonnegative sparse regularizer to extract significant nonnegative features. We manage to explore an analytical optimization approach regarding this unified scheme and figure out a novel efficient method to address the challenging non-negative constraint. Finally, the proposed coding method is extended for robust multispectral palmprint recognition. Namely, we develop a constrained particle swarm optimizer to search for the feasible parameters to fuse the extracted robust features of different spectrums. Extensive experimental results on both contactless and contact-based multispectral palmprint databases verify the flexibility and robustness of our methods.
Correntropy Induced L2 Graph for Robust Subspace Clustering

Canyi Lu,Jinhui Tang,Min Lin,Liang Lin,Shuicheng Yan,Zhouchen Lin

DOI: https://doi.org/10.1109/iccv.2013.226

2013-01-01

Abstract:In this paper, we study the robust subspace clustering problem, which aims to cluster the given possibly noisy data points into their underlying subspaces. A large pool of previous subspace clustering methods focus on the graph construction by different regularization of the representation coefficient. We instead focus on the robustness of the model to non-Gaussian noises. We propose a new robust clustering method by using the correntropy induced metric, which is robust for handling the non-Gaussian and impulsive noises. Also we further extend the method for handling the data with outlier rows/features. The multiplicative form of half-quadratic optimization is used to optimize the non-convex correntropy objective function of the proposed models. Extensive experiments on face datasets well demonstrate that the proposed methods are more robust to corruptions and occlusions.
Correntropy induced metric based graph regularized non-negative matrix factorization

Bin Mao,Naiyang Guan,Dacheng Tao,Xuhui Huang

DOI: https://doi.org/10.1109/SPAC.2014.6982679

2014-01-01

Abstract:Non-negative matrix factorization (NMF) is an efficient dimension reduction method and plays an important role in many pattern recognition and computer vision tasks. However, conventional NMF methods are not robust since the objective functions are sensitive to outliers and do not consider the geometric structure in datasets. In this paper, we proposed a correntropy graph regularized NMF (CGNMF) to overcome the aforementioned problems. CGNMF maximizes the correntropy between data matrix and its reconstruction to filter out the noises of large magnitudes, and expects the coefficients to preserve the intrinsic geometric structure of data. We also proposed a modified version of our CGNMF which construct the adjacent graph by using sparse representation to enhance its reliability. Experimental results on popular image datasets confirm the effectiveness of CGNMF.
Dynamic Saliency-Aware Regularization for Correlation Filter based Object Tracking.

Wei Feng,Ruize Han,Qing Guo,Jianke Zhu,Song Wang

DOI: https://doi.org/10.1109/TIP.2019.2895411

IF: 10.6

2019-01-01

IEEE Transactions on Image Processing

Abstract:With a good balance between tracking accuracy and speed, correlation filter (CF) has become one of the best object tracking frameworks, based on which many successful trackers have been developed. Recently, spatially regularized CF tracking (SRDCF) has been developed to remedy the annoying boundary effects of CF tracking, thus further boosting the tracking performance. However, SRDCF uses a fixed ...
Improved Robust Kernel Subspace for Object-Based Registration and Change Detection

Zhaoyang Zhang,Zheng Tian,Mingtao Ding,Anup Basu

DOI: https://doi.org/10.1109/LGRS.2012.2224314

2013-01-01

Abstract:Postclassification comparison is an approach to detect changes in remote sensing images that have strongly inhomogeneous scenes. It is a challenging task to register pre- and postevent scenarios, because variform classifications may induce an inadequate number of homologous points to be used as tie points. In this letter, we show how the variform objects can be precisely registered using their robust kernel subspace. There are two primary contributions in our work. First, a robust kernel subspace analysis method is proposed to capture the common patterns of the variform objects. Second, a registration method based on the common patterns and their preimage are derived. The power of the proposed approach is demonstrated by two real applications: one for lake monitoring in the Jiayu region and the other for damage mapping of earthquake-induced barrier lake at Tangjiashan. The results show that the proposed method is effective in structural pattern analysis and object registration.
Correntropy Supervised Non-Negative Matrix Factorization

Wenju Zhang,Naiyang Guan,Dacheng Tao,Bin Mao,Xuhui Huang,Zhigang Luo

DOI: https://doi.org/10.1109/IJCNN.2015.7280629

2015-01-01

Abstract:Non-negative matrix factorization (NMF) is a powerful dimension reduction method and has been widely used in many pattern recognition and computer vision problems. However, conventional NMF methods are neither robust enough as their loss functions are sensitive to outliers, nor discriminative because they completely ignore labels in a dataset. In this paper, we proposed a correntropy supervised NMF (CSNMF) to simultaneously overcome aforementioned deficiencies. In particular, CSNMF maximizes the correntropy between the data matrix and its reconstruction in low-dimensional space to inhibit outliers during learning the subspace, and narrows the minimizes the distances between coefficients of any two samples with the same class labels to enhance the subsequent classification performance. To solve CSNMF, we developed a multiplicative update rules and theoretically proved its convergence. Experimental results on popular face image datasets verify the effectiveness of CSNMF comparing with NMF, its supervised variants, and its robustified variants.
A Grouped Structure-based Regularized Regression Model for Text Categorization

Wenbin Zheng,Yuntao Qian,Minchao Ye

DOI: https://doi.org/10.4304/jsw.7.9.2119-2124

2012-01-01

Journal of Software

Abstract:The lasso regularization has successfully been used in regression models for feature selection; however, lasso considers all variable to be independent and noncorrelative, which will yield an excessively sparse solution (i.e., some important discriminating features might be discarded) if the features are highly correlated. This paper proposes a novel approach in which a sparse model was developed for text categorization. We firstly constructed a grouped structure according the correlation of text features, and then embedded the structure into a regression model via a between- and within- group sparse manner. The goal of such manner is that the groups containing many discriminating features can be selected even the features in these groups are highly correlated, and the noise within the selected groups could be discarded simultaneously, which is beneficial for classification. The experimental results show that the proposed method achieves a good tradeoff between performance and sparsity on three benchmark data sets.
Multi-Target Regression Via Robust Low-Rank Learning.

Xiantong Zhen,Mengyang Yu,Xiaofei He,Shuo Li

DOI: https://doi.org/10.1109/tpami.2017.2688363

IF: 23.6

2018-01-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:Multi-target regression has recently regained great popularity due to its capability of simultaneously learning multiple relevant regression tasks and its wide applications in data mining, computer vision and medical image analysis, while great challenges arise from jointly handling inter-target correlations and input-output relationships. In this paper, we propose Multi-layer Multi-target Regression (MMR) which enables simultaneously modeling intrinsic inter-target correlations and nonlinear input-output relationships in a general framework via robust low-rank learning. Specifically, the MMR can explicitly encode inter-target correlations in a structure matrix by matrix elastic nets (MEN); the MMR can work in conjunction with the kernel trick to effectively disentangle highly complex nonlinear input-output relationships; the MMR can be efficiently solved by a new alternating optimization algorithm with guaranteed convergence. The MMR leverages the strength of kernel methods for nonlinear feature learning and the structural advantage of multi-layer learning architectures for inter-target correlation modeling. More importantly, it offers a new multi-layer learning paradigm for multi-target regression which is endowed with high generality, flexibility and expressive ability. Extensive experimental evaluation on 18 diverse real-world datasets demonstrates that our MMR can achieve consistently high performance and outperforms representative state-of-the-art algorithms, which shows its great effectiveness and generality for multivariate prediction.
Robust Rigid Registration Algorithm Based on Pointwise Correspondence and Correntropy

Shaoyi Du,Guanglin Xu,Sirui Zhang,Xuetao Zhang,Yue Gao,Badong Chen

DOI: https://doi.org/10.1016/j.patrec.2018.06.028

IF: 4.757

2018-01-01

Pattern Recognition Letters

Abstract:The iterative closest point (ICP) algorithm is fast and accurate for rigid point set registration, but it works badly when handling noisy data or point clouds with outliers. This paper instead proposes a novel method based on the ICP algorithm to deal with this problem. Firstly, correntropy is introduced into the rigid registration problem which could handle noises and outliers well, and then a new energy function based on maximum correntropy criterion is proposed. After that, a new ICP algorithm based on correntropy is proposed, which performs well in dealing with rigid registration with noises and outliers. This new algorithm converges monotonically from any given parameters, which is similar to the ICP algorithm. Experimental results demonstrate its accuracy and robustness compared with the traditional ICP algorithm. (c) 2018ElsevierB.V. Allrightsreserved.
Robust Low-Rank Regularized Regression for Face Recognition with Occlusion

Jianjun Qian,Jian Yang,Fanlong Zhang,Zhouchen Lin

DOI: https://doi.org/10.1109/cvprw.2014.9

2014-01-01

Abstract:Recently, regression analysis based classification methods are popular for robust face recognition. These methods use a pixel-based error model, which assumes that errors of pixels are independent. This assumption does not hold in the case of contiguous occlusion, where the errors are spatially correlated. Observing that occlusion in a face image generally leads to a low-rank error image, we propose a low-rank regularized regression model and use the alternating direction method of multipliers (ADMM) to solve it. We thus introduce a novel robust low-rank regularized regression (RLR3) method for face recognition with occlusion. Compared with the existing structured sparse error coding models, which perform error detection and error support separately, our method integrates error detection and error support into one regression model. Experiments on benchmark face databases demonstrate the effectiveness and robustness of our method, which outperforms state-of-the-art methods.
Robust deep multi-view subspace clustering networks with a correntropy-induced metric

Xiaomeng Si,Qiyue Yin,Xiaojie Zhao,Li Yao

DOI: https://doi.org/10.1007/s10489-022-03209-9

IF: 5.3

2022-03-30

Applied Intelligence

Abstract:Since multi-view subspace clustering combines the advantages of deep learning to capture the nonlinear nature of data, deep multi-view subspace clustering methods have demonstrated superior ability to shallow multi-view subspace clustering methods. Most existing methods assume that sample reconstruction errors incurred by noise conform to the prior distribution of the corresponding norm, allowing for simplification of the problem and focus on designing specific regularization on self-representation matrices to exploit consistent and diverse information among different views. However, the noise distributions in different views are always very complex, and in practice the noise distributions do not necessarily conform to this hypothesis. Furthermore, the commonly used diversity regularization based on value-awareness to enhance diversity among different view representations is not sufficiently accurate. To alleviate the above deficiencies, we propose novel robust deep multi-view subspace clustering networks with a correntropy-induced metric (RDMSCNet). (1) A correntropy-induced metric (CIM) is utilized to flexibly handle various complex noise distributions in a data-driven manner to improve the robustness of the model. (2) A position-aware diversity regularization based on the exclusivity definition is employed to enforce the diversity of the different view representations for modelling the consistency and diversity simultaneously. Extensive experiments show that RDMSCNet can deliver enhanced performance over state-of-the-art approaches.

computer science, artificial intelligence
Correntropy-Induced Robust Low-Rank Hypergraph.

Taisong Jin,Rongrong Ji,Yue Gao,Xiaoshuai Sun,Xibin Zhao,Dacheng Tao

DOI: https://doi.org/10.1109/TIP.2018.2889960

2019-01-01

Abstract:Hypergraph learning has been widely exploited in various image processing applications, due to its advantages in modeling the high-order information. Its efficacy highly depends on building an informative hypergraph structure to accurately and robustly formulate the underlying data correlation. However, the existing hypergraph learning methods are sensitive to non- Gaussian noise, which hurts the corresponding performance. In this paper, we present a noise-resistant hypergraph learning model, which provides superior robustness against various non- Gaussian noises. In particular, our model adopts low-rank representation to construct a hypergraph, which captures the globally linear data structure as well as preserving the grouping effect of highly-correlated data. We further introduce a correntropyinduced local metric to measure the reconstruction errors, which is particularly robust to non-Gaussian noises. Finally, the Frobenious-norm based regularization is proposed to combine with the low-rank regularizer, which enables our model to regularize the singular values of the coefficient matrix. By such, the non-zero coefficients are selected to generate a hyperedge set as well as the hyperedge weights. We have evaluated the proposed hypergraph model in the tasks of image clustering and semi-supervised image classification. Quantitatively, our scheme significantly enhances the performance of the state-of-the-art hypergraph models on several benchmark datasets.
Robust Regularized Low-Rank Matrix Models for Regression and Classification

Teng Zhang,Hsin-Hsiung Huang,Xing Fan,Feng Yu

DOI: https://doi.org/10.48550/arXiv.2205.07106

2022-05-14

Abstract:While matrix variate regression models have been studied in many existing works, classical statistical and computational methods for the analysis of the regression coefficient estimation are highly affected by high dimensional and noisy matrix-valued predictors. To address these issues, this paper proposes a framework of matrix variate regression models based on a rank constraint, vector regularization (e.g., sparsity), and a general loss function with three special cases considered: ordinary matrix regression, robust matrix regression, and matrix logistic regression. We also propose an alternating projected gradient descent algorithm. Based on analyzing our objective functions on manifolds with bounded curvature, we show that the algorithm is guaranteed to converge, all accumulation points of the iterates have estimation errors in the order of $O(1/\sqrt{n})$ asymptotically and substantially attaining the minimax rate. Our theoretical analysis can be applied to general optimization problems on manifolds with bounded curvature and can be considered an important technical contribution to this work. We validate the proposed method through simulation studies and real image data examples.

Computer Science,Mathematics
Adaptive sparseness for correntropy-based robust regression via automatic relevance determination

Yuanhao Li,Badong Chen,Okito Yamashita,Natsue Yoshimura,Yasuharu Koike

DOI: https://doi.org/10.48550/arXiv.2302.00082

IF: 5.414

2023-01-31

Machine Learning

Abstract:Sparseness and robustness are two important properties for many machine learning scenarios. In the present study, regarding the maximum correntropy criterion (MCC) based robust regression algorithm, we investigate to integrate the MCC method with the automatic relevance determination (ARD) technique in a Bayesian framework, so that MCC-based robust regression could be implemented with adaptive sparseness. To be specific, we use an inherent noise assumption from the MCC to derive an explicit likelihood function, and realize the maximum a posteriori (MAP) estimation with the ARD prior by variational Bayesian inference. Compared to the existing robust and sparse L1-regularized MCC regression, the proposed MCC-ARD regression can eradicate the troublesome tuning for the regularization hyper-parameter which controls the regularization strength. Further, MCC-ARD achieves superior prediction performance and feature selection capability than L1-regularized MCC, as demonstrated by a noisy and high-dimensional simulation study.

A regularized correntropy framework for robust pattern recognition

A Regularized Correntropy Framework for Robust Pattern Recognition

Robust spectral regression for face recognition

Learning with the Maximum Correntropy Criterion Induced Losses for Regression.

Error Analysis on Regularized Regression Based on the Maximum Correntropy Criterion

Robust non-negative matrix factorization with multiple correntropy-induced hypergraph regularizer

Robust Regularized Kernel Regression.

Correntropy-Induced Discriminative Nonnegative Sparse Coding for Robust Palmprint Recognition

Correntropy Induced L2 Graph for Robust Subspace Clustering

Correntropy induced metric based graph regularized non-negative matrix factorization

Dynamic Saliency-Aware Regularization for Correlation Filter based Object Tracking.

Improved Robust Kernel Subspace for Object-Based Registration and Change Detection

Correntropy Supervised Non-Negative Matrix Factorization

A Grouped Structure-based Regularized Regression Model for Text Categorization

Multi-Target Regression Via Robust Low-Rank Learning.

Robust Rigid Registration Algorithm Based on Pointwise Correspondence and Correntropy

Robust Low-Rank Regularized Regression for Face Recognition with Occlusion

Robust deep multi-view subspace clustering networks with a correntropy-induced metric

Correntropy-Induced Robust Low-Rank Hypergraph.

Robust Regularized Low-Rank Matrix Models for Regression and Classification

Adaptive sparseness for correntropy-based robust regression via automatic relevance determination