Abstract:This paper studies the nonparametric modal regression problem systematically from a statistical learning view. Originally motivated by pursuing a theoretical understanding of the maximum correntropy criterion based regression (MCCR), our study reveals that MCCR with a tending-to-zero scale parameter is essentially modal regression. We show that nonparametric modal regression problem can be approached via the classical empirical risk minimization. Some efforts are then made to develop a framework for analyzing and implementing modal regression. For instance, the modal regression function is described, the modal regression risk is defined explicitly and its \textit{Bayes} rule is characterized; for the sake of computational tractability, the surrogate modal regression risk, which is termed as the generalization risk in our study, is introduced. On the theoretical side, the excess modal regression risk, the excess generalization risk, the function estimation error, and the relations among the above three quantities are studied rigorously. It turns out that under mild conditions, function estimation consistency and convergence may be pursued in modal regression as in vanilla regression protocols, such as mean regression, median regression, and quantile regression. However, it outperforms these regression models in terms of robustness as shown in our study from a re-descending M-estimation view. This coincides with and in return explains the merits of MCCR on robustness. On the practical side, the implementation issues of modal regression including the computational algorithm and the tuning parameters selection are discussed. Numerical assessments on modal regression are also conducted to verify our findings empirically.

Sparse Modal Additive Model

Error Bound of Mode-Based Additive Models

Tilted Sparse Additive Models.

Additive Models with Spatio-Temporal Data

Convex-constrained Sparse Additive Modeling and Its Extensions

Sparse Additive Machine With the Correntropy-Induced Loss

Classification by sparse generalized additive models

Efficient Probabilistic Latent Semantic Analysis with Sparsity Control

Sparse Additive Machine with Ramp Loss

Regularized Modal Regression with Data-Dependent Hypothesis Spaces.

Sparse additive machine with pinball loss

Regularized Modal Regression with Applications in Cognitive Impairment Prediction.

A Statistical Learning Approach to Modal Regression

Sparse Model Identification and Learning for Ultra-high-dimensional Additive Partially Linear Models

Minimax Signal Detection in Sparse Additive Models

Huber Additive Models for Non-stationary Time Series Analysis

Partial Multi-Modal Sparse Coding Via Adaptive Similarity Structure Regularization

Sparse Analysis Model Based Multiplicative Noise Removal with Enhanced Regularization.

High-Dimensional Sparse Additive Hazards Regression

Data-adaptive additive modeling

Sparse Additive Subspace Clustering