Abstract:Kernel regression is a popular non-parametric fitting technique. It aims at learning a function which estimates the targets for test inputs as precise as possible. Generally, the function value for a test input is estimated by a weighted average of the surrounding training examples. The weights are typically computed by a distance-based kernel function and they strongly depend on the distances between examples. In this paper, we first review the latest developments of sparse metric learning and kernel regression. Then a novel kernel regression method involving sparse metric learning, which is called kernel regression with sparse metric learning (KR$\_$SML), is proposed. The sparse kernel regression model is established by enforcing a mixed $(2,1)$-norm regularization over the metric matrix. It learns a Mahalanobis distance metric by a gradient descent procedure, which can simultaneously conduct dimensionality reduction and lead to good prediction results. Our work is the first to combine kernel regression with sparse metric learning. To verify the effectiveness of the proposed method, it is evaluated on 19 data sets for regression. Furthermore, the new method is also applied to solving practical problems of forecasting short-term traffic flows. In the end, we compare the proposed method with other three related kernel regression methods on all test data sets under two criterions. Experimental results show that the proposed method is much more competitive.

Kernel-Based Mixture Of Experts Models For Linear Regression

Generalized Convexity-Based Inexact Projection Method for Multiple Kernel Learning

Model-based clustering and classification using mixtures of multivariate skewed power exponential distributions

Adaptive Bayesian nonparametric regression using kernel mixture of polynomials with application to partial linear model

Robust Regularized Kernel Regression.

Prediction Sets for High-Dimensional Mixture of Experts Models

Robust kernel ensemble regression in diversified kernel space with shared parameters

Hierarchical Mixture-of-Experts Model for Large-Scale Gaussian Process Regression

An efficient multiple kernel computation method for regression analysis of economic data

A novel multivariate regression approach based on kernel partial least squares with orthogonal signal correction

Kernel Least-Mean Mixed-Norm Algorithm

Implicit Surface Representation Using Epanechnikov Mixture Regression

Kernel Regression with Sparse Metric Learning

Bayesian Kernel and Mutual $k$-Nearest Neighbor Regression

Families of Parsimonious Finite Mixtures of Regression Models

Machine learning embedded EM algorithms for semiparametric mixture regression models

Multivariate Universal Local Linear Kernel Estimators in Nonparametric Regression: Uniform Consistency

Adaptive sequential Monte Carlo by means of mixture of experts

Estimating Mixture of Gaussian Processes by Kernel Smoothing

On Parameter Estimation in Deviated Gaussian Mixture of Experts

Model Selection for Exponential Power Mixture Regression Models