Abstract:We present a hierarchy of semidefinite programs (SDPs) for the problem of fitting a shape-constrained (multivariate) polynomial to noisy evaluations of an unknown shape-constrained function. These shape constraints include convexity or monotonicity over a box. We show that polynomial functions that are optimal to any fixed level of our hierarchy form a consistent estimator of the underlying shape-constrained function. As a byproduct of the proof, we establish that sum-of-squares-convex polynomials are dense in the set of polynomials that are convex over an arbitrary box. A similar sum of squares type density result is established for monotone polynomials. In addition, we classify the complexity of convex and monotone polynomial regression as a function of the degree of the polynomial regressor. While our results show NP-hardness of these problems for degree three or larger, we can check numerically that our SDP-based regressors often achieve similar training error at low levels of the hierarchy. Finally, on the computational side, we present an empirical comparison of our SDP-based convex regressors with the convex least squares estimator introduced in [Hildreth, 1954] and [Holloway, 1979] and show that our regressor is valuable in settings where the number of data points is large and the dimension is relatively small. We demonstrate the performance of our regressor for the problem of computing optimal transport maps in a color transfer task and that of estimating the optimal value function of a conic program. A real-time application of the latter problem to inventory management contract negotiation is presented.

Shape constrained kernel-weighted least squares: Application to production function estimation for Chilean manufacturing industries

Linear programming approach to nonparametric inference under shape restrictions: with an application to regression kink designs

Novel Robust Least-Squares Estimator for Linear Dynamic Data Reconciliation

Estimation of Additive Frontier Functions with Shape Constraints

An axiomatic nonparametric production function estimator: Modeling production in Japan's cardboard industry

Adaptive Estimation and Uniform Confidence Bands for Nonparametric Structural Functions and Elasticities

Shape-constrained Estimation in Functional Regression with Bernstein Polynomials

SPLINE-BACKFITTED KERNEL SMOOTHING OF ADDITIVE COEFFICIENT MODEL

Least Squares Estimation of a Quasiconvex Regression Function

Product Quality Improvement Method In Manufacturing Process Based On Kernel Optimisation Algorithm

A flexible stochastic production frontier model with panel data

Shape-Constrained Regression using Sum of Squares Polynomials

Nonlinear Multivariate Quality Estimation and Prediction Based on Kernel Partial Least Squares

Dynamic Large Spatial Covariance Matrix Estimation in Application to Semiparametric Model Construction via Variable Clustering: the SCE approach

Kernel density weighted method of obtaining the least square support vector machines weightes

Scalable Statistical Inference in Non-parametric Least Squares

Calibrating Least Squares Semidefinite Programming with Equality and Inequality Constraints

A compromise criterion for weighted least squares estimates

Incorporating Shape Knowledge into Regression Models

Weighted Profile Least Squares Estimation for a Panel Data Varying-Coefficient Partially Linear Model

Efficient and fast spline-backfitted kernel smoothing of additive models