Abstract:In data mining, regression analysis is a computational tool that predicts continuous output variables from a number of independent input variables, by approximating their complex inner relationship. A large number of methods have been successfully proposed, based on various methodologies, including linear regression, support vector regression, neural network, piece-wise regression, etc. In terms of piece-wise regression, the existing methods in literature are usually restricted to problems of very small scale, due to their inherent non-linear nature. In this work, a more efficient piece-wise linear regression method is introduced based on a novel integer linear programming formulation. The proposed method partitions one input variable into multiple mutually exclusive segments, and fits one multivariate linear regression function per segment to minimise the total absolute error. Assuming both the single partition feature and the number of regions are known, the mixed integer linear model is proposed to simultaneously determine the locations of multiple break-points and regression coefficients for each segment. Furthermore, an efficient heuristic procedure is presented to identify the key partition feature and final number of break-points. 7 real world problems covering several application domains have been used to demonstrate the efficiency of our proposed method. It is shown that our proposed piece-wise regression method can be solved to global optimality for datasets of thousands samples, which also consistently achieves higher prediction accuracy than a number of state-of-the-art regression methods. Another advantage of the proposed method is that the learned model can be conveniently expressed as a small number of if-then rules that are easily interpretable. Overall, this work proposes an efficient rule-based multivariate regression method based on piece-wise functions and achieves better prediction performance than state-of-the-arts approaches. This novel method can benefit expert systems in various applications by automatically acquiring knowledge from databases to improve the quality of knowledge base.

A Review of Piecewise Linearization Methods

Piecewise Linear Programming and Its Application

An Efficient Interior-Point Algorithm with New Non-Monotone Line Search Filter Method for Nonlinear Constrained Programming

Linearly Constrained Global Optimization Via Piecewise-Linear Approximation

A new linearization technique for minimax linear fractional programming

Sequential Piecewise Linear Programming for Convergent Optimization of Non-Convex Problems

Transformation and Linearization Techniques in Optimization: A State-of-the-Art Survey

Sequential Global Linear Programming Algorithm for Continuous Piecewise Linear Programming

Finding All Solutions of Piecewise-Linear Equations Using Linear Programming

Linear Direct Transcription for Nonlinear Constrained Optimal Control Problems

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Global Optimization of Piecewise Linear (PWL) Function with a Linear Constraint

Mathematical programming for piecewise linear regression analysis

Efficient continuous piecewise linear regression for linearising univariate non-linear functions

Generalized Uniformly Optimal Methods for Nonlinear Programming

Global optimization of signomial mixed-integer nonlinear programming problems with free variables

Global optimization of nonlinear fractional programming problems in engineering design

Optimizing Scheduling of Refinery Operations Based on Piecewise Linear Models

Global optimization of separable objective functions on convex polyhedra via piecewise-linear approximation

A New Global Optimization Algorithm for a Class of Linear Fractional Programming