Abstract:In high-dimensional data modeling, variable selection plays a crucial role in improving predictive accuracy and enhancing model interpretability through sparse representation. Unfortunately, certain variable selection methods encounter challenges such as insufficient model sparsity, high computational overhead, and difficulties in handling large-scale data. Recently, axis-aligned random projection techniques have been applied to address these issues by selecting variables. However, these techniques have seen limited application in handling complex data within the regression framework. In this study, we propose a novel method, sparse partial least squares via axis-aligned random projection, designed for the analysis of high-dimensional data. Initially, axis-aligned random projection is utilized to obtain a sparse loading vector, significantly reducing computational complexity. Subsequently, partial least squares regression is conducted within the subspace of the top-ranked significant variables. The submatrices are iteratively updated until an optimal sparse partial least squares model is achieved. Comparative analysis with some state-of-the-art high-dimensional regression methods demonstrates that the proposed method exhibits superior predictive performance. To illustrate its effectiveness, we apply the method to four cases, including one simulated dataset and three real-world datasets. The results show the proposed method's ability to identify important variables in all four cases.

Research on the Application of Multi-Dimensional Orthogonal Total Least Squares

Total Least Squares with Application in Geospatial Data Processing

An Error-in-variable Projection to Latent Structure Framework for Monitoring Technical Systems with Orthogonal Signal Components

Orthogonal Least Squares Regression for Feature Extraction

A Least-Squares Model To Orthogonal Linear Discriminant Analysis

Orthogonal multi-view analysis by successive approximations via eigenvectors

Orthogonal Vs. Uncorrelated Least Squares Discriminant Analysis for Feature Extraction.

Orthogonal least squares based on singular value decomposition for spare basis selection

Author Response for "partial Least Median of Squares Regression"

Orthogonal projection analysis

Orthogonal Subsampling for Big Data Linear Regression

Consistent estimation with the use of orthogonal projections for a linear regression model with errors in the variables

Robust Total Least Squares with Reweighting Iteration for Three-Dimensional Similarity Transformation

On Total Least Squares for Quadratic Form Estimation

Variable selection using axis-aligned random projections for partial least-squares regression

Weighted and Unweighted Total Least Square Methods and Applications to Heteroscedastic 3D Coordinate Transformation

Multiview partial least squares

One component partial least squares, high dimensional regression, data splitting, and the multitude of models

No penalty no tears: Least squares in high-dimensional linear models

Supervised Dimensionality Reduction Methods via Recursive Regression

Randomized algorithms for total least squares problems.