Abstract:The conventional Quadratic Discriminant Analysis (QDA) encounters a significant hurdle due to parameter scaling complexities on the order of O(p 2 ), rendering it impractical for the analysis of high or ultrahigh dimensional data. This arises especially when estimating the covariance matrix or its inverse, a necessity in such scenarios. In this research, we present an innovative two‐stage QDA procedure that mitigates this obstacle by reducing the dimensionality from p to a manageable level of o(min{n, p}). This reduction allows for a direct application of QDA even when the dimensionality is growing exponentially in terms of p. We observe that, under certain sparsity assumptions, the Bayes rule can be reformulated in a low‐dimensional form. This observation motivates us to select the most relevant classification features in the first stage using feature screening methods. Subsequently, we concentrate solely on this reduced subspace to formulate classifiers in the second stage. In addition to using QDA directly in the second stage, we introduce sparse QDA, resulting in three methods for constructing classifiers in the second stage. Under appropriate sparsity assumptions, we establish the consistency and misclassification rate of our proposed procedure. Numerical simulations and real data analyses demonstrate the effectiveness of our proposed method in finite‐sample scenarios.

Dimensionality Determination: A Thresholding Double Ridge Ratio Approach

Computational and Theoretical Analysis of Supervised Dimensionality Reduction

Data-driven Dimension Reduction for High-Dimensional Random Vibration Systems with Cubic Nonlinearity

Theoretical Analysis and Comparison of Several Criteria on Linear Model Dimension Reduction

Sufficient Dimension Reduction Using Hilbert-Schmidt Independence Criterion

Ratio sum formula for dimensionality reduction

Two‐Stage Estimation for Ultrahigh Dimensional Sparse Quadratic Discriminant Analysis

A new sufficient dimension reduction method via rank divergence

Helping Effects Against Curse of Dimensionality in Threshold Factor Models for Matrix Time Series

A variant of the trace quotient formulation for dimensionality reduction

A New Quality Assessment Criterion for Nonlinear Dimensionality Reduction

Neighborhood Structure Preserving Ridge Regression for Dimensionality Reduction.

Order Determination for Spiked Models

A Theoretical Investigation of Several Model Selection Criteria for Dimensionality Reduction

Dimension Reduction via Gaussian Ridge Functions

A New Estimator for Efficient Dimension Reduction in Regression.

Optimal Dimensionality of Metric Space for Classification

Estimation of the Number of Endmembers Via Thresholding Ridge Ratio Criterion

Regularized Discriminant Analysis, Ridge Regression and Beyond

On partial sufficient dimension reduction with applications to partially linear multi-index models

Supervised Dimensionality Reduction Methods via Recursive Regression