Abstract:We consider two-class linear classification in a high-dimensional, low-sample size setting. Only a small fraction of the features are useful, the useful features are unknown to us, and each useful feature contributes weakly to the classification decision -- this setting was called the rare/weak model (RW Model). We select features by thresholding feature $z$-scores. The threshold is set by {\it higher criticism} (HC). Let $\pee_i$ denote the $P$-value associated to the $i$-th $z$-score and $\pee_{(i)}$ denote the $i$-th order statistic of the collection of $P$-values. The HC threshold (HCT) is the order statistic of the $z$-score corresponding to index $i$ maximizing $(i/n - \pee_{(i)})/\sqrt{\pee_{(i)}(1-\pee_{(i)})}$. The ideal threshold optimizes the classification error. In \cite{PNAS} we showed that HCT was numerically close to the ideal threshold. We formalize an asymptotic framework for studying the RW model, considering a sequence of problems with increasingly many features and relatively fewer observations. We show that along this sequence, the limiting performance of ideal HCT is essentially just as good as the limiting performance of ideal thresholding. Our results describe two-dimensional {\it phase space}, a two-dimensional diagram with coordinates quantifying "rare" and "weak" in the RW model. Phase space can be partitioned into two regions -- one where ideal threshold classification is successful, and one where the features are so weak and so rare that it must fail. Surprisingly, the regions where ideal HCT succeeds and fails make the exact same partition of the phase diagram. Other threshold methods, such as FDR threshold selection, are successful in a substantially smaller region of the phase space than either HCT or Ideal thresholding.

Classification of threefold canonical thresholds

Infinitesimal structure of log canonical thresholds

Classification of real hyperplane singularities by real log canonical thresholds

Accumulation point theorem for generalized log canonical thresholds

The Canonical Volume of 3-Folds of General Type with Χ ⩽ 0

ACC for generalized log canonical thresholds for complex analytic spaces

General hyperplane sections of log canonical threefolds in positive characteristic

The canonical volume of threefolds of general type with χ<1

Feature selection by Higher Criticism thresholding: optimal phase diagram

Log canonical thresholds of Burniat surfaces with $K^2 = 5$

On the Iitaka volumes of log canonical surfaces and threefolds

ACC for log canonical thresholds for complex analytic spaces

BCM-thresholds of hypersurfaces

Multiplicities and log canonical threshold

Thin Set Versions of Hindman's Theorem

Log canonical thresholds of high multiplicity reduced plane curves

Supplementary Material of Generalized Singular Value Thresholding

CANONICAL THRESHOLDING FOR NON-SPARSE HIGH-DIMENSIONAL LINEAR REGRESSION

Counting Calabi-Yau Threefolds

Toward the equivalence of the ACC for $a$-log canonical thresholds and the ACC for minimal log discrepancies