Abstract:Deep neural networks have achieved tremendous success due to their representation power and adaptation to low-dimensional structures. Their potential for estimating structured regression functions has been recently established in the literature. However, most of the studies require the input dimension to be fixed and consequently ignore the effect of dimension on the rate of convergence and hamper their applications to modern big data with high dimensionality. In this paper, we bridge this gap by analyzing a $k^{th}$ order nonparametric interaction model in both growing dimension scenarios ($d$ grows with $n$ but at a slower rate) and in high dimension ($d \gtrsim n$). In the latter case, sparsity assumptions and associated regularization are required in order to obtain optimal rates of convergence. A new challenge in diverging dimension setting is in calculation mean-square error, the covariance terms among estimated additive components are an order of magnitude larger than those of the variances and they can deteriorate statistical properties without proper care. We introduce a critical debiasing technique to amend the problem. We show that under certain standard assumptions, debiased deep neural networks achieve a minimax optimal rate both in terms of $(n, d)$. Our proof techniques rely crucially on a novel debiasing technique that makes the covariances of additive components negligible in the mean-square error calculation. In addition, we establish the matching lower bounds.

Nonlinear Interaction Detection Through Model-Based Sufficient Dimension Reduction

Detecting Interaction for Quantitative Trait by Generalized Multifactor Dimensionality Reduction

An Efficient Model‐free Approach to Interaction Screening for High Dimensional Data

On efficient dimension reduction with respect to the interaction between two response variables

Multifactor Dimensionality Reduction for Detecting Haplotype-Haplotype Interaction.

A high-dimensional single-index regression for interactions between treatment and covariates

Reluctant Interaction Modeling in Generalized Linear Models

Specification Testing of Partially Linear Single-Index Models: a Groupwise Dimension Reduction-Based Adaptive-to-model Approach

The Application of Multifactor Dimensionality Reduction for Detecting Gene-Gene Interactions

Estimation and inference in generalized additive coefficient models for nonlinear interactions with high-dimensional covariates

Strong Sure Screening of Ultra-high Dimensional Data with Interaction Effects

Integrative interaction analysis using threshold gradient directed regularization: Integrative interaction analysis

Missing Data Analysis with Sufficient Dimension Reduction

Two-Sample Tests for High-Dimensional Linear Regression with an Application to Detecting Interactions.

Asymmetric Independence Model for Detecting Interactions between Variables

Deep Neural Networks for Nonparametric Interaction Models with Diverging Dimension

Detection of Interaction Effects in a Nonparametric Concurrent Regression Model

Impact of Sufficient Dimension Reduction in Nonparametric Estimation of Causal Effect

On partial sufficient dimension reduction with applications to partially linear multi-index models

Sufficient Dimension Reduction in the Presence of Controlling Variables

High-dimensional Interactions Detection with Sparse Principal Hessian Matrix