Abstract:Recently, many regression-based conditional independence (CI) test methods have been proposed to solve the problem of causal discovery. These methods provide alternatives to test CI of x , y given Z by first removing the information of the controlling set Z from x and y , and then testing the independence between the two residuals R x , Z and R y , Z . When the residuals are linearly uncorrelated, the independence test between them is nontrivial. With the ability to calculate inner product in high-dimensional space, kernel-based methods are usually used to achieve this goal, but they are considerably time-consuming. In this paper, we test the independence between two linear combinations under linear structural equation model. We show that the dependence between the two residuals can be captured by the difference between the similarity of R x , Z and R y , Z and that of R x , Z and R r ( R r is an independent copy of R y , Z ) in high-dimensional space. With this result, we provide a new way to test CI based on the similarity between residuals, which is called SCIT — the abbreviation of Similarity-based CI Testing. Furthermore, we develop two versions of the proposal, called Kernel-SCIT and Neural-SCIT respectively. Kernel-SCIT calculates the similarity by using kernel functions, while Neural-SCIT approximates the upper bound of the similarity by using deep neural networks. In both algorithms, random permutation tests are performed to control Type I error rate. The proposed tests are evaluated on (conditional) independence test and causal discovery with both synthetic and real datasets. Experimental results show that Kernel-SCIT is simpler yet more efficient and effective than the typical existing kernel-based methods HSIC and KCIT in the cases of small sample size, and Neural-SCIT can significantly boost the performance of CI testing when sufficient samples are available. The source code is available at https://github.com/xyw5vplus1/SCIT.

A low variance consistent test of relative dependency

Fast Non-Parametric Tests of Relative Dependency and Similarity

A nonparametric Bayesian test of dependence

Distance-based and RKHS-based dependence metrics in high dimension

New improvements in the use of dependence measures for sensitivity analysis and screening

Test for independence of long-range dependent time series using distance covariance

Testing Independence with High-Dimensional Correlated Samples

Kernel-based Tests for Joint Independence

A Test of General Asymmetric Dependence

RANK-BASED INDICES FOR TESTING INDEPENDENCE BETWEEN TWO HIGH-DIMENSIONAL VECTORS

Randomization Tests for Equality in Dependence Structure

Universally Consistent K-Sample Tests via Dependence Measures

Conditional Independence Test Based on Residual Similarity

A New Nonparametric Test for High-Dimensional Regression Coefficients

Variance-estimation-free Test of Significant Covariates in High-Dimensional Regression

BET on Independence

The Projected Covariance Measure for assumption-lean variable significance testing

Distribution-free tests of multivariate independence based on center-outward quadrant, Spearman, Kendall, and van der Waerden statistics

CONDITIONAL DISTANCE CORRELATION.

Testing independence in high dimensions with sums of rank correlations

A Distribution-Free Test of Independence and Its Application to Variable Selection