Abstract:The inference performance of the pseudolikelihood method is discussed in the framework of the inverse Ising problem when the l (2)-regularized (ridge) linear regression is adopted. This setup is introduced for theoretically investigating the situation where the data generation model is different from the inference one, namely the model mismatch situation. In the teacher-student scenario under the assumption that the teacher couplings are sparse, the analysis is conducted using the replica and cavity methods, with a special focus on whether the presence/absence of teacher couplings is correctly inferred or not. The result indicates that despite the model mismatch, one can perfectly identify the network structure using naive linear regression without regularization when the number of spins N is smaller than the dataset size M, in the thermodynamic limit N -> infinity. Further, to access the underdetermined region M < N, we examine the effect of the l (2) regularization, and find that biases appear in all the coupling estimates, preventing the perfect identification of the network structure. We, however, find that the biases are shown to decay exponentially fast as the distance from the center spin chosen in the pseudolikelihood method grows. Based on this finding, we propose a two-stage estimator: in the first stage, the ridge regression is used and the estimates are pruned by a relatively small threshold; in the second stage the naive linear regression is conducted only on the remaining couplings, and the resultant estimates are again pruned by another relatively large threshold. This estimator with the appropriate regularization coefficient and thresholds is shown to achieve the perfect identification of the network structure even in 0 < M/N < 1. Results of extensive numerical experiments support these findings.

Ising Model Selection Using 𝓁-Regularized Linear Regression.

Ising Model Selection Using ℓ1ℓ1\ell_{1}-Regularized Linear Regression: A Statistical Mechanics Analysis

Ising Model Selection Using ℓ_1-Regularized Linear Regression: A Statistical Mechanics Analysis

On Model Selection Consistency of Lasso for High-Dimensional Ising Models

Structure Learning in Inverse Ising Problems Using ℓ_2-Regularized Linear Estimator

Regularization and variable selection for infinite variance autoregressive models

Meta Learning for High-dimensional Ising Model Selection Using $\ell_1$-regularized Logistic Regression

Asymptotic Behavior for S-estimators in Random Design Linear Model with Long-Range-dependent Errors

High-Dimensional Graphical Model Selection Using $\ell_1$-Regularized Logistic Regression

Bayesian Model Selection for High-Dimensional Ising Models, With Applications to Educational Data

Logistic regression and Ising networks: prediction and estimation when violating lasso assumptions

Interaction Screening: Efficient and Sample-Optimal Learning of Ising Models

Variational Pseudolikelihood for Regularized Ising Inference

A statistical physics approach to learning curves for the Inverse Ising problem

High-dimensional Ising model selection with Bayesian information criteria

Exact results for the Ising model on a small-world network

Ferromagnetic Ising Model on multiregular random graphs

Simultaneous support recovery in high dimensions: Benefits and perils of block $\ell_1/\ell_\infty$-regularization

Model Selection in Undirected Graphical Models with the Elastic Net

Robust Estimation of Tree Structured Ising Models

Exact goodness-of-fit testing for the Ising model