Abstract:The inference performance of the pseudolikelihood method is discussed in the framework of the inverse Ising problem when the l (2)-regularized (ridge) linear regression is adopted. This setup is introduced for theoretically investigating the situation where the data generation model is different from the inference one, namely the model mismatch situation. In the teacher-student scenario under the assumption that the teacher couplings are sparse, the analysis is conducted using the replica and cavity methods, with a special focus on whether the presence/absence of teacher couplings is correctly inferred or not. The result indicates that despite the model mismatch, one can perfectly identify the network structure using naive linear regression without regularization when the number of spins N is smaller than the dataset size M, in the thermodynamic limit N -> infinity. Further, to access the underdetermined region M < N, we examine the effect of the l (2) regularization, and find that biases appear in all the coupling estimates, preventing the perfect identification of the network structure. We, however, find that the biases are shown to decay exponentially fast as the distance from the center spin chosen in the pseudolikelihood method grows. Based on this finding, we propose a two-stage estimator: in the first stage, the ridge regression is used and the estimates are pruned by a relatively small threshold; in the second stage the naive linear regression is conducted only on the remaining couplings, and the resultant estimates are again pruned by another relatively large threshold. This estimator with the appropriate regularization coefficient and thresholds is shown to achieve the perfect identification of the network structure even in 0 < M/N < 1. Results of extensive numerical experiments support these findings.

Analysis of Network Lasso for Semi-Supervised Regression

Structure Learning in Inverse Ising Problems Using ℓ_2-Regularized Linear Estimator

On the Duality between Network Flows and Network Lasso

Semi-supervised Image Classification Via Nonnegative Least-Squares Regression

Classifying Partially Labeled Networked Data via Logistic Network Lasso

Network Lasso: Clustering and Optimization in Large Graphs

Robust and Efficient Network Reconstruction in Complex System via Adaptive Signal Lasso

Network Regression with Graph Laplacians

The Network Representation Learning Algorithm Based on Semi-Supervised Random Walk

Combining Clustering Coefficient-Based Active Learning and Semi-Supervised Learning on Networked Data

Non-negative low rank and sparse graph for semi-supervised learning

Overlapping coefficient in network-based semi-supervised clustering

Semi-supervised regression with label-guided adaptive graph optimization

Non-parametric regression for networks

Efficient And Robust Semi-Supervised Learning Over A Sparse-Regularized Graph

LOGISTIC REGRESSION WITH NETWORK STRUCTURE

Noise-robust Semi-Supervised Learning Via Fast Sparse Coding

Variance-enlarged Poisson Learning for Graph-based Semi-Supervised Learning with Extremely Sparse Labeled Data

Semi-supervised Regression via Parallel Field Regularization.

Network Inference With the Lasso

Analysis of $p$-Laplacian Regularization in Semi-Supervised Learning