Abstract:In a paper that initiated the modern study of the stochastic block model, Decelle et al., backed by Mossel et al., made the following conjecture: Denote by $k$ the number of balanced communities, $a/n$ the probability of connecting inside communities and $b/n$ across, and set $\mathrm{SNR}=(a-b)^2/(k(a+(k-1)b)$; for any $k \geq 2$, it is possible to detect communities efficiently whenever $\mathrm{SNR}>1$ (the KS threshold), whereas for $k\geq 4$, it is possible to detect communities information-theoretically for some $\mathrm{SNR}<1$. Massoulié, Mossel et al.\ and Bordenave et al.\ succeeded in proving that the KS threshold is efficiently achievable for $k=2$, while Mossel et al.\ proved that it cannot be crossed information-theoretically for $k=2$. The above conjecture remained open for $k \geq 3$. This paper proves this conjecture, further extending the efficient detection to non-symmetrical SBMs with a generalized notion of detection and KS threshold. For the efficient part, a linearized acyclic belief propagation (ABP) algorithm is developed and proved to detect communities for any $k$ down to the KS threshold in time $O(n \log n)$. Achieving this requires showing optimality of ABP in the presence of cycles, a challenge for message passing algorithms. The paper further connects ABP to a power iteration method with a nonbacktracking operator of generalized order, formalizing the interplay between message passing and spectral methods. For the information-theoretic (IT) part, a non-efficient algorithm sampling a typical clustering is shown to break down the KS threshold at $k=4$. The emerging gap is shown to be large in some cases; if $a=0$, the KS threshold reads $b \gtrsim k^2$ whereas the IT bound reads $b \gtrsim k \ln(k)$, making the SBM a good study-case for information-computation gaps.

Detectability thresholds of general modular graphs

Computational and Statistical Thresholds in Multi-layer Stochastic Block Models

Modularity and partially observed graphs

Modular decomposition of Markov chain: detecting hierarchical organization of pervasive communities

Complexity of detectability, opacity and A-diagnosability for modular discrete event systems

Generalized modularity matrices

[math]-Modules in Graphs

Detection in the stochastic block model with multiple clusters: proof of the achievability conjectures, acyclic BP, and the information-computation gap

Detectability of communities in heterogeneous networks

Modularity of regular and treelike graphs

Network community detection using modularity density measures

Unraveling the Detectability of Stochastic Block Model with Overlapping Communities.

Algorithmic infeasibility of community detection in higher-order networks

Sparse graph counting and Kelley-Meka bounds for binary systems

Subgraph densities and scaling limits of random graphs with a prescribed modular decomposition

Weak recovery, hypothesis testing, and mutual information in stochastic block models and planted factor graphs

Modularity bounds for clusters located by leading eigenvectors of the normalized modularity matrix

Communities and bottlenecks: Trees and treelike networks have high modularity

Scalable detection of statistically significant communities and hierarchies, using message-passing for modularity

A Spectral Method To Detect Community Structure Based On Distance Modularity Matrix

A computational transition for detecting correlated stochastic block models by low-degree polynomials