Beyond One Solution: The Case for a Comprehensive Exploration of Solution Space in Community Detection

Fabio Morea,Domenico De Stefano

2024-10-25

Abstract:This article explores the importance of examining the solution space in community detection, highlighting its role in achieving reliable results when dealing with real-world problems. A Bayesian framework is used to estimate the stability of the solution space and classify it into categories Single, Dominant, Multiple, Sparse or Empty. By applying this approach to real-world networks, the study highlights the importance of considering multiple solutions rather than relying on a single partition. This ensures more reliable results and efficient use of computational resources in community detection analysis.

Social and Information Networks

What problem does this paper attempt to address?

This paper aims to solve a key problem in community detection: how to comprehensively explore the solution space to ensure reliable results when dealing with real - world problems. Specifically, the author points out that traditional community detection algorithms usually produce only a single partition result, ignoring the situation where there may be multiple valid partitions. This may lead to an incomplete or biased understanding of the network structure.

### Main Research Questions

1. **Stability of the Solution Space**:
- The research objective is to determine the minimum number of trials \( t_c \) so that it can be confidently asserted that the solution space \( S \) is stable, that is, no new solutions will emerge.

2. **Classification of the Solution Space**:
- A taxonomy is introduced to describe different types of solution spaces, based on the number of unique partitions \( n_s \) found and the relative frequency of each partition.

### Solution Space Classification

According to the paper, the solution space \( S \) can be divided into the following types:

- **Single type**: The solution space is stable and has only one valid partition (\( n_s = 1 \)).
- **Dominant type**: There are multiple valid partitions (\( n_s>1 \)), but one partition is dominant, that is, its lower frequency limit \( \max(p_{\text{lower}})>0.5 \).
- **Multiple type**: There are multiple valid partitions (\( n_s>1 \)), but no partition is dominant, that is, \( \max(p_{\text{lower}})<0.5 \).
- **Sparse type**: There are a large number of solutions (\( n_s\approx t \)), and the probability of each solution is very low (\( \max(p_{\text{upper}})\approx0 \)).
- **Empty type**: There are no valid solutions, or all solutions are invalid (\( n_s = 0 \)).

### Methodology

To explore the stability of the solution space and the importance of each solution, the author designed an experimental setup and constructed a probability model \( M \) using a Bayesian framework. The model \( M \) is initialized with a Beta - Binomial distribution and is continuously updated as the trials progress. The specific steps are as follows:

1. **Input**: Graph \( G \), community detection algorithm \( A \), maximum number of trials \( t_{\text{max}} \) and threshold \( \tau \).
2. **Initialization**: Empty solution space \( S \) and non - informative prior Beta - Binomial model \( M \).
3. **Loop**: Conduct trials from 1 to \( t_{\text{max}} \):
- Shuffle the graph \( G \) to generate a random permutation \( G^* \).
- Run the community detection algorithm \( A(G^*, \rho) \) to obtain partition \( P_i \).
- If \( P_i\notin S \), add the new solution \( P_i \) to the solution space \( S \) and update the model \( M \).
4. **Termination Condition**: Exit the loop when \( t_{\text{max}} \) is reached or \( p_{\text{stable}} \) exceeds the threshold \( \tau \).

### Results and Conclusions

By analyzing large - scale dense networks in the "Horizon Projects Network" dataset, the author shows the types of solution spaces produced by different community detection algorithms (such as Infomap and Louvain). For example:

- Using the Infomap algorithm, the obtained solution space \( S_{\text{IM}} \) is of the Dominant type, with 4 valid solutions, and the maximum frequency is approximately 0.85.