Abstract:Core decomposition is a fundamental graph problem with a large number of applications. Most existing approaches for core decomposition assume that the graph is kept in memory of a machine. Nevertheless, many real-world graphs are big and may not reside in memory. In the literature, there is only one work for I/O efficient core decomposition that avoids loading the whole graph in memory. However, this approach is not scalable to handle big graphs because it cannot bound the memory size and may load most parts of the graph in memory. In addition, this approach can hardly handle graph updates. In this paper, we study I/O efficient core decomposition following a semi-external model, which only allows node information to be loaded in memory. This model works well in many web-scale graphs. We propose a semi-external algorithm and two optimized algorithms for I/O efficient core decomposition using very simple structures and data access model. To handle dynamic graph updates, we show that our algorithm can be naturally extended to handle edge deletion. We also propose an I/O efficient core maintenance algorithm to handle edge insertion, and an improved algorithm to further reduce I/O and CPU cost by investigating some new graph properties. We conduct extensive experiments on 12 real large graphs. Our optimal algorithm significantly outperform the existing I/O efficient algorithm in terms of both processing time and memory consumption. In many memory-resident graphs, our algorithms for both core decomposition and maintenance can even outperform the in-memory algorithm due to the simple structures and data access model used. Our algorithms are very scalable to handle web-scale graphs. As an example, we are the first to handle a web graph with 978.5 million nodes and 42.6 billion edges using less than 4.2 GB memory.

Core-GAE: Toward Generation of IoT Networks

Energy Efficient Resource Allocation Based on Virtual Network Embedding for IoT Data Generation

Efficient Learning-based Community-Preserving Graph Generation

General Graph Generators: Experiments, Analyses, and Improvements

Generative Model for Node Generation.

Empowering Wireless Networks with Artificial Intelligence Generated Graph

Graph Convolutional Network Aided Virtual Network Embedding for Internet of Thing.

CGGM: A conditional graph generation model with adaptive sparsity for node anomaly detection in IoT networks

Global Core, and Galaxy Structure of Networks

DGE: Deep Generative Network Embedding Based on Commonality and Individuality.

I/O Efficient Core Graph Decomposition at Web Scale

K-Core Decomposition on Super Large Graphs with Limited Resources

Efficient and Scalable Graph Generation through Iterative Local Expansion

Gcore: Exploring Cross-layer Cohesiveness in Multi-layer Graphs.

Knowledge Enhanced GAN for IoT Traffic Generation

Dynamic and Textual Graph Generation Via Large-Scale LLM-based Agent Simulation

Generative Models and Learning Algorithms for Core-Periphery Structured Graphs

Generating Real-World Hypergraphs Via Deep Generative Models

Virtualized Network Function Forwarding Graph Placing in SDN and NFV-Enabled IoT Networks: A Graph Neural Network Assisted Deep Reinforcement Learning Method

Anchored Coreness: Efficient Reinforcement of Social Networks

Core Knowledge Learning Framework for Graph Adaptation and Scalability Learning