SOCRATES: A System For Scalable Graph Analytics

Cetin Savkli,Ryan Carr,Matthew Chapman,Brant Chee,David Minch

DOI: https://doi.org/10.48550/arXiv.1708.05076

2017-08-16

Distributed, Parallel, and Cluster Computing

Abstract:A distributed semantic graph processing system that provides locality control, indexing, graph query, and parallel processing capabilities is presented.

What problem does this paper attempt to address?

Graph3S: A Simple, Speedy and Scalable Distributed Graph Processing System

Wang Xubo,Qin Lu,Chang Lijun,Zhang Ying,Wen Dong,Lin Xuemin

DOI: https://doi.org/10.48550/arxiv.2003.00680

2020-01-01

Abstract: Graph is a ubiquitous structure in many domains. The rapidly increasing data volume calls for efficient and scalable graph data processing. In recent years, designing distributed graph processing systems has been an increasingly important area to fulfil the demands of processing big graphs in a distributed environment. Though a variety of distributed graph processing systems have been developed, very little attention has been paid to achieving a good combinational system performance in terms of usage simplicity, efficiency and scalability. To contribute to the study of distributed graph processing system, this work tries to fill this gap by designing a simple, speedy and scalable system. Our observation is that enforcing the communication flexibility of a system leads to the gains of both system efficiency and scalability as well as simple usage. We realize our idea in a system Graph3S and conduct extensive experiments with diverse algorithms over big graphs from different domains to test its performance. The results show that, besides simple usage, our system has outstanding performance over various graph algorithms and can even reach up to two orders of magnitude speedup over existing in-memory systems when applying to some algorithms. Also, its scalability is competitive to disk-based systems and even better when less machines are used.
SQL-G: Efficient Graph Analytics by SQL

Kangfei Zhao,Jiao Su,Jeffrey Xu Yu,Hao Zhang

DOI: https://doi.org/10.1109/TKDE.2019.2950620

IF: 9.235

2021-01-01

IEEE Transactions on Knowledge and Data Engineering

Abstract:Querying graphs and conducting graph analytics become important in data processing since many real applications are dealing with massive graphs, such as online social networks, Semantic Web, knowledge graphs, etc. Over the years, many distributed graph processing systems have been developed to support graph analytics using various programming models, and many graph querying languages have been proposed. A natural question that arises is how to integrate graph data and traditional non-graph data in a distributed system for users to conduct analytics. There are two issues. One issue is related to expressiveness on how to specify graph analytics as well as data analytics by a querying language. The other issue is related to efficiency on how to process analytics in a distributed system. For the first issue, SQL is a best candidate, since SQL is a well-accepted language for data processing. We concentrate on SQL for graph analytics. Our early work shows that graph analytics can be supported by SQL in a way from "semiring + while" to "relational algebra + while" via the enhanced recursive SQL queries. In this article, we focus on the second issue on how to process such enhanced recursive SQL queries based on the GAS (Gather-Apply-Scatter) model under which efficient graph processing systems can be developed. To demonstrate the efficiency, we implemented a system by tightly coupling Spark SQL and GraphX on Spark which is one of the most popular in-memory data-flow processing platforms. First, we enhance Spark SQL by adding the capability of supporting the enhanced recursive SQL queries for graph analytics. In this regard, graph analytics can be processed using a distributed SQL engine alone. Second, we further propose new transformation rules to optimize/translate the operations for recursive SQL queries to the operations by GraphX. In this regard, graph analytics by SQL can be processed in a similar way as done by a distributed graph processing system using the APIs provided by the system. We conduct extensive performance studies to test graph analytics using large real graphs. We show that our approach can achieve similar or even higher efficiency, in comparison to the built-in graph algorithms in the existing graph processing systems.
SparkRDF: Elastic Discreted RDF Graph Processing Engine with Distributed Memory

Xi Chen,Huajun Chen,Ningyu Zhang,Songyang Zhang

DOI: https://doi.org/10.1109/wi-iat.2015.186

2014-01-01

Abstract:With the explosive growth of semantic data on the Web over the past years, many large-scale RDF knowledge bases with billions of facts are generating. This poses significant challenges for the storage and query of big RDF graphs. Current systems still have many limitations in processing big RDF graphs including scalability and real-time. In this paper, we introduce the SparkRDF, an elastic discreted RDF graph processing engine with distributed memory. To reduce the high I/O and communication cost in distributed processing platforms, SparkRDF implements SPARQL query based on Spark, a novel in-memory distributed computing framework for big data processing. All the intermediate results are modeled as Resilient Discreted SubGraph, which are cached in the distributed memory to support fast iterative join operations. To cut down the search space and avoid the overhead of memory, we split the RDF graph into the small Multi-layer Elastic SubGraph based on the relations and classes. For SPARQL query optimization, SparkRDF deploys a serials of optimization strategies, leading to effective reduction on the size of intermediate results, the number of joins and the cost of communication. Our extensive evaluation demonstrates that SparkRDF can efficiently implement non-selective joins faster than both current state-of-the-art distributed and centralized stores, while being able to process other queries in real time, scaling linearly to the amount of data.
Large scale graph processing systems: survey and an experimental evaluation

Omar Batarfi,Radwa El Shawi,Ayman G. Fayoumi,Reza Nouri,Seyed-Mehdi-Reza Beheshti,Ahmed Barnawi,Sherif Sakr

DOI: https://doi.org/10.1007/s10586-015-0472-6

2015-07-24

Cluster Computing

Abstract:Graph is a fundamental data structure that captures relationships between different data entities. In practice, graphs are widely used for modeling complicated data in different application domains such as social networks, protein networks, transportation networks, bibliographical networks, knowledge bases and many more. Currently, graphs with millions and billions of nodes and edges have become very common. In principle, graph analytics is an important big data discovery technique. Therefore, with the increasing abundance of large graphs, designing scalable systems for processing and analyzing large scale graphs has become one of the most timely problems facing the big data research community. In general, scalable processing of big graphs is a challenging task due to their size and the inherent irregular structure of graph computations. Thus, in recent years, we have witnessed an unprecedented interest in building big graph processing systems that attempted to tackle these challenges. In this article, we provide a comprehensive survey over the state-of-the-art of large scale graph processing platforms. In addition, we present an extensive experimental study of five popular systems in this domain, namely, GraphChi, Apache Giraph, GPS, GraphLab and GraphX. In particular, we report and analyze the performance characteristics of these systems using five common graph processing algorithms and seven large graph datasets. Finally, we identify a set of the current open research challenges and discuss some promising directions for future research in the domain of large scale graph processing.
Towards Scale-out Capability on Social Graphs

Haichuan Shang,Xiang Zhao,R. Uday Kiran,Masaru Kitsuregawa

DOI: https://doi.org/10.1145/2806416.2806420

2015-01-01

Abstract:The development of cloud storage and computing has facilitated the rise of various big data applications. As a representative high performance computing (HPC) workload, graph processing is becoming a part of cloud computing. However, scalable computing on large graphs is still dominated by HPC solutions, which require high performance all-to-all collective operations over torus (or mesh) networking. Implementing those torus-based algorithms on commodity clusters, e.g., cloud computing infrastructures, can result in great latency due to inefficient communication. Moreover, designing a highly scalable system for large social graphs, is far from being trivial, as intrinsic features of social graphs, e.g., degree skewness and lacking of locality, often profoundly limit the extent of parallelism. To resolve the challenges, we explore the iceberg of developing a scalable system for processing large social graphs on commodity clusters. In particular, we focus on the scale-out capability of the system. We propose a novel separator-combiner based query processing engine which provides native load-balancing and very low communication overhead, such that increasinglylarger graphs can be simply addressed by adding more computing nodes to the cluster.The proposed system achieves remarkable scale-out capability in processing large social graphs with skew degree distributions, while providing many critical features for big data analytics, such as easy-to-use API, fault-tolerance and recovery. We implement the system as a portable and easily configurable library, and conduct comprehensive experimental studies to demonstrate its effectiveness and efficiency.
A Distributed Graph-Parallel Computing System with Lightweight Communication Overhead

Yue Zhao,Kenji Yoshigoe,Jiang Bian,Mengjun Xie,Zhe Xue,Yong Feng

DOI: https://doi.org/10.1109/tbdata.2016.2532907

2016-01-01

IEEE Transactions on Big Data

Abstract:In order to process complex and large-scale graph data numerous distributed graph-parallel computing platforms have been proposed. However, excessive communications among computing nodes in these systems not only aggravate the network I/O workload of the underlying computing hardware systems but may also cause a decrease in runtime performance and scalability. In this paper, we propose and implement a system called Ligraph, which computes large-scale graph data in distributed mode with lightweight communication overhead. Ligraph is similar to PowerGraph system with three new features: (1) a Gather partial sum difference based computing model; (2) a corresponding lightweight Gather communication mechanism; (3) for PageRank-like algorithms Ligraph additionally employs a lightweight synchronizing communication mechanism and an edge direction-aware graph partition strategy proposed by our former work LightGraph, which is specially designed for PageRank-like algorithms. We have conducted extensive experiments using real-world data sets, and our results verified the effectiveness of Ligraph on reducing the communication overhead and improving the runtime performance and the scalability compared with PowerGraph and LightGraph. For example, compared with PowerGraph under Random partition scenario Ligraph can not only reduce up to 35.2 percent of the communication overhead but also cut up to 21.8 percent of the runtime for PageRank algorithm while processing Twitter data set. Our experiment results also demonstrate that compared with several other representative existing systems Ligraph also outperforms them in graph computing rate.
Demonstrating Interactive Multi-resolution Large Graph Exploration

Zhiyuan Lin,Nan Cao,Hanghang Tong,Fei Wang,U. Kang,Duen Horng Polo Chau

DOI: https://doi.org/10.1109/ICDMW.2013.124

2013-01-01

Abstract:We present a scalable, interactive graph visualization system to support multi-resolution exploration of million-node graphs in real time. By adapting a state-of-the-art graph algorithm, called Slash & Burn, our prototype system generates a multi-resolution view of graphs with up to 69 million edges under a few seconds. We are experimenting with interaction techniques that help users interactively explore this overview and drill down into details. While many visualization systems for million-node graphs require dedicated servers to process the graphs, our prototype runs on a commodity laptop computer. We aim to handle graphs that are at least an order of magnitude (100M edges) larger than what current systems can support. We demonstrate our system's usage, benefits, and scalability using two large graphs: a Live Journal friendship network with 69 million edges, and a related-movies network from Rotten Tomatoes with 200K edges.
Graphu: A Unified Vertex-Centric Parallel Graph Processing Platform

Jing Su,Qun Chen,Zhuo Wang,Murtadha H. M. Ahmed,Zhanhuai Li

DOI: https://doi.org/10.1109/ICDCS.2018.00160

2018-01-01

Abstract:Many synchronous and asynchronous distributed platforms based on the Bulk Synchronous Parallel (BSP) model have been built for large-scale vertex-centric graph processing. Unfortunately, a program designed for a synchronous platform may not work properly on an asynchronous one. As a result, given the same problem, end users may be required to design different parallel algorithms for different platforms. Recently, we have proposed a unified programming model, DFA-G (Deterministic Finite Automaton for Graph processing), which expresses the computation at a vertex as a series of message-driven state transitions. It has the attractive property that any program modeled after it can run properly across synchronous and asynchronous platforms.In this demo, we first propose a framework of complexity analysis for DFA-G automaton and show that it can significantly facilitate complexity analysis on asynchronous programs. Due to the existing BSP platforms' deficiency in supporting efficient DFA-G execution, we then develop a new prototype platform, GraphU. GraphU was built on the popular open-source Giraph project. But it entirely removes synchronization barriers and decouples remote communication from vertex computation. Finally, we empirically evaluate the performance of various DFA-G programs on GraphU by a comparative study. Our experiments validate the efficacy of the proposed complexity analysis approach and the efficiency of GraphU.Video: http://www.wowbigdata.com.cn/GraphU/video.html.
Scalable Processing of Massive Uncertain Graph Data: A Simultaneous Processing Approach.

Zhaonian Zou,Faming Li,Jianzhong Li,Yingshu Li

DOI: https://doi.org/10.1109/icde.2017.70

2017-01-01

Abstract:This paper studies a novel approach to processing massive uncertain graph data. In this approach, we propose a new framework to simultaneously process a query on a set of randomly sampled possible worlds of an uncertain graph. Based on this framework, we develop a series of algorithms to analyze massive uncertain graphs, including breadth-first search, shortest distance queries, triangle counting, and core decomposition. We implement this approach based on GraphLab, one of the stateof-the-art graph processing frameworks. By sharing fine-grained internal processing steps on common substructures of sampled possible worlds, the new approach achieves tens to hundreds of times speedup in execution time on a cluster of 20 servers.
Processing Concurrent Graph Analytics with Decoupled Computation Model.

Jilong Xue,Zhi Yang,Shian Hou,Yafei Dai

DOI: https://doi.org/10.1109/tc.2016.2618923

IF: 3.183

2016-01-01

IEEE Transactions on Computers

Abstract:Graph processing systems have been widely used in enterprises like online social networks to process their daily jobs. With the fast growing of social applications, they have to efficiently handle massive concurrent jobs. However, due to the inherent design for a single job, existing systems incur great inefficiency in terms of memory usage, execution and fault tolerance. Motivated by this issue, in this paper we introduce Seraph, a graph processing system that enables efficient job-level parallelism. Seraph is designed based on a decoupled computation model, which decouples both the runtime job data and the computation logic. Decoupling the runtime data allows multiple concurrent jobs to share graph structure data in memory, which fundamentally increases job-level concurrency and reduces fault tolerance overhead. Decoupling computation logic could extend scheduling space, which benefits both execution performance and memory consumption. Seraph adopts a copy-on-write semantic to isolate the graph mutation of concurrent jobs, and a lazy snapshot protocol to generate consistent graph snapshots for jobs submitted at different time. Based on the decoupled model, it provides unified programming interfaces for both synchronous and asynchronous graph applications. Moreover, Seraph implements a lightweight checkpoint mechanism which can tremendously reduce the fault tolerance overhead. The evaluation results show that Seraph significantly outperforms popular systems (such as Giraph, Spark, GraphX and PowerLyra) in both memory usage and job completion time, when executing concurrent graph jobs.
System G Distributed Graph Database

Gabriel Tanase,Toyotaro Suzumura,Jinho Lee,Chun-Fu Chen,Jason Crawford,Hiroki Kanezashi,Song Zhang,Warut D. Vijitbenjaronk,Warut D.Vijitbenjaronk

DOI: https://doi.org/10.48550/arXiv.1802.03057

2018-02-08

Databases

Abstract:Motivated by the need to extract knowledge and value from interconnected data, graph analytics on big data is a very active area of research in both industry and academia. To support graph analytics efficiently a large number of in memory graph libraries, graph processing systems and graph databases have emerged. Projects in each of these categories focus on particular aspects such as static versus dynamic graphs, off line versus on line processing, small versus large graphs, etc. While there has been much advance in graph processing in the past decades, there is still a need for a fast graph processing, using a cluster of machines with distributed storage. In this paper, we discuss a novel distributed graph database called System G designed for efficient graph data storage and processing on modern computing architectures. In particular we describe a single node graph database and a runtime and communication layer that allows us to compose a distributed graph database from multiple single node instances. From various industry requirements, we find that fast insertions and large volume concurrent queries are critical parts of the graph databases and we optimize our database for such features. We experimentally show the efficiency of System G for storing data and processing graph queries on state-of-the-art platforms.
SOCRAT: a Dynamic Web Toolbox for Interactive Data Processing, Analysis and Visualization

Alexandr A Kalinin,Selvam Palanimalai,Junqi Zhu,Wenyi Wu,Nikhil Devraj,Chunchun Ye,Nellie Ponarul,Syed S Husain,Ivo D Dinov

DOI: https://doi.org/10.3390/info13110547

Abstract:Many systems for exploratory and visual data analytics require platform-dependent software installation, coding skills, and analytical expertise. The rapid advances in data-acquisition, web-based information, and communication and computation technologies promoted the explosive growth of online services and tools implementing novel solutions for interactive data exploration and visualization. However, web-based solutions for visual analytics remain scattered and relatively problem-specific. This leads to per-case re-implementations of common components, system architectures, and user interfaces, rather than focusing on innovation and building sophisticated applications for visual analytics. In this paper, we present the Statistics Online Computational Resource Analytical Toolbox (SOCRAT), a dynamic, flexible, and extensible web-based visual analytics framework. The SOCRAT platform is designed and implemented using multi-level modularity and declarative specifications. This enables easy integration of a number of components for data management, analysis, and visualization. SOCRAT benefits from the diverse landscape of existing in-browser solutions by combining them with flexible template modules into a unique, powerful, and feature-rich visual analytics toolbox. The platform integrates a number of independently developed tools for data import, display, storage, interactive visualization, statistical analysis, and machine learning. Various use cases demonstrate the unique features of SOCRAT for visual and statistical analysis of heterogeneous types of data.
Odyssey: A Journey in the Land of Distributed Data Series Similarity Search

Manos Chatzakis,Panagiota Fatourou,Eleftherios Kosmas,Themis Palpanas,Botao Peng

DOI: https://doi.org/10.14778/3579075.3579087

IF: 2.5

2023-01-01

Proceedings of the VLDB Endowment

Abstract:This paper presents Odyssey, a novel distributed data-series processing framework that efficiently addresses the critical challenges of exhibiting good speedup and ensuring high scalability in data series processing by taking advantage of the full computational capacity of modern distributed systems comprised of multi-core servers. Odyssey addresses a number of challenges in designing efficient and highly-scalable distributed data series index, including efficient scheduling, and load-balancing without paying the prohibitive cost of moving data around. It also supports a flexible partial replication scheme, which enables Odyssey to navigate through a fundamental trade-off between data scalability and good performance during query answering. Through a wide range of configurations and using several real and synthetic datasets, our experimental analysis demonstrates that Odyssey achieves its challenging goals.

computer science, information systems, theory & methods
Efficient and Scalable Graph Parallel Processing with Symbolic Execution

Long Zheng,Xiaofei Liao,Hai Jin

DOI: https://doi.org/10.1145/3170434

IF: 1.444

2018-01-01

ACM Transactions on Architecture and Code Optimization

Abstract:Existing graph processing essentially relies on the underlying iterative execution with synchronous (Sync) and/or asynchronous (Async) engine. Nevertheless, they both suffer from a wide class of inherent serialization arising from data interdependencies within a graph. In this article, we present SymGraph, a judicious graph engine with symbolic iteration that enables the parallelism of dependent computation on vertices. SymGraph allows using abstract symbolic value (instead of the concrete value) for the computation if the desired data is unavailable. To maximize the potential of symbolic iteration, we propose a chain of tailored sophisticated techniques, enabling SymGraph to scale out with a new milestone of efficiency for large-scale graph processing. We evaluate SymGraph in comparison to Sync, Async, and a hybrid of Sync and Async engines. Our results on 12 nodes show that SymGraph outperforms all three graph engines by 1.93x (vs. Sync), 1.98x (vs. Async), and 1.57x (vs. Hybrid) on average. In particular, the performance for PageRank on 32 nodes can be dramatically improved by 16.5x (vs. Sync), 23.3x (vs. Async), and 12.1x (vs. Hybrid), respectively. The efficiency of SymGraph is also validated with at least one order of magnitude improvement in contrast to three specialized graph systems (Naiad, GraphX, and PGX.D).
Graph Analytics Through Fine-Grained Parallelism.

Zechao Shang,Feifei Li,Jeffrey Xu Yu,Zhiwei Zhang,Hong Cheng

DOI: https://doi.org/10.1145/2882903.2915238

2016-01-01

Abstract:Large graphs are getting increasingly popular and even indispensable in many applications, for example, in social media data, large networks, and knowledge bases. Efficient graph analytics thus becomes an important subject of study. To increase efficiency and scalability, in-memory computation and parallelism have been explored extensively to speed up various graph analytical workloads. In many graph analytical engines (e.g., Pregel, Neo4j, GraphLab), parallelism is achieved via one of the three concurrency control models, namely, bulk synchronization processing (BSP), asynchronous processing, and synchronous processing. Among them, synchronous processing has the potential to achieve the best performance due to fine-grained parallelism, while ensuring the correctness and the convergence of the computation, if an effective concurrency control scheme is used. This paper explores the topological properties of the underlying graph to design and implement a highly effective concurrency control scheme for efficient synchronous processing in an in-memory graph analytical engine. Our design uses a novel hybrid approach that combines 2PL (two-phase locking) with OCC (optimistic concurrency control), for high degree and low degree vertices in a graph respectively. Our results show that the proposed hybrid synchronous scheduler has significantly outperformed other synchronous schedulers in existing graph analytical engines, as well as BSP and asynchronous schedulers.
Large-scale graph processing systems: a survey

Ning Liu,Dong-sheng Li,Yi-ming Zhang,Xiong-lve Li

DOI: https://doi.org/10.1631/FITEE.1900127

2020-04-02

Abstract:Graph is a significant data structure that describes the relationship between entries. Many application domains in the real world are heavily dependent on graph data. However, graph applications are vastly different from traditional applications. It is inefficient to use general-purpose platforms for graph applications, thus contributing to the research of specific graph processing platforms. In this survey, we systematically categorize the graph workloads and applications, and provide a detailed review of existing graph processing platforms by dividing them into general-purpose and specialized systems. We thoroughly analyze the implementation technologies including programming models, partitioning strategies, communication models, execution models, and fault tolerance strategies. Finally, we analyze recent advances and present four open problems for future research.

engineering, electrical & electronic,computer science, information systems, software engineering
Demystifying Graph Databases: Analysis and Taxonomy of Data Organization, System Designs, and Graph Queries

Maciej Besta,Robert Gerstenberger,Emanuel Peter,Marc Fischer,Michał Podstawski,Claude Barthels,Gustavo Alonso,Torsten Hoefler

DOI: https://doi.org/10.1145/3604932

IF: 16.6

2023-06-26

ACM Computing Surveys

Abstract:Numerous irregular graph datasets, for example social networks or web graphs, may contain even trillions of edges. Often, their structure changes over time and they have domain-specific rich data associated with vertices and edges. Graph database systems such as Neo4j enable storing, processing, and analyzing such large, evolving, and rich datasets. Due to the sheer size and irregularity of such datasets, these systems face unique design challenges. To facilitate the understanding of this emerging domain, we present the first survey and taxonomy of graph database systems. We focus on identifying and analyzing fundamental categories of these systems (e.g., document stores, tuple stores, native graph database systems, or object-oriented systems), the associated graph models (e.g., RDF or Labeled Property Graph), data organization techniques (e.g., storing graph data in indexing structures or dividing data into records), and different aspects of data distribution and query execution (e.g., support for sharding and ACID). 51 graph database systems are presented and compared, including Neo4j, OrientDB, and Virtuoso. We outline graph database queries and relationships with associated domains (NoSQL stores, graph streaming, and dynamic graph algorithms). Finally, we outline future research and engineering challenges related to graph databases.

computer science, theory & methods
SympleGraph: Distributed Graph Processing with Precise Loop-Carried Dependency Guarantee.

Youwei Zhuo,Jingji Chen,Qinyi Luo,Yanzhi Wang,Hailong Yang,Depei Qian,Xuehai Qian

DOI: https://doi.org/10.1145/3385412.3385961

2020-01-01

Abstract:Graph analytics is an important way to understand relationships in real-world applications. At the age of big data, graphs have grown to billions of edges. This motivates distributed graph processing. Graph processing frameworks ask programmers to specify graph computations in user- defined functions (UDFs) of graph-oriented programming model. Due to the nature of distributed execution, current frameworks cannot precisely enforce the semantics of UDFs, leading to unnecessary computation and communication. In essence, there exists a gap between programming model and runtime execution. This paper proposes SympleGraph, a novel distributed graph processing framework that precisely enforces loop-carried dependency, i.e., when a condition is satisfied by a neighbor, all following neighbors can be skipped. SympleGraph instruments the UDFs to express the loop-carried dependency, then the distributed execution framework enforces the precise semantics by performing dependency propagation dynamically. Enforcing loop-carried dependency requires the sequential processing of the neighbors of each vertex distributed in different nodes. Therefore, the major challenge is to enable sufficient parallelism to achieve high performance. We propose to use circulant scheduling in the framework to allow different machines to process disjoint sets of edges/vertices in parallel while satisfying the sequential requirement. It achieves a good trade-off between precise semantics and parallelism. The significant speedups in most graphs and algorithms indicate that the benefits of eliminating unnecessary computation and communication overshadow the reduced parallelism. Communication efficiency is further optimized by 1) selectively propagating dependency for large-degree vertices to increase net benefits; 2) double buffering to hide communication latency. In a 16-node cluster, SympleGraph outperforms the state-of-the-art system Gemini and D-Galois on average by 1.42× and 3.30×, and up to 2.30× and 7.76×, respectively. The communication reduction compared to Gemini is 40.95% on average and up to 67.48%.
Big Graph Management Based on Scalable Computing Platforms

Gao Jun,Yang Dongqing

2013-01-01

Abstract:With the development of data collection and storage techniques, big graphs with massive size, complex intra-relationships and different graph queries, appear in social network, bio-informatics, and transportation network, etc. The classic graph algorithms which are suitable in memory of a single computer cannot be used to big graphs directly. The exploration of existing scalable platforms provides feasible solutions to manage big graphs. In this paper, we first investigate different kinds of graph queries, and then analyze the advantages and disadvantages of different approaches to management of big graphs using various platforms, including relational database, MapReduce framework, BSP(Bulk Synchronous Parallel) model and third party outsourced servers, and finally outline the future work.
Software Systems Implementation and Domain-Specific Architectures Towards Graph Analytics

Hai Jin,Hao Qi,Jin Zhao,Xinyu Jiang,Yu Huang,Chuangyi Gui,Qinggang Wang,Xinyang Shen,Yi Zhang,Ao Hu,Dan Chen,Chaoqiang Liu,Haifeng Liu,Haiheng He,Xiangyu Ye,Runze Wang,Jingrui Yuan,Pengcheng Yao,Yu Zhang,Long Zheng,Xiaofei Liao

DOI: https://doi.org/10.34133/2022/9806758

2022-01-01

Abstract:Graph analytics, which mainly includes graph processing, graph mining, and graph learning, has become increasingly important in several domains, including social network analysis, bioinformatics, and machine learning. However, graph analytics applications suffer from poor locality, limited bandwidth, and low parallelism owing to the irregular sparse structure, explosive growth, and dependencies of graph data. To address those challenges, several programming models, execution modes, and messaging strategies are proposed to improve the utilization of traditional hardware and performance. In recent years, novel computing and memory devices have emerged, e.g., HMCs, HBM, and ReRAM, providing massive bandwidth and parallelism resources, making it possible to address bottlenecks in graph applications. To facilitate understanding of the graph analytics domain, our study summarizes and categorizes current software systems implementation and domain-specific architectures. Finally, we discuss the future challenges of graph analytics.

SOCRATES: A System For Scalable Graph Analytics

Graph3S: A Simple, Speedy and Scalable Distributed Graph Processing System

SQL-G: Efficient Graph Analytics by SQL

SparkRDF: Elastic Discreted RDF Graph Processing Engine with Distributed Memory

Large scale graph processing systems: survey and an experimental evaluation

Towards Scale-out Capability on Social Graphs

A Distributed Graph-Parallel Computing System with Lightweight Communication Overhead

Demonstrating Interactive Multi-resolution Large Graph Exploration

Graphu: A Unified Vertex-Centric Parallel Graph Processing Platform

Scalable Processing of Massive Uncertain Graph Data: A Simultaneous Processing Approach.

Processing Concurrent Graph Analytics with Decoupled Computation Model.

System G Distributed Graph Database

SOCRAT: a Dynamic Web Toolbox for Interactive Data Processing, Analysis and Visualization

Odyssey: A Journey in the Land of Distributed Data Series Similarity Search

Efficient and Scalable Graph Parallel Processing with Symbolic Execution

Graph Analytics Through Fine-Grained Parallelism.

Large-scale graph processing systems: a survey

Demystifying Graph Databases: Analysis and Taxonomy of Data Organization, System Designs, and Graph Queries

SympleGraph: Distributed Graph Processing with Precise Loop-Carried Dependency Guarantee.

Big Graph Management Based on Scalable Computing Platforms

Software Systems Implementation and Domain-Specific Architectures Towards Graph Analytics