Feature generation for graphs and networks

Yuan Yao,Hanghang Tong,Feng Xu,Jian Lu

DOI: https://doi.org/10.1201/9781315181080-7

2018-01-01

Abstract:In this chapter, we focus on the feature generation problem for graphs and networks. 1 The generated features are essential for various graph analysis tasks including node classification [5], link prediction [17], clustering [27], anomaly detection [2], etc.

What problem does this paper attempt to address?

Defragging Subgraph Features for Graph Classification.

Haishuai Wang,Peng Zhang,Ivor W. Tsang,Ling Chen,Chengqi Zhang

DOI: https://doi.org/10.1145/2806416.2806585

2015-01-01

Abstract:Graph classification is an important tool for analysing structured and semi-structured data, where subgraphs are commonly used as the feature representation. However, the number and size of subgraph features crucially depend on the threshold parameters of frequent subgraph mining algorithms. Any improper setting of the parameters will generate many trivial short-pattern subgraph fragments which dominate the feature space, distort graph classifiers and bury interesting long-pattern subgraphs. In this paper, we propose a new Subgraph Join Feature Selection (SJFS) algorithm. The SJFS algorithm, by forcing graph classifiers to join short-pattern subgraph fragments, can defrag trivial subgraph features and deliver long-pattern interesting subgraphs. Experimental results on both synthetic and real-world social network graph data demonstrate the performance of the proposed method.
Incremental Subgraph Feature Selection for Graph Classification.

Haishuai Wang,Peng Zhang,Xingquan Zhu,Ivor Wai-Hung Tsang,Ling Chen,Chengqi Zhang,Xindong Wu

DOI: https://doi.org/10.1109/tkde.2016.2616305

IF: 9.235

2017-01-01

IEEE Transactions on Knowledge and Data Engineering

Abstract:Graph classification is an important tool for analyzing data with structure dependency, where subgraphs are often used as features for learning. In reality, the dimension of the subgraphs crucially depends on the threshold setting of the frequency support parameter, and the number may become extremely large. As a result, subgraphs may be incrementally discovered to form a feature stream and require the underlying graph classifier to effectively discover representative subgraph features from the subgraph feature stream. In this paper, we propose a primal-dual incremental subgraph feature selection algorithm (ISF) based on a max-margin graph classifier. The ISF algorithm constructs a sequence of solutions that are both primal and dual feasible. Each primal-dual pair shrinks the dual gap and renders a better solution for the optimal subgraph feature set. To avoid bias of ISF algorithm on short-pattern subgraph features, we present a new incremental subgraph join feature selection algorithm (ISJF) by forcing graph classifiers to join short-pattern subgraphs and generate long-pattern subgraph features. We evaluate the performance of the proposed models on both synthetic networks and real-world social network data sets. Experimental results demonstrate the effectiveness of the proposed methods.
Feature Network Methods in Machine Learning and Applications

Xinying Mu,Mark Kon

2024-01-10

Abstract:A machine learning (ML) feature network is a graph that connects ML features in learning tasks based on their similarity. This network representation allows us to view feature vectors as functions on the network. By leveraging function operations from Fourier analysis and from functional analysis, one can easily generate new and novel features, making use of the graph structure imposed on the feature vectors. Such network structures have previously been studied implicitly in image processing and computational biology. We thus describe feature networks as graph structures imposed on feature vectors, and provide applications in machine learning. One application involves graph-based generalizations of convolutional neural networks, involving structured deep learning with hierarchical representations of features that have varying depth or complexity. This extends also to learning algorithms that are able to generate useful new multilevel features. Additionally, we discuss the use of feature networks to engineer new features, which can enhance the expressiveness of the model. We give a specific example of a deep tree-structured feature network, where hierarchical connections are formed through feature clustering and feed-forward learning. This results in low learning complexity and computational efficiency. Unlike "standard" neural features which are limited to modulated (thresholded) linear combinations of adjacent ones, feature networks offer more general feedforward dependencies among features. For example, radial basis functions or graph structure-based dependencies between features can be utilized.

Machine Learning
Node Attribute Generation on Graphs

Xu Chen,Siheng Chen,Huangjie Zheng,Jiangchao Yao,Kenan Cui,Ya Zhang,Ivor W. Tsang

DOI: https://doi.org/10.48550/arxiv.1907.09708

2019-01-01

Abstract:Graph structured data provide two-fold information: graph structures and node attributes. Numerous graph-based algorithms rely on both information to achieve success in supervised tasks, such as node classification and link prediction. However, node attributes could be missing or incomplete, which significantly deteriorates the performance. The task of node attribute generation aims to generate attributes for those nodes whose attributes are completely unobserved. This task benefits many real-world problems like profiling, node classification and graph data augmentation. To tackle this task, we propose a deep adversarial learning based method to generate node attributes; called node attribute neural generator (NANG). NANG learns a unifying latent representation which is shared by both node attributes and graph structures and can be translated to different modalities. We thus use this latent representation as a bridge to convert information from one modality to another. We further introduce practical applications to quantify the performance of node attribute generation. Extensive experiments are conducted on four real-world datasets and the empirical results show that node attributes generated by the proposed method are high-qualitative and beneficial to other applications. The datasets and codes are available online.
ForGen: Autoregressive Generation of Sparse Graphs with Preferential Forest.

Yao Shi,Yu Liu,Lei Zou

DOI: https://doi.org/10.1007/978-3-031-25158-0_40

2023-01-01

Abstract:Graph is a natural way to model interactions between objects, such as in the field of biology and social science. Over the past decades, modeling and generating graphs have been a popular research topic, largely inspired by observed properties of real-world graphs. Since traditional approaches relying on hand-crafted mechanisms are only capable of capturing some specific graph properties, recent focus has been shifted to deep neural methods. However, the task is still challenging in terms of efficiency. To address this issue, we observe that the connectivity (i.e., degree) of nodes follows scale-free distribution for most real-world graphs, which can be utilized to accelerate the generation process. We propose ForGen, a Forest-based Generation model that contains a graph-level and an edge-level autoregressive generator. Specifically, for the edge-level model, motivated by the skewed distribution of node degree and the Huffman tree, we design a forest-like data structure to accelerate edge connection via shallow tree searches and better parallelism. Experiments on both synthetic and real-world graph datasets show that ForGen is two times faster than the current state-of-the-art method for graph generation, and guarantees better generated graph quality.
Feature-enriched hyperbolic network geometry

Roya Aliakbarisani,M. Ángeles Serrano,Marián Boguñá

2023-11-24

Abstract:Graph-structured data provide a comprehensive description of complex systems, encompassing not only the interactions among nodes but also the intrinsic features that characterize these nodes. These features play a fundamental role in the formation of links within the network, making them valuable for extracting meaningful topological information. Notably, features are at the core of deep learning techniques such as Graph Convolutional Neural Networks (GCNs) and offer great utility in tasks like node classification, link prediction, and graph clustering. In this paper, we present a comprehensive framework that treats features as tangible entities and establishes a bipartite graph connecting nodes and features. By assuming that nodes sharing similarities should also share features, we introduce a hyperbolic geometric space where both nodes and features coexist, shaping the structure of both the node network and the bipartite network of nodes and features. Through this framework, we can identify correlations between nodes and features in real data and generate synthetic datasets that mimic the topological properties of their connectivity patterns. The approach provides insights into the inner workings of GCNs by revealing the intricate structure of the data.

Physics and Society
Feature Expansion for Graph Neural Networks

Jiaqi Sun,Lin Zhang,Guangyi Chen,Peng XU,Kun Zhang,Yujiu Yang

2023-01-01

Abstract:Graph neural networks aim to learn representations for graph-structured data and show impressive performance in node classification. Recently, many methods have studied the representations of GNNs from the perspective of optimization goals and spectral graph theory. However, the feature space that dominates representation learning has not been systematically studied in graph neural networks. In this paper, we propose to fill this gap by analyzing the feature space of both spatial and spectral models. We decompose graph neural networks into determined feature spaces and trainable weights, providing the convenience of studying the feature space explicitly using matrix space analysis. In particular, we find theoretically that the feature space tends to be linearly correlated due to repeated aggregations. In this case, the feature space is bounded by the poor representation of shared weights or the limited dimensionality of node attributes in existing models, leading to poor performance. Motivated by these findings, we propose 1) feature subspaces flattening and 2) structural principal components to expand the feature space. Extensive experiments verify the effectiveness of our proposed more comprehensive feature space, with comparable inference time to the baseline, and demonstrate its efficient convergence capability.
Dense Graphs with Scale-Free Feature.

Fei Ma,Xiaomin Wang,Ping Wang,Xudong Luo

DOI: https://doi.org/10.1103/physreve.101.052317

2019-01-01

Abstract:While previous works have shown that an overwhelming number of scale-freenetworks are sparse, there still exist some real-world networks includingsocial networks, urban networks, information networks, which are by observationdense. In this paper, we propose a novel framework for generating scale-freegraphs with dense feature using two simple yet helpful operations, first-ordersubdivision and Line-operation. From the theoretical point of view, our methodcan be used not only to produce desired scale-free graphs with density feature,i.e. power-law exponent γ falling into the interval 1<γ≤2, butalso to establish many other unexpected networked models, for instance,power-law models having large diameter. In addition, the networked modelsgenerated upon our framework show especially assortative structure. That is,their own Pearson correlation coefficients are able to achieve the theoreticalupper bound. Last but not the least, we find the sizes of community in theproposed models to follow power-law in form with respect to modularitymaximization.
Feature-Based Classification of Networks

Ian Barnett,Nishant Malik,Marieke L. Kuijjer,Peter J. Mucha,Jukka-Pekka Onnela

DOI: https://doi.org/10.48550/arXiv.1610.05868

2016-10-19

Abstract:Network representations of systems from various scientific and societal domains are neither completely random nor fully regular, but instead appear to contain recurring structural building blocks. These features tend to be shared by networks belonging to the same broad class, such as the class of social networks or the class of biological networks. At a finer scale of classification within each such class, networks describing more similar systems tend to have more similar features. This occurs presumably because networks representing similar purposes or constructions would be expected to be generated by a shared set of domain specific mechanisms, and it should therefore be possible to classify these networks into categories based on their features at various structural levels. Here we describe and demonstrate a new, hybrid approach that combines manual selection of features of potential interest with existing automated classification methods. In particular, selecting well-known and well-studied features that have been used throughout social network analysis and network science and then classifying with methods such as random forests that are of special utility in the presence of feature collinearity, we find that we achieve higher accuracy, in shorter computation time, with greater interpretability of the network classification results.

Social and Information Networks,Physics and Society,Methodology
Learning Features of Network Structures Using Graphlets.

Kun Tu,Jian Li,Don Towsley,Dave Braines,Liam Turner

DOI: https://doi.org/10.48550/arxiv.1812.05473

2018-01-01

Abstract:Networks are fundamental to the study of complex systems, ranging from social contacts, message transactions, to biological regulations and economical networks. In many realistic applications, these networks may vary over time. Modeling and analyzing such temporal properties is of additional interest as it can provide a richer characterization of relations between nodes in networks. In this paper, we explore the role of \emph{graphlets} in network classification for both static and temporal networks. Graphlets are small non-isomorphic induced subgraphs representing connected patterns in a network and their frequency can be used to assess network structures. We show that graphlet features, which are not captured by state-of-the-art methods, play a significant role in enhancing the performance of network classification. To that end, we propose two novel graphlet-based techniques, \emph{gl2vec} for network embedding, and \emph{gl-DCNN} for diffusion-convolutional neural networks. We demonstrate the efficacy and usability of \emph{gl2vec} and \emph{gl-DCNN} through extensive experiments using several real-world static and temporal networks. We find that features learned from graphlets can bring notable performance increases to state-of-the-art methods in network analysis.
Generative Model for Node Generation.

Boyu Zhang,Xin Wang,Kai Liu

DOI: https://doi.org/10.1145/3404555.3404599

2020-01-01

Abstract:We present a generative model applied to graph-structured data for node generation by incorporating the graph convolutional architecture and semi-supervised learning with variational auto-encoder. This idea is motivated by successful applications of deep generative models for images and speeches. However, when applied to graph-structured data, especially social network data, existing deep generative models usually don't work: these models can not learn underlying distributions of social network data effectively. In order to address this problem, we construct a deep generative model, using architectures and techniques that prove to be effective for modelling network data in practice. Experimental results show that our model can successfully learn the underlying distribution from the social network dataset, and generate reasonable nodes, which can be altered by varying latent variables. This provides us a way to study social network data in the same way we study image data.
Heterogeneous Graph Generation: A Hierarchical Approach using Node Feature Pooling

Hritaban Ghosh,Chen Changyu,Arunesh Sinha,Shamik Sural

2024-10-16

Abstract:Heterogeneous graphs are present in various domains, such as social networks, recommendation systems, and biological networks. Unlike homogeneous graphs, heterogeneous graphs consist of multiple types of nodes and edges, each representing different entities and relationships. Generating realistic heterogeneous graphs that capture the complex interactions among diverse entities is a difficult task due to several reasons. The generator has to model both the node type distribution along with the feature distribution for each node type. In this paper, we look into solving challenges in heterogeneous graph generation, by employing a two phase hierarchical structure, wherein the first phase creates a skeleton graph with node types using a prior diffusion based model and in the second phase, we use an encoder and a sampler structure as generator to assign node type specific features to the nodes. A discriminator is used to guide training of the generator and feature vectors are sampled from a node feature pool. We conduct extensive experiments with subsets of IMDB and DBLP datasets to show the effectiveness of our method and also the need for various architecture components.

Social and Information Networks,Machine Learning
Attributed Graph Force Learning

Ke Sun,Feng Xia,Jiaying Liu,Bo Xu,Vidya Saikrishna,Charu C. Aggarwal

DOI: https://doi.org/10.1109/tnnls.2022.3221100

IF: 14.255

2022-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:In numerous network analysis tasks, feature representation plays an imperative role. Due to the intrinsic nature of networks being discrete, enormous challenges are imposed on their effective usage. There has been a significant amount of attention on network feature learning in recent times that has the potential of mapping discrete features into a continuous feature space. The methods, however, lack preserving the structural information owing to the utilization of random negative sampling during the training phase. The ability to effectively join attribute information to embedding feature space is also compromised. To address the shortcomings identified, a novel attribute force-based graph (AGForce) learning model is proposed that keeps the structural information intact along with adaptively joining attribute information to the node's features. To demonstrate the effectiveness of the proposed framework, comprehensive experiments on benchmark datasets are performed. AGForce based on the spring-electrical model extends opportunities to simulate node interaction for graph learning.

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture
Feature-rich networks: going beyond complex network topologies

Roberto Interdonato,Martin Atzmueller,Sabrina Gaito,Rushed Kanawati,Christine Largeron,Alessandra Sala

DOI: https://doi.org/10.1007/s41109-019-0111-x

2019-01-31

Applied Network Science

Abstract:The growing availability of multirelational data gives rise to an opportunity for novel characterization of complex real-world relations, supporting the proliferation of diverse network models such as Attributed Graphs, Heterogeneous Networks, Multilayer Networks, Temporal Networks, Location-aware Networks, Knowledge Networks, Probabilistic Networks, and many other task-driven and data-driven models. In this paper, we propose an overview of these models and their main applications, described under the common denomination of Feature-rich Networks, i. e. models where the expressive power of the network topology is enhanced by exposing one or more peculiar features. The aim is also to sketch a scenario that can inspire the design of novel feature-rich network models, which in turn can support innovative methods able to exploit the full potential of mining complex network structures in domain-specific applications.
Feature Selection and Extraction for Graph Neural Networks

Deepak Bhaskar Acharya,Huaming Zhang

DOI: https://doi.org/10.1145/3374135.3385309

2020-04-02

Abstract:Graph Neural Networks (GNNs) have been a latest hot research topic in data science, due to the fact that they use the ubiquitous data structure graphs as the underlying elements for constructing and training neural networks. In a GNN, each node has numerous features associated with it. The entire task (for example, classification, or clustering) utilizes the features of the nodes to make decisions, at node level or graph level. In this paper, (1) we extend the feature selection algorithm presented in via Gumbel Softmax to GNNs. We conduct a series of experiments on our feature selection algorithms, using various benchmark datasets: Cora, Citeseer and Pubmed. (2) We implement a mechanism to rank the extracted features. We demonstrate the effectiveness of our algorithms, for both feature selection and ranking. For the Cora dataset, (1) we use the algorithm to select 225 features out of 1433 features. Our experimental results demonstrate their effectiveness for the same classification problem. (2) We extract features such that they are linear combinations of the original features, where the coefficients for extracted features are non-negative and sum up to one. We propose an algorithm to rank the extracted features in the sense that when using them for the same classification problem, the accuracy goes down gradually for the extracted features within the rank 1 - 50, 51 - 100, 100 - 150, and 151 - 200.
Learning from Networks

Xiao Huang,Peng Cui,Yuxiao Dong,Jundong Li,Huan Liu,Jian Pei,Le Song,Jie Tang,Fei Wang,Hongxia Yang,Wenwu Zhu

DOI: https://doi.org/10.1145/3292500.3332293

2019-01-01

Abstract:Arguably, every entity in this universe is networked in one wayr another. With the prevalence of network data collected, such as social media and biological networks, learning from networks has become an essential task in many applications. It is well recognized that network data is intricate and large-scale, and analytic tasks on network data become more and more sophisticated. In this tutorial, we systematically review the area of learning from networks, including algorithms, theoretical analysis, and illustrative applications. Starting with a quick recollection of the exciting history of the area, we formulate the core technical problems. Then, we introduce the fundamental approaches, that is, the feature selection based approaches and the network embedding based approaches. Next, we extend our discussion to attributed networks, which are popular in practice. Last, we cover the latest hot topic, graph neural based approaches. For each group of approaches, we also survey the associated theoretical analysis and real-world application examples. Our tutorial also inspires a series of open problems and challenges that may lead to future breakthroughs. The authors are productive and seasoned researchers active in this area who represent a nice combination of academia and industry.
GraphGAN: Graph Representation Learning with Generative Adversarial Nets

Hongwei Wang,Jialin Wang,Jia Wang,Miao Zhao,Weinan Zhang,Fuzheng Zhang,Wenjie Li,Xing Xie,Minyi Guo

DOI: https://doi.org/10.1109/tkde.2019.2961882

2017-01-01

Abstract:The goal of graph representation learning is to embed each vertex in a graph into a low-dimensional vector space. Existing graph representation learning methods can be classified into two categories: generative models that learn the underlying connectivity distribution in the graph, and discriminative models that predict the probability of edge existence between a pair of vertices. In this paper, we propose Graph-GAN, an innovative graph representation learning framework unifying above two classes of methods, in which the generative model and discriminative model play a game-theoretical minimax game. Specifically, for a given vertex, the generative model tries to fit its underlying true connectivity distribution over all other vertices and produces "fake" samples to fool the discriminative model, while the discriminative model tries to detect whether the sampled vertex is from ground truth or generated by the generative model. With the competition between these two models, both of them can alternately and iteratively boost their performance. Moreover, when considering the implementation of generative model, we propose a novel graph softmax to overcome the limitations of traditional softmax function, which can be proven satisfying desirable properties of normalization, graph structure awareness, and computational efficiency. Through extensive experiments on real-world datasets, we demonstrate that Graph-GAN achieves substantial gains in a variety of applications, including link prediction, node classification, and recommendation, over state-of-the-art baselines.
Graph Representation Learning: Foundations, Methods, Applications and Systems

Wei Jin,Yao Ma,Yiqi Wang,Xiaorui Liu,Jiliang Tang,Yukuo Cen,Jiezhong Qiu,Jie Tang,Chuan Shi,Yanfang Ye,Jiawei Zhang,Philip S. Yu

DOI: https://doi.org/10.1145/3447548.3470824

2021-01-01

Abstract:ABSTRACTGraphs such as social networks and molecular graphs are ubiquitous data structures in the real world. Due to their prevalence, it is of great research importance to extract meaningful patterns from graph structured data so that downstream tasks can be facilitated. Instead of designing hand-engineered features, graph representation learning has emerged to learn representations that can encode the abundant information about the graph. It has achieved tremendous success in various tasks such as node classification, link prediction, and graph classification and has attracted increasing attention in recent years. In this tutorial, we systematically review the foundations, techniques, applications and advances in graph representation learning. We first introduce the foundations on graph theory and graph Fourier analysis. We then cover the key achievements of graph representation learning in recent years. Concretely, we discuss the six topics: 1) network embedding theories and systems; 2) foundations of graph neural networks (GNNs); 3) CogDL toolkit for GNNs; 4) scalable GNNs; 5) self-supervised learning in GNNs and 6) heterogeneous graphs and heterogeneous GNNs. Finally, we will introduce the applications of graph representation learning with a focus on recommender systems.
Graph Feature Gating Networks

Wei Jin,Xiaorui Liu,Yao Ma,Tyler Derr,Charu Aggarwal,Jiliang Tang

DOI: https://doi.org/10.48550/arXiv.2105.04493

2021-05-11

Abstract:Graph neural networks (GNNs) have received tremendous attention due to their power in learning effective representations for graphs. Most GNNs follow a message-passing scheme where the node representations are updated by aggregating and transforming the information from the neighborhood. Meanwhile, they adopt the same strategy in aggregating the information from different feature dimensions. However, suggested by social dimension theory and spectral embedding, there are potential benefits to treat the dimensions differently during the aggregation process. In this work, we investigate to enable heterogeneous contributions of feature dimensions in GNNs. In particular, we propose a general graph feature gating network (GFGN) based on the graph signal denoising problem and then correspondingly introduce three graph filters under GFGN to allow different levels of contributions from feature dimensions. Extensive experiments on various real-world datasets demonstrate the effectiveness and robustness of the proposed frameworks.

Machine Learning,Artificial Intelligence
Recent advances on mechanisms of network generation: Community, exchangeability, and scale-free properties

Zhengpin Li,Yanxi Hou,Tiandong Wang

DOI: https://doi.org/10.1002/wics.1651

2024-01-01

Abstract:The mechanisms of network generation have undergone extensive analysis and found broad applications in various real-world scenarios. Among the fruitful literature on network models, numerous studies seek to explore and interpret fundamental graph structure properties, including the clustering effect, exchangeability, and scale-free properties. In this paper, we present a comprehensive review of the statistical modeling methods for the mechanisms of network generation. We specifically focus on three representative classes of models, namely the stochastic block models, the exchangeable network models, and the preferential attachment models. For each model type, our approach begins by reviewing existing methods and model setups, followed by an exploration of the core modeling principles behind them. We also summarize relevant statistical inference techniques and provide a unified understanding of theoretical analyses. Furthermore, we emphasize several challenges and open problems that could shed light on future research. We conclude this review with the identification of some possible directions for future study. This article is categorized under:<br /> Statistical and Graphical Methods of Data Analysis > Modeling Methods and Algorithms Algorithms and Computational Methods > Networks and Security

Feature generation for graphs and networks

Defragging Subgraph Features for Graph Classification.

Incremental Subgraph Feature Selection for Graph Classification.

Feature Network Methods in Machine Learning and Applications

Node Attribute Generation on Graphs

ForGen: Autoregressive Generation of Sparse Graphs with Preferential Forest.

Feature-enriched hyperbolic network geometry

Feature Expansion for Graph Neural Networks

Dense Graphs with Scale-Free Feature.

Feature-Based Classification of Networks

Learning Features of Network Structures Using Graphlets.

Generative Model for Node Generation.

Heterogeneous Graph Generation: A Hierarchical Approach using Node Feature Pooling

Attributed Graph Force Learning

Feature-rich networks: going beyond complex network topologies

Feature Selection and Extraction for Graph Neural Networks

Learning from Networks

GraphGAN: Graph Representation Learning with Generative Adversarial Nets

Graph Representation Learning: Foundations, Methods, Applications and Systems

Graph Feature Gating Networks

Recent advances on mechanisms of network generation: Community, exchangeability, and scale-free properties