Analyzing a nail-biting election

D. Saari

DOI: https://doi.org/10.1007/s003550100136

2001-07-12

Social Choice and Welfare

Abstract:

What problem does this paper attempt to address?

Flexible Attributed Network Embedding.

Enya Shen,Zhidong Cao,Changqing Zou,Jianmin Wang

DOI: https://doi.org/10.48550/arxiv.1811.10789

2018-01-01

Abstract:Network embedding aims to find a way to encode network by learning an embedding vector for each node in the network. The network often has property information which is highly informative with respect to the node's position and role in the network. Most network embedding methods fail to utilize this information during network representation learning. In this paper, we propose a novel framework, FANE, to integrate structure and property information in the network embedding process. In FANE, we design a network to unify heterogeneity of the two information sources, and define a new random walking strategy to leverage property information and make the two information compensate. FANE is conceptually simple and empirically powerful. It improves over the state-of-the-art methods on Cora dataset classification task by over 5%, more than 10% on WebKB dataset classification task. Experiments also show that the results improve more than the state-of-the-art methods as increasing training size. Moreover, qualitative visualization show that our framework is helpful in network property information exploration. In all, we present a new way for efficiently learning state-of-the-art task-independent representations in complex attributed networks. The source code and datasets of this paper can be obtained from this https URL.
Discrete Embedding for Attributed Graphs

Hong Yang,Ling Chen,Shirui Pan,Haishuai Wang,Peng Zhang

DOI: https://doi.org/10.1016/j.patcog.2021.108368

IF: 8

2022-01-01

Pattern Recognition

Abstract:Attributed graphs refer to graphs where both node links and node attributes are observable for analysis. Attributed graph embedding enables joint representation learning of node links and node attributes. Different from classical graph embedding methods such as Deepwalk and node2vec that first project node links into low-dimensional vectors which are then linearly concatenated with node attribute vectors as node representation, attributed graph embedding fully explores data dependence between node links and attributes by either using node attributes as class labels to supervise structure learning from node links, or reversely using node links to supervise the learning from node attributes. However, existing attributed graph embedding models are designed in continuous Euclidean spaces which often introduce data redundancy and impose challenges to storage and computation costs. In this paper, we study a new problem of discrete embedding for attributed graphs that can learn succinct node representations. Specifically, we present a Binarized Attributed Network Embedding model (BANE for short) to learn binary node representation by factorizing a Weisfeiler-Lehman proximity matrix under the constraint of binary node representation. Furthermore, based on BANE, we propose a new Low-bit Quantization for Attributed Network Representation learning model (LQANR for short) to learn even more compact node representation of bit-width values. Theoretical analysis and empirical studies on real-world datasets show that the new discrete embedding models outperform benchmark methods.
Path-LLM: A Shortest-Path-based LLM Learning for Unified Graph Representation

Wenbo Shang,Xuliang Zhu,Xin Huang

2024-08-10

Abstract:Unified graph representation learning aims to produce node embeddings, which can be applied to multiple downstream applications. However, existing studies based on graph neural networks and language models either suffer from the limitations of numerous training needed toward specific downstream predictions or have shallow semantic features. In this work, we propose a novel Path-LLM model to learn unified graph representation, which leverages a powerful large language model (LLM) to incorporate our proposed path features. Our Path-LLM framework consists of several well-designed techniques. First, we develop a new mechanism of long-to-short shortest path (L2SP) selection, which covers essential connections between different dense groups. An in-depth comparison of different path selection plans is offered to illustrate the strength of our designed L2SP. Then, we design path textualization to obtain L2SP-based training texts. Next, we feed the texts into a self-supervised LLM training process to learn embeddings. Extensive experiments on benchmarks validate the superiority of Path-LLM against the state-of-the-art WalkLM method on two classical graph learning tasks (node classification and link prediction) and one NP-hard graph query processing task (keyword search), meanwhile saving more than 90% of training paths.

Computation and Language
Large Language Models as Topological Structure Enhancers for Text-Attributed Graphs

Shengyin Sun,Yuxiang Ren,Chen Ma,Xuecang Zhang

2024-07-24

Abstract:The latest advancements in large language models (LLMs) have revolutionized the field of natural language processing (NLP). Inspired by the success of LLMs in NLP tasks, some recent work has begun investigating the potential of applying LLMs in graph learning tasks. However, most of the existing work focuses on utilizing LLMs as powerful node feature augmenters, leaving employing LLMs to enhance graph topological structures an understudied problem. In this work, we explore how to leverage the information retrieval and text generation capabilities of LLMs to refine/enhance the topological structure of text-attributed graphs (TAGs) under the node classification setting. First, we propose using LLMs to help remove unreliable edges and add reliable ones in the TAG. Specifically, we first let the LLM output the semantic similarity between node attributes through delicate prompt designs, and then perform edge deletion and edge addition based on the similarity. Second, we propose using pseudo-labels generated by the LLM to improve graph topology, that is, we introduce the pseudo-label propagation as a regularization to guide the graph neural network (GNN) in learning proper edge weights. Finally, we incorporate the two aforementioned LLM-based methods for graph topological refinement into the process of GNN training, and perform extensive experiments on four real-world datasets. The experimental results demonstrate the effectiveness of LLM-based graph topology refinement (achieving a 0.15%--2.47% performance gain on public benchmarks).

Artificial Intelligence,Computation and Language,Machine Learning
Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs

Zhikai Chen,Haitao Mao,Hang Li,Wei Jin,Hongzhi Wen,Xiaochi Wei,Shuaiqiang Wang,Dawei Yin,Wenqi Fan,Hui Liu,Jiliang Tang

2024-01-17

Abstract:Learning on Graphs has attracted immense attention due to its wide real-world applications. The most popular pipeline for learning on graphs with textual node attributes primarily relies on Graph Neural Networks (GNNs), and utilizes shallow text embedding as initial node representations, which has limitations in general knowledge and profound semantic understanding. In recent years, Large Language Models (LLMs) have been proven to possess extensive common knowledge and powerful semantic comprehension abilities that have revolutionized existing workflows to handle text data. In this paper, we aim to explore the potential of LLMs in graph machine learning, especially the node classification task, and investigate two possible pipelines: LLMs-as-Enhancers and LLMs-as-Predictors. The former leverages LLMs to enhance nodes' text attributes with their massive knowledge and then generate predictions through GNNs. The latter attempts to directly employ LLMs as standalone predictors. We conduct comprehensive and systematical studies on these two pipelines under various settings. From comprehensive empirical results, we make original observations and find new insights that open new possibilities and suggest promising directions to leverage LLMs for learning on graphs. Our codes and datasets are available at <a class="link-external link-https" href="https://github.com/CurryTang/Graph-LLM" rel="external noopener nofollow">this https URL</a>.

Machine Learning,Artificial Intelligence
Attributed Graph Embedding with Random Walk Regularization and Centrality-Based Attention

Yuxuan Yang,Beibei Han,Zanxi Ran,Min Gao,Yingmei Wei

DOI: https://doi.org/10.3390/math11081830

IF: 2.4

2023-04-13

Mathematics

Abstract:Graph-embedding learning is the foundation of complex information network analysis, aiming to represent nodes in a graph network as low-dimensional dense real-valued vectors for the application in practical analysis tasks. In recent years, the study of graph network representation learning has received increasing attention from researchers, and, among them, graph neural networks (GNNs) based on deep learning are playing an increasingly important role in this field. However, the fact that higher-order neighborhood information cannot be used effectively is a problem of most existing graph neural networks. Moreover, it tends to ignore the influence of latent representation and structural properties on graph embedding. In hopes of solving these issues, we introduce centrality encoding to learn the node properties, add an attention mechanism consideration to better distinguish the significance of neighboring nodes, and introduce random walk regularization to make sample neighbors that consistently satisfy predetermined criteria. This allows us to learn a representation of a potential node. We tested the performance of our model on node-clustering and link prediction tasks using three widely recognized benchmark datasets. The outcomes of our experiments demonstrate that our model significantly surpasses the baseline method in both tasks, indicating that the graph embedding it generates is highly expressive.

mathematics
NT-LLM: A Novel Node Tokenizer for Integrating Graph Structure into Large Language Models

Yanbiao Ji,Chang Liu,Xin Chen,Yue Ding,Dan Luo,Mei Li,Wenqing Lin,Hongtao Lu

2024-10-15

Abstract:Graphs are a fundamental data structure for representing relationships in real-world scenarios. With the success of Large Language Models (LLMs) across various natural language processing (NLP) tasks, there has been growing interest in integrating LLMs for graph learning. However, applying LLMs to graph-related tasks poses significant challenges, as these models are not inherently designed to capture the complex structural information present in graphs. Existing approaches address this challenge through two strategies: the chain of tasks approach, which uses Graph Neural Networks (GNNs) to encode the graph structure so that LLMs are relieved from understanding spatial positions; and Graph-to-Text Conversion, which translates graph structures into semantic text representations that LLMs can process. Despite their progress, these methods often struggle to fully preserve the topological information of graphs or require extensive computational resources, limiting their practical applicability. In this work, we introduce Node Tokenizer for Large Language Models (NT-LLM), a novel framework that efficiently encodes graph structures by selecting key nodes as anchors and representing each node based on its relative distance to these anchors. This position-anchored encoding effectively captures the graph topology, enabling enhanced reasoning capabilities in LLMs over graph data. Additionally, we implement a task-specific tuning procedure to further improve structural understanding within LLMs. Through extensive empirical evaluations, NT-LLM demonstrates significant performance improvements across a variety of graph-related tasks.

Artificial Intelligence
Bridging Large Language Models and Graph Structure Learning Models for Robust Representation Learning

Guangxin Su,Yifan Zhu,Wenjie Zhang,Hanchen Wang,Ying Zhang

2024-10-16

Abstract:Graph representation learning, involving both node features and graph structures, is crucial for real-world applications but often encounters pervasive noise. State-of-the-art methods typically address noise by focusing separately on node features with large language models (LLMs) and on graph structures with graph structure learning models (GSLMs). In this paper, we introduce LangGSL, a robust framework that integrates the complementary strengths of pre-trained language models and GSLMs to jointly enhance both node feature and graph structure learning. In LangGSL, we first leverage LLMs to filter noise in the raw data and extract valuable cleaned information as features, enhancing the synergy of downstream models. During the mutual learning phase in LangGSL, the core idea is to leverage the relatively small language model (LM) to process local attributes and generate reliable pseudo-labels and informative node embeddings, which are then integrated into the GSLM's prediction phase. This approach enriches the global context and enhances overall performance. Meanwhile, GSLM refines the evolving graph structure constructed from the LM's output, offering updated labels back to the LM as additional guidance, thus facilitating a more effective mutual learning process. The LM and GSLM work synergistically, complementing each other's strengths and offsetting weaknesses within a variational information-maximizing framework, resulting in enhanced node features and a more robust graph structure. Extensive experiments on diverse graph datasets of varying scales and across different task scenarios demonstrate the scalability and effectiveness of the proposed approach.

Machine Learning,Artificial Intelligence
Leveraging Large Language Models for Node Generation in Few-Shot Learning on Text-Attributed Graphs

Jianxiang Yu,Yuxiang Ren,Chenghua Gong,Jiaqi Tan,Xiang Li,Xuecang Zhang

2024-12-11

Abstract:Text-attributed graphs have recently garnered significant attention due to their wide range of applications in web domains. Existing methodologies employ word embedding models for acquiring text representations as node features, which are subsequently fed into Graph Neural Networks (GNNs) for training. Recently, the advent of Large Language Models (LLMs) has introduced their powerful capabilities in information retrieval and text generation, which can greatly enhance the text attributes of graph data. Furthermore, the acquisition and labeling of extensive datasets are both costly and time-consuming endeavors. Consequently, few-shot learning has emerged as a crucial problem in the context of graph learning tasks. In order to tackle this challenge, we propose a lightweight paradigm called LLM4NG, which adopts a plug-and-play approach to empower text-attributed graphs through node generation using LLMs. Specifically, we utilize LLMs to extract semantic information from the labels and generate samples that belong to these categories as exemplars. Subsequently, we employ an edge predictor to capture the structural information inherent in the raw dataset and integrate the newly generated samples into the original graph. This approach harnesses LLMs for enhancing class-level information and seamlessly introduces labeled nodes and edges without modifying the raw dataset, thereby facilitating the node classification task in few-shot scenarios. Extensive experiments demonstrate the outstanding performance of our proposed paradigm, particularly in low-shot scenarios. For instance, in the 1-shot setting of the ogbn-arxiv dataset, LLM4NG achieves a 76% improvement over the baseline model.

Machine Learning
Network Embedding As Matrix Factorization: Unifying DeepWalk, LINE, PTE, and Node2vec

Jiezhong Qiu,Yuxiao Dong,Hao Ma,Jian Li,Kuansan Wang,Jie Tang

2017-01-01

Abstract:Since the invention of word2vec, the skip-gram model has significantly advanced the research of network embedding, such as the recent emergence of the DeepWalk, LINE, PTE, and node2vec approaches. In this work, we show that all of the aforementioned models with negative sampling can be unified into the matrix factorization framework with closed forms. Our analysis and proofs reveal that: (1) DeepWalk empirically produces a low-rank transformation of a network's normalized Laplacian matrix; (2) LINE, in theory, is a special case of DeepWalk when the size of vertices' context is set to one; (3) As an extension of LINE, PTE can be viewed as the joint factorization of multiple networks' Laplacians; (4) node2vec is factorizing a matrix related to the stationary distribution and transition probability tensor of a 2nd-order random walk. We further provide the theoretical connections between skip-gram based network embedding algorithms and the theory of graph Laplacian. Finally, we present the NetMF method as well as its approximation algorithm for computing network embedding. Our method offers significant improvements over DeepWalk and LINE for conventional network mining tasks. This work lays the theoretical foundation for skip-gram based network embedding methods, leading to a better understanding of latent network representation learning.
Exploring the Potential of Large Language Models for Heterophilic Graphs

Yuxia Wu,Shujie Li,Yuan Fang,Chuan Shi

2024-10-21

Abstract:Large language models (LLMs) have presented significant opportunities to enhance various machine learning applications, including graph neural networks (GNNs). By leveraging the vast open-world knowledge within LLMs, we can more effectively interpret and utilize textual data to better characterize heterophilic graphs, where neighboring nodes often have different labels. However, existing approaches for heterophilic graphs overlook the rich textual data associated with nodes, which could unlock deeper insights into their heterophilic contexts. In this work, we explore the potential of LLMs for modeling heterophilic graphs and propose a novel two-stage framework: LLM-enhanced edge discriminator and LLM-guided edge reweighting. In the first stage, we fine-tune the LLM to better identify homophilic and heterophilic edges based on the textual content of their nodes. In the second stage, we adaptively manage message propagation in GNNs for different edge types based on node features, structures, and heterophilic or homophilic characteristics. To cope with the computational demands when deploying LLMs in practical scenarios, we further explore model distillation techniques to fine-tune smaller, more efficient models that maintain competitive performance. Extensive experiments validate the effectiveness of our framework, demonstrating the feasibility of using LLMs to enhance node classification on heterophilic graphs.

Machine Learning,Artificial Intelligence,Computation and Language,Social and Information Networks
Representation Learning of Enhanced Graphs Using Random Walk Graph Convolutional Network

Xing Li,Wei,Ruizhi Zhang,Zhenyu Shi,Zhiming Zheng,Xiangnan Feng

DOI: https://doi.org/10.1145/3582841

IF: 5

2023-01-01

ACM Transactions on Intelligent Systems and Technology

Abstract:Nowadays, graph structure data has played a key role in machine learning because of its simple topological structure, and therefore, the graph representation learning methods have attracted great attention. And it turns out that the low-dimensional embedding representation obtained by graph representation learning is extremely useful in various typical tasks, such as node classification and content recommendation. However, most of the existing methods do not further dig out potential structural information on the original graph structure. Here, we propose wGCN, which utilizes random walk to obtain the node-specific mesoscopic structures (high-order local structure) of the graph and utilizes these mesoscopic structures to enhance the graph and organize the characteristic information of the nodes. Our method can effectively generate node embedding for data of previously unknown categories, which has been proven in a series of experiments conducted on many types of graph networks. And compared to baselines, our method shows the best performance on most datasets and achieves competitive results on others. It is believed that combining the mesoscopic structure to further explore the structural information of the graph will greatly improve the learning efficiency of the graph neural network.
STAGE: Simplified Text-Attributed Graph Embeddings Using Pre-trained LLMs

Aaron Zolnai-Lucas,Jack Boylan,Chris Hokamp,Parsa Ghaffari

2024-07-10

Abstract:We present Simplified Text-Attributed Graph Embeddings (STAGE), a straightforward yet effective method for enhancing node features in Graph Neural Network (GNN) models that encode Text-Attributed Graphs (TAGs). Our approach leverages Large-Language Models (LLMs) to generate embeddings for textual attributes. STAGE achieves competitive results on various node classification benchmarks while also maintaining a simplicity in implementation relative to current state-of-the-art (SoTA) techniques. We show that utilizing pre-trained LLMs as embedding generators provides robust features for ensemble GNN training, enabling pipelines that are simpler than current SoTA approaches which require multiple expensive training and prompting stages. We also implement diffusion-pattern GNNs in an effort to make this pipeline scalable to graphs beyond academic benchmarks.

Computation and Language,Artificial Intelligence
LOGIN: A Large Language Model Consulted Graph Neural Network Training Framework

Yiran Qiao,Xiang Ao,Yang Liu,Jiarong Xu,Xiaoqian Sun,Qing He

2024-06-06

Abstract:Recent prevailing works on graph machine learning typically follow a similar methodology that involves designing advanced variants of graph neural networks (GNNs) to maintain the superior performance of GNNs on different graphs. In this paper, we aim to streamline the GNN design process and leverage the advantages of Large Language Models (LLMs) to improve the performance of GNNs on downstream tasks. We formulate a new paradigm, coined "LLMs-as-Consultants," which integrates LLMs with GNNs in an interactive manner. A framework named LOGIN (LLM Consulted GNN training) is instantiated, empowering the interactive utilization of LLMs within the GNN training process. First, we attentively craft concise prompts for spotted nodes, carrying comprehensive semantic and topological information, and serving as input to LLMs. Second, we refine GNNs by devising a complementary coping mechanism that utilizes the responses from LLMs, depending on their correctness. We empirically evaluate the effectiveness of LOGIN on node classification tasks across both homophilic and heterophilic graphs. The results illustrate that even basic GNN architectures, when employed within the proposed LLMs-as-Consultants paradigm, can achieve comparable performance to advanced GNNs with intricate designs. Our codes are available at <a class="link-external link-https" href="https://github.com/QiaoYRan/LOGIN" rel="external noopener nofollow">this https URL</a>.

Machine Learning,Artificial Intelligence
Disentangled Representation Learning with Large Language Models for Text-Attributed Graphs

Yijian Qin,Xin Wang,Ziwei Zhang,Wenwu Zhu

2024-03-10

Abstract:Text-attributed graphs (TAGs) are prevalent on the web and research over TAGs such as citation networks, e-commerce networks and social networks has attracted considerable attention in the web community. Recently, large language models (LLMs) have demonstrated exceptional capabilities across a wide range of tasks. However, the existing works focus on harnessing the potential of LLMs solely relying on prompts to convey graph structure information to LLMs, thus suffering from insufficient understanding of the complex structural relationships within TAGs. To address this problem, in this paper we present the Disentangled Graph-Text Learner (DGTL) model, which is able to enhance the reasoning and predicting capabilities of LLMs for TAGs. Our proposed DGTL model incorporates graph structure information through tailored disentangled graph neural network (GNN) layers, enabling LLMs to capture the intricate relationships hidden in text-attributed graphs from multiple structural factors. Furthermore, DGTL operates with frozen pre-trained LLMs, reducing computational costs and allowing much more flexibility in combining with different LLM models. Experimental evaluations demonstrate the effectiveness of the proposed DGTL model on achieving superior or comparable performance over state-of-the-art baselines. Additionally, we also demonstrate that our DGTL model can offer natural language explanations for predictions, thereby significantly enhancing model interpretability.

Computation and Language,Machine Learning
Bootstrapping Heterogeneous Graph Representation Learning via Large Language Models: A Generalized Approach

Hang Gao,Chenhao Zhang,Fengge Wu,Junsuo Zhao,Changwen Zheng,Huaping Liu

2024-12-11

Abstract:Graph representation learning methods are highly effective in handling complex non-Euclidean data by capturing intricate relationships and features within graph structures. However, traditional methods face challenges when dealing with heterogeneous graphs that contain various types of nodes and edges due to the diverse sources and complex nature of the data. Existing Heterogeneous Graph Neural Networks (HGNNs) have shown promising results but require prior knowledge of node and edge types and unified node feature formats, which limits their applicability. Recent advancements in graph representation learning using Large Language Models (LLMs) offer new solutions by integrating LLMs' data processing capabilities, enabling the alignment of various graph representations. Nevertheless, these methods often overlook heterogeneous graph data and require extensive preprocessing. To address these limitations, we propose a novel method that leverages the strengths of both LLM and GNN, allowing for the processing of graph data with any format and type of nodes and edges without the need for type information or special preprocessing. Our method employs LLM to automatically summarize and classify different data formats and types, aligns node features, and uses a specialized GNN for targeted learning, thus obtaining effective graph representations for downstream tasks. Theoretical analysis and experimental validation have demonstrated the effectiveness of our method.

Machine Learning,Computation and Language,Social and Information Networks
LLM-Enhanced User-Item Interactions: Leveraging Edge Information for Optimized Recommendations

Xinyuan Wang,Liang Wu,Liangjie Hong,Hao Liu,Yanjie Fu

2024-02-15

Abstract:The extraordinary performance of large language models has not only reshaped the research landscape in the field of NLP but has also demonstrated its exceptional applicative potential in various domains. However, the potential of these models in mining relationships from graph data remains under-explored. Graph neural networks, as a popular research area in recent years, have numerous studies on relationship mining. Yet, current cutting-edge research in graph neural networks has not been effectively integrated with large language models, leading to limited efficiency and capability in graph relationship mining tasks. A primary challenge is the inability of LLMs to deeply exploit the edge information in graphs, which is critical for understanding complex node relationships. This gap limits the potential of LLMs to extract meaningful insights from graph structures, limiting their applicability in more complex graph-based analysis. We focus on how to utilize existing LLMs for mining and understanding relationships in graph data, applying these techniques to recommendation tasks. We propose an innovative framework that combines the strong contextual representation capabilities of LLMs with the relationship extraction and analysis functions of GNNs for mining relationships in graph data. Specifically, we design a new prompt construction framework that integrates relational information of graph data into natural language expressions, aiding LLMs in more intuitively grasping the connectivity information within graph data. Additionally, we introduce graph relationship understanding and analysis functions into LLMs to enhance their focus on connectivity information in graph data. Our evaluation on real-world datasets demonstrates the framework's ability to understand connectivity information in graph data.

Artificial Intelligence,Information Retrieval
DeepWalk: Online Learning of Social Representations

Bryan Perozzi,Rami Al-Rfou,Steven Skiena

DOI: https://doi.org/10.1145/2623330.2623732

2014-06-28

Abstract:We present DeepWalk, a novel approach for learning latent representations of vertices in a network. These latent representations encode social relations in a continuous vector space, which is easily exploited by statistical models. DeepWalk generalizes recent advancements in language modeling and unsupervised feature learning (or deep learning) from sequences of words to graphs. DeepWalk uses local information obtained from truncated random walks to learn latent representations by treating walks as the equivalent of sentences. We demonstrate DeepWalk's latent representations on several multi-label network classification tasks for social networks such as BlogCatalog, Flickr, and YouTube. Our results show that DeepWalk outperforms challenging baselines which are allowed a global view of the network, especially in the presence of missing information. DeepWalk's representations can provide $F_1$ scores up to 10% higher than competing methods when labeled data is sparse. In some experiments, DeepWalk's representations are able to outperform all baseline methods while using 60% less training data. DeepWalk is also scalable. It is an online learning algorithm which builds useful incremental results, and is trivially parallelizable. These qualities make it suitable for a broad class of real world applications such as network classification, and anomaly detection.

Social and Information Networks,Machine Learning
MARU: Meta-context Aware Random Walks for Heterogeneous Network Representation Learning.

Jyun-Yu Jiang,Zeyu Li,Chelsea J-T Ju,Wei Wang

DOI: https://doi.org/10.1145/3340531.3412040

2020-01-01

Abstract:Information networks, such as social and citation networks, are ubiquitous in the real world so that network analysis plays an important role in data mining and knowledge discovery. To alleviate the sparsity problem of network analysis, it is common to capture the network semantics by projecting nodes onto a vector space as network embeddings. Moreover, random walks are usually exploited to efficiently learn node embeddings and preserve network proximity. In addition to proximity structure, heterogeneous networks have more knowledge about the types of nodes. However, to profit from heterogeneous knowledge, most of the existing approaches guide the random walks through predefined meta-paths or specific strategies, which can distort the understanding of network structures. Furthermore, traditional random walk-based approaches much favor the nodes with higher degrees while other nodes are equivalently important for the downstream applications. In this paper, we propose Meta-context Aware Random Walks (MARU) to overcome these challenges, thereby learning richer and more unbiased representations for heterogeneous networks. To reduce the bias in classical random walks, the algorithm of bidirectional extended random walks is introduced to improve the fairness of representation learning. Based on the enhanced random walks, the meta-context aware skip-gram model is then presented to learn robust network embeddings with dynamic meta-contexts. Therefore, MARU can not only fairly understand the overall network structures but also leverage the sophisticated heterogeneous knowledge in the networks. Extensive experiments have been conducted on three real-world large-scale publicly available datasets. The experimental results demonstrate that MARU significantly outperforms state-of-the-art heterogeneous network embedding methods across three general machine learning tasks, including multi-label node classification, node clustering, and link prediction.
Network Embedding for Graphs with Node Attributes

Cheng Yang,Chuan Shi,Zhiyuan Liu,Cunchao Tu,Maosong Sun

DOI: https://doi.org/10.1007/978-3-031-01590-8_3

2021-01-01

Abstract:Most NE methods investigate network structures for learning. In reality, nodes in a graph usually contain rich feature information, which cannot be well applied with typical representation learning methods. Taking text feature as an example, we will introduce text-associated DeepWalk (TADW) model for learning NEs with node attributes in this chapter. Inspired by the proof that DeepWalk, a state-of-the-art network representation method, is actually equivalent to matrix factorization (MF), TADW incorporates text features of vertices into network representation learning under the framework of matrix factorization. We evaluate TADW and various baseline methods by applying them to the task of multi-class classification of vertices. The experimental results show that, TADW outperforms other baselines on all three datasets, especially when networks are noisy and training ratio is small.

Analyzing a nail-biting election

Flexible Attributed Network Embedding.

Discrete Embedding for Attributed Graphs

Path-LLM: A Shortest-Path-based LLM Learning for Unified Graph Representation

Large Language Models as Topological Structure Enhancers for Text-Attributed Graphs

Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs

Attributed Graph Embedding with Random Walk Regularization and Centrality-Based Attention

NT-LLM: A Novel Node Tokenizer for Integrating Graph Structure into Large Language Models

Bridging Large Language Models and Graph Structure Learning Models for Robust Representation Learning

Leveraging Large Language Models for Node Generation in Few-Shot Learning on Text-Attributed Graphs

Network Embedding As Matrix Factorization: Unifying DeepWalk, LINE, PTE, and Node2vec

Exploring the Potential of Large Language Models for Heterophilic Graphs

Representation Learning of Enhanced Graphs Using Random Walk Graph Convolutional Network

STAGE: Simplified Text-Attributed Graph Embeddings Using Pre-trained LLMs

LOGIN: A Large Language Model Consulted Graph Neural Network Training Framework

Disentangled Representation Learning with Large Language Models for Text-Attributed Graphs

Bootstrapping Heterogeneous Graph Representation Learning via Large Language Models: A Generalized Approach

LLM-Enhanced User-Item Interactions: Leveraging Edge Information for Optimized Recommendations

DeepWalk: Online Learning of Social Representations

MARU: Meta-context Aware Random Walks for Heterogeneous Network Representation Learning.

Network Embedding for Graphs with Node Attributes